4

I'm counting words in LaTeX files and intuitively used the command:

$ cat *tex | detex | wc -w

I then immediately noticed he useless use of cat, and did the task again with:

$ detex *tex | wc -w

Being proud of myself for avoiding an extra process, I wanted to check how many precious milliseconds I saved by not using cat. I was then very surprised to see that the version with cat was literally 10 times faster:

$ time cat *tex | detex | wc -w
    5000

real    0m0.013s
user    0m0.005s
sys 0m0.007s
$ time detex *tex | wc -w
    5000

real    0m0.144s
user    0m0.124s
sys 0m0.014s

What is the reason for this massive speed difference? Is cat just so much faster in reading files from a disk than detex?


I generated 5 example "LaTeX" files for benchmarking with:

$ for i in {1..5}; do shuf -n 1000 /usr/share/dict/words > "file${i}.tex"; done
10
  • 5
    One possibility I feel likely is that detex reinitializes some costly setup after each file, while cat assembles them all into a single stream which is detexed only once.
    – LSerni
    Mar 14, 2021 at 22:05
  • 2
    straceing detex (from texlive on Arch Linux) in the two cases shows that, when invoked with arguments, it opens and reads several files (several ls-R, texmf.cnf, texmf, texmf-dist) that are not open when it reads from standard input. Though I have no clue on the reason.
    – fra-san
    Mar 14, 2021 at 23:02
  • What version of detex are you using? This is presumably related to Kpathsea but I don’t reproduce it on Debian 10. Mar 15, 2021 at 5:59
  • You may want to check that the result from the two ways of running detex are identical. Note also that the detex manual says Running LaTeX source without a ``\begin{document}'' through detex may produce errors., so since your "test data" isn't actual TeX or LaTeX input, I would be wary about interpreting any timing results.
    – Kusalananda
    Mar 15, 2021 at 8:40
  • 1
    @Kusalananda The results were the same, $ diff <(cat *tex | detex) <(detex *tex) gives no diff. But I posted an answer where I show that my initial results were just noise.
    – ruohola
    Mar 16, 2021 at 11:45

1 Answer 1

0

The initial results seem to just be an inconsistency.

Installed the latest detex from Homebrew:

$ detex -v

OpenDetex version 2.8.9
https://github.com/pkubowicz/opendetex

Got an example LaTeX file:

$ curl https://raw.githubusercontent.com/latex3/latex3/main/articles/2011-current-state.tex > file.tex

Made 100 copies of the file:

$ for i in {1..100}; do cp file.tex "file${i}.tex"; done

Ran the cat version 5 times:

$ for i in {1..5}; do time cat *tex | detex | wc -w; done
   78174

real    0m0.024s
user    0m0.025s
sys 0m0.012s
   78174

real    0m0.021s
user    0m0.023s
sys 0m0.009s
   78174

real    0m0.019s
user    0m0.020s
sys 0m0.008s
   78174

real    0m0.019s
user    0m0.019s
sys 0m0.008s
   78174

real    0m0.017s
user    0m0.018s
sys 0m0.008s

Ran the non-cat version 5 times:

$ for i in {1..5}; do time detex *tex | wc -w; done
   78174

real    0m0.027s
user    0m0.023s
sys 0m0.009s
   78174

real    0m0.024s
user    0m0.021s
sys 0m0.007s
   78174

real    0m0.021s
user    0m0.020s
sys 0m0.005s
   78174

real    0m0.019s
user    0m0.017s
sys 0m0.005s
   78174

real    0m0.018s
user    0m0.016s
sys 0m0.005s

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.