# Why this 'useless use of cat' with detex is 10x faster than the alternative?

I'm counting words in LaTeX files and intuitively used the command:

$cat *tex | detex | wc -w  I then immediately noticed he useless use of cat, and did the task again with: $ detex *tex | wc -w


Being proud of myself for avoiding an extra process, I wanted to check how many precious milliseconds I saved by not using cat. I was then very surprised to see that the version with cat was literally 10 times faster:

$time cat *tex | detex | wc -w 5000 real 0m0.013s user 0m0.005s sys 0m0.007s  $ time detex *tex | wc -w
5000

real    0m0.144s
user    0m0.124s
sys 0m0.014s


What is the reason for this massive speed difference? Is cat just so much faster in reading files from a disk than detex?

I generated 5 example "LaTeX" files for benchmarking with:

$for i in {1..5}; do shuf -n 1000 /usr/share/dict/words > "file${i}.tex"; done

• One possibility I feel likely is that detex reinitializes some costly setup after each file, while cat assembles them all into a single stream which is detexed only once. Mar 14, 2021 at 22:05
• straceing detex (from texlive on Arch Linux) in the two cases shows that, when invoked with arguments, it opens and reads several files (several ls-R, texmf.cnf, texmf, texmf-dist) that are not open when it reads from standard input. Though I have no clue on the reason. Mar 14, 2021 at 23:02
• What version of detex are you using? This is presumably related to Kpathsea but I don’t reproduce it on Debian 10. Mar 15, 2021 at 5:59
• You may want to check that the result from the two ways of running detex are identical. Note also that the detex manual says Running LaTeX source without a \begin{document}'' through detex may produce errors., so since your "test data" isn't actual TeX or LaTeX input, I would be wary about interpreting any timing results. Mar 15, 2021 at 8:40
• @Kusalananda The results were the same, $diff <(cat *tex | detex) <(detex *tex) gives no diff. But I posted an answer where I show that my initial results were just noise. Mar 16, 2021 at 11:45 ## 1 Answer The initial results seem to just be an inconsistency. Installed the latest detex from Homebrew: $ detex -v

OpenDetex version 2.8.9
https://github.com/pkubowicz/opendetex


Got an example LaTeX file:

$curl https://raw.githubusercontent.com/latex3/latex3/main/articles/2011-current-state.tex > file.tex  Made 100 copies of the file: $ for i in {1..100}; do cp file.tex "file${i}.tex"; done  Ran the cat version 5 times: $ for i in {1..5}; do time cat *tex | detex | wc -w; done
78174

real    0m0.024s
user    0m0.025s
sys 0m0.012s
78174

real    0m0.021s
user    0m0.023s
sys 0m0.009s
78174

real    0m0.019s
user    0m0.020s
sys 0m0.008s
78174

real    0m0.019s
user    0m0.019s
sys 0m0.008s
78174

real    0m0.017s
user    0m0.018s
sys 0m0.008s


Ran the non-cat version 5 times:

\$ for i in {1..5}; do time detex *tex | wc -w; done
78174

real    0m0.027s
user    0m0.023s
sys 0m0.009s
78174

real    0m0.024s
user    0m0.021s
sys 0m0.007s
78174

real    0m0.021s
user    0m0.020s
sys 0m0.005s
78174

real    0m0.019s
user    0m0.017s
sys 0m0.005s
78174

real    0m0.018s
user    0m0.016s
sys 0m0.005s