I am writing a bash script that reads the first 10 lines and the last 10 lines of a .txt file. It looks for started (head) and completed (tail) and compares the number of occurrences using grep. The files are quite large which is why I opted to only read the head and tail of the files instead of the entire text. However, when I run the script the large files take a long time to "finish up" (which consists of reading the first 10 lines and last 10 lines and the compare, a task that should only take a moment or two).
While watching the script output text, I noticed this issue. So I decided to see if it would take a similar amount of time when I simply executed the head/tail (plus grep, as to simulate what is executing in the script) command straight from the command line. Surprisingly, the commands executed near instantly. I thought this was strange and I ran the script again. This time, the script would scream through the file it was stuck on before until it got to the next "large" file I had not already run the head/tail/grep command on.
This got me thinking, does bash store the results of a command similar to caching? Also, what could be causing these commands:
head -n 10 /file/path/myfile.txt | grep -w -c 'lead word' tail -n 10 /file/path/myfile.txt | grep -w -c 'end word'
to be taking so long to execute?
edit: The reason I believe it is the head/tail lines above that are the source of problem is because there are echo lines that should print upon completion of head/tail individually. I have checked the line size of the files and they are not terribly longer than ones that are finishing within a few moments.
Could someone give me a more detailed explanation of how head/tail works at a more technical level? I have a very basic understanding of "the first/last x lines" of a file.