I have a script that listens to Twitter and stores tweets with a certain keyword in a JSON file. A new destination file is created every time the script starts.
Sometimes my script crashes and automatically restarts, creating a new JSON file in the process.
I would like to show a running log of the incoming tweets. With a single file I can do this with (piping to
jq to show only a single field from the JSON):
tail -f file1.json | jq '.text'
However, once the script has crashed and restarted, a new file is created (e.g.
file2.json) and the above command listens to a file which is no longer updated.
To work around this issue I thought I should perhaps concatenate all files in the directory and
tail -f | jq '.text' the result.
However, while I can do
cat * to concatenate all files currently in the folder, new files seem not to be added automatically to the concatenation.
How can I continuously concatenate all files in a folder, such that I can always see the latest rows of the newest file?