Slightly unusual usecase: I have a download from firefox which is actually a stream of data. I want to process it in (near) real-time, without saving the data (it's high volume and would fill my drive quickly). Is there a way I can convert the file being downloaded to a pipe/socket of some sort and just read from it directly programmatically somehow?
Things I tried so far:
Having a script move the file to a different location periodically, read the moved file into a buffer, and repeat to read the stream piece by piece. Unfortunately doing so stops the download (firefox keeps downloading, but there is no new .part file that appears after moving the original one).
I found that if I just wrote to the file (echo '' > file.part), the download does continue. So I could read the contents, overwrite the file (to avoid filling up space), and repeat indefinitely - however between reading the file and overwriting it, a few more bytes get written by firefox in the meantime so the stream gets corrupted.
Some details as per questions in comments:
The download is a capture (pcap) file whose download is initiated by a Fritzbox web UI. I can't curl it because it's the fritzbox that somehow initiates the download, there's no url I can use in curl. Not sure how the download is implemented, as it appears as a "normal" download in Firefox but has no set size and it keeps adding data until I press "stop" in the web ui.