0

I'd like to create a temporary file that will be read by multiple scripts after its creation, but I don't have an easy way of monitoring when the last script finishes reading this temporary file to delete it (it may be a different script each time). I'd like to know if there's a standard way of solving this problem with command-line tools that will autodelete this file when it passes a specific interval of time without being read by any program, is it possible? Or the only way to solve this problem would be to figure out a way of knowing when the last script finishes reading this file for deleting it?

4
  • The stat command will give you the information you need, but there's no auto-delete mechanism that I know of. – ajgringo619 Dec 9 '20 at 1:33
  • The stat command gives you the last access time of a file in seconds-since-epoch. That could go in a script initiated as a job using at. Depending on the outcome of the stat and date +%s, it will either decide to remove the file, or re-issue an at command to re-run itself after a delay based on the last access time plus required delay. – Paul_Pedant Dec 9 '20 at 11:27
  • Thanks @ajgringo619 and @Paul_Pedant... I've checked the stat command as well but the problem I faced with it is that I don't have the time that the file was last closed using stat... inotifywait worked better because it gives me that information. As an example, for testing the stat command I've created a small script that reads my file line by line and sleeps 5 seconds in each line, in my case, the stat knows that the file was accessed but it doesn't know when it was closed. That's a problem because there may still have some outside program reading the file when it's deleted. – raylight Dec 9 '20 at 12:53
  • stat is frequently no use at all. The Linux default (set by the relatime mount option) does not update atime unless (a) an access is made that is newer than the mtime or ctime of the file (essentially shows that a file has been read since changed, but not other recent accesses). And (b) previous access time is more than 24 hours out of date. – Paul_Pedant Dec 9 '20 at 18:34
0

Using inotifywait, you can add the following to your scripts that access the file.

if ! inotifywait $FILE -t $SECONDS >/dev/null 2>&1
then rm $FILE
fi &

The script will spawn a process that waits until something accesses $FILE.
if something does, it does nothing and exits.
if the timeout is reached after $SECONDS, it will remove the file

alternatively using a seperate script, that you run at creation of the file

while inotifywait $FILE -t $SECONDS >/dev/null 2>&1
do sleep 1
done
rm $FILE

it will loop whenever the file is accessed,
if nothing accesses it within the timeout, it breaks and removes the file.

1
  • Thanks, your answer gave me the basis for solving this problem... The only problem with this answer is that if a script stays with the file open for an undetermined period of time without closing the file your solutions will timeout before the time. I tried to create a bash solution but I didn't manage to do it... Although with node.js using inotifywait it was simpler... I'll post my solution as well. – raylight Dec 9 '20 at 4:19
0

Based on taiyu's answer using inotifywait I've created a node.js solution to this problem... It required more details than I expected when I asked it. Sorry if that's not the right place for posting node.js code but the asynchronous nature of the language made things simpler for me... My solution is the following:

const fs = require('fs');
const spawn = require("child_process").spawn;

const file='/home/user/hugefile.csv';
let counter = 0; // counter used for identifying how many times the file was opened and closed by a different program

const child = spawn('/bin/bash', [ '-c', `inotifywait --format=%e -q -m -e access,close ${file}` ])

child.stdout.on('data', (data) => {
    let line = data.toString().split('\n').filter(item => item); // get events inside a javascript array and filter empty values
    // loop through the inotify events
    line.forEach(function(event) {
        if ( event === "ACCESS" )
            counter++;
        else if ( event === "CLOSE_NOWRITE,CLOSE" )
            counter--;

        // asynchronous function that checks the value of counter after 10 seconds
        async function timer() {
            await sleep(10000);
            console.log(counter);
            if ( counter === 0 ){
                fs.unlinkSync(file); // erase file
                console.log("tmpfile erased!")
                process.exit();
            }
        }
        timer();
    });
});

function sleep(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
}

Basically, I used inotifywait as a basis for solving the problem... I just need to execute this script after creating the temporary file and it will erase the file after all other programs finish reading the file (after 10 seconds).


OBS: My problem trying to solve it with bash is that when I make a function run as a process with & I lose control of the new values of global variables inside that function. So I couldn't get the counter state using the same logic that I've used on node.js... If someone knows a workaround for this, feel free to write in the comments here. :)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.