I'm trying to fix my YARN problem. When task is submitted to YARN it's creates new directory with all needed settings and scripts. After finishing this task directory is removed.
My task is failing after 0-2 seconds so files are removed so fast I cannot save them. I also don't know exact name of file before running task (there is autoincementing counter), but I know parent directory and I could try to guess exact path.
I would like to protect or copy this files (whole directory) in some way. I guess I could set up cron running every second and copying parent directory (I don't know if size of files wouln't exceed
cp capabilities). I probably could also change (every second) every file in directory into read-only mode (but it could interfere with YARN processes and create new problems). I can't change permissions to directory - YARN wouldn't be able to create them in first place.
Is there better solution? (YARN detail is not important, but maybe someone knows how to use some unknown to me YARN features).