How to make a temporary file in RAM?

Question

I have a script that will pipe its output to |tee scriptnameYYMMDD.txt. After each cycle of the for loop in which the output is generated, I'll be reversing the file contents with tac scriptnameYYYYMMDD.txt > /var/www/html/logs/scriptname.txt so that the log output is visible in a browser window with the newest lines at the top.

I'll have several scripts doing this in parallel. I'm trying to minimize the disk activity, so output from |tee scriptnameYYYYMMDD.txt to a RAMdisk would be best. mktemp creates a file in the /tmp folder, but that doesn't appear to be off-disk.

Answer 1

You can mount a tmpfs partititon and write the file there:

mount -t tmpfs -o size=500m tmpfs /mountpoint

This partition now is limited to 500 MB. If your temporary file grows larger than 500 MB an error will occur: no space left on device. But, it doesn't matter when you specify a larger amount of space than your systems RAM has. tmpfs uses swap space too, so you cannot force a system crash, as opposed to ramfs.

You can now write your file into /mountpoint:

command | tee /mountpoint/scriptnameYYYYMMDD.txt

Answer 2

The following answer was discovered by investigating the previous answers and the info in this question here and would not have been found without them. Cudos to them.

On my linuxmint system (and I would assume most ubuntu based systems and possibly debian based too) there is a user owned tmpfs mounted automatically on /run/user/1000/

Use df -T to check.

11:41:11 jesse@Limbo:~$ df -T
Filesystem     Type      1K-blocks       Used Available Use% Mounted on
udev           devtmpfs   15904812          4  15904808   1% /dev
tmpfs          tmpfs       3184120       1700   3182420   1% /run
/dev/sdb2      ext4       14248880   11464788   2037240  85% /
none           tmpfs             4          0         4   0% /sys/fs/cgroup
none           tmpfs          5120          0      5120   0% /run/lock
none           tmpfs      15920584        848  15919736   1% /run/shm
none           tmpfs        102400         12    102388   1% /run/user
/dev/sdb3      ext4      100861352   90755700   4959136  95% /mnt/data

Under /run/user/ there is a directory for each normal user on the system

12:07:35 jesse@Limbo:~$ ls -l /run/user
total 0
drwx------ 2 root  root   40 Aug  7 09:50 0
drwx------ 8 jesse jesse 180 Aug  7 11:38 1000

These directories are named after their respective user's ids. We can get the user id with id -u see man id for details on this command.

12:07:43 jesse@Limbo:~$ ls -l /run/user/$(id -u)
total 0
drwx------ 2 jesse jesse  60 Aug  7 09:50 dconf
dr-x------ 2 jesse jesse   0 Aug  7 09:50 gvfs
drwx------ 2 jesse jesse  80 Aug  7 09:50 pulse
lrwxrwxrwx 1 root  root   17 Aug  7 09:50 X11-display -> /tmp/.X11-unix/X0

We can then use the mktemp command with the --tmpdir option to create temp files and directories in this tempfilesystem thus creating tempfiles in RAM.

Following the advice given here I create a temp directory first and then create my temp files in that:

mydir=$(mktemp -dt "$(basename $0).XXXXXXXX" --tmpdir=/run/user/$(id -u))

to create a temp directory /run/user/1000/bash.w42BYxbG/ then

myfile=$(mktemp -t "$(basename $0).XXXXXXXX" --tmpdir=$mydir)

to create a tempfile in it.

This makes cleaning up these file easy since all I have to do is rm -r $mydir.

By default all these files are owned and readable only by the user who created them.

Note: The $(basename $0) portion of the command extracts the name of the script/process that executed mktemp. If I have a script /home/jesse/scripts/myScript.sh then $(basename $0) returns myScript.sh when executed by this script. Thus the above commands would create /run/user/1000/myScript.sh.w42BYxbG/ and /run/user/1000/myScript.sh.w42BYxbG/myScript.sh.BCzSmq06 respectively.

Answer 3

Try this with Ubuntu:

ramtmp="$(mktemp -p /dev/shm/)"
tac scriptnameYYYYMMDD.txt > "$ramtmp"

Answer 4

As I understand it, the goals are: (1) do store the reversed file to disk, to be served to the browser (note: this could be tmpfs, etc, as all the other answers already detail); but, (2) avoid writing the first tmp file to disk, with the original output; and yet, (3) still display the original output to stdout.

If so, then the following might meet your needs, using bash process substitution (i.e., basically a named pipe):

command | tee >( tac > /var/www/html/logs/scriptname.txt )

Note that this is continuously printing output to stdout, and when the command ends (when your loop terminates), reverses that output and writes it to the output file. Illustrated via the following, which prints one line per second, and after termination the file /tmp/foo.txt contains the reversed lines:

i=0
while [ $i -lt 10 ]; do
    ((i = i+1))
    echo "==$i== $(date)"
    sleep 1
done | tee >( tac >> /tmp/foo.txt )