9

I have a folder called statistics in an Ubuntu server in which data files are regularly stored. How can I rename statistics folder to backup-xx while re-creating statistics folder to be available for storing new files?

The files in statistics folder is created by PHP file_put_contents.

I prefer renaming the folder, as there are many files in the statistics folder.

  • At first I thought you meant with "at the same time" that the operation should be "atomic" (as far as it is possible). – phk Apr 29 '17 at 12:59
  • @phk Yes, it is exactly what I meant. – Googlebot Apr 29 '17 at 13:22
  • The correct terminology would be "directory" and not "folder" by the way. – Berk Özbalcı Apr 29 '17 at 23:55
7
mv statistics backup-xx && mkdir statistics

This would rename the existing statistics directory to backup-xx, and if that succeeds it would carry on to create a new statistics directory.

For a more atomic operation, consider creating a directory statistics-001 (or similar, maybe by replacing 001 with today's date in a suitable format), and a symbolic link to it called statistics:

mkdir statistics-001
ln -s statistics-001 statistics

When you want to "rotate" this so that new data goes into a clean directory, create the directory first, then recreate the statistics link to it:

mkdir statistics-002
ln -sf statistics-002 statistics

mv statistics-001 backup-001

This way, any program writing to the statistics directory (i.e. the directory that this symbolic link points to) will never1 fail to find it.

If you need special permissions or ownership set on the directory that statistics points to, set these before (re-)creating the link.

1Or rather, this way, the time that a program would be without a valid target directory is minimized as much as practically possible using standard Unix tools.

  • 1
    You may also want a chown --reference=backup-xx statistics ; chmod --reference=backup-xx statistics to set ownerships and permissions. Ideall you would do this before renaming, so a mkdir new ; chown ; chmod ; mv stats backup ; mv new stats type process. – Stephen Harris Apr 29 '17 at 12:31
  • @StephenHarris Yes, on systems that have those flags. – Kusalananda Apr 29 '17 at 13:18
  • This isn't atomic, though. It's close to atomic, but not quite. – a CVn Apr 29 '17 at 14:17
  • @MichaelKjörling Fixed it. – Kusalananda Apr 29 '17 at 14:25
  • ln -sf is not atomic. It's specified to unlink the old symlink then make a new one, which is necessarily non-atomic. Instead you must make an additional temporary symlink and use the rename function (via the mv command, for instance) to atomically replace the old one. See my approach to it here: git.musl-libc.org/cgit/musl/tree/tools/… – R.. Apr 30 '17 at 3:29
12

There's no way to atomically replace a directory by another directory. You can move the old directory then create a new directory:

mv statistics backup-xx
mkdir statistics

But this leaves a small window of time during which statistics doesn't exist. This is a problem if some process may drop files in the directory at any time.

To effectively atomically replace a directory, you need to use symbolic links. Create a directory whose name includes the time period from the start:

mkdir "statistics-$(date +%Y%m%d)"

(or however you want to choose the directory naming convention). Create a symbolic link with a fixed name to the current location:

ln -s … statistics

To replace the directory, first create a new directory and a new symbolic link, and then move it to overwrite the old symbolic link. Note that neither a plain mv on the symbolic link nor a plain ln -s will do this: they would create an entry inside the target directory. GNU coreutils's ln -snf isn't suitable either because it removes the existing symlink before creating a new one, which leaves a small time window during which the path doesn't exist. You can use GNU coreutils's mv -T on the new symbolic link.

new_dir="statistics-$(date +%Y%m%d)"
mkdir "$new_dir"
ln -s statistics.new
mv -Tf statistics.new statistics
  • busybox' ln -sf also isn't atomic, I don't know of any implementation that is. – phk Apr 29 '17 at 13:01
  • 1
    Beside phk's comment symlinks are also not 100% safe because a running process may still have an open directory file descriptor (symlink target) and it could create more new files in the renamed directory (think about openat() or CWD). I would only move the content of the directory to a new one. That's is still unsafe regarding open files, but a bit better. In doubt use logrotate which should respect such issues best possible. – rudimeier Apr 29 '17 at 13:50
  • 1
    @phk The symlink system call won't overwrite an existing file, so it's impossible to directly switch a symbolic link atomically, unfortunately. The rename system call will overwrite an existing file, but annoyingly there's no portable way to do it in the shell; it's possible with GNU coreutils thanks to the -T option. – Gilles Apr 29 '17 at 14:21
3

Don't rename the directory at all. You said that you prefer renaming the directory because it has a lot of files. The only reason I can think of that you would want this is because copying the files would take too long. However, moving (i.e. renaming) the files is instantaneous as long as they are moved to a location on the same file system. I assume this is what you ant to do since if you're changing file systems, mv will take as much time as cp no matter whether it is moving a directory or its contents.

So, just do:

mkdir backup-xx && mv statistics/* backup-xx

If you need to also get hidden files, you can do:

mkdir backup-xx && mv statistics/* statistics/.* backup-xx

Or, if using bash:

shopt -s dotglob; mkdir backup-xx && mv statistics/* statistics/.* backup-xx

This way, the directory is always there but you still move its contents in a simple and fast operation.

1

You can move the contents of the statistics folder to a newly created directory rather than moving folder itself. If you move entire folder then you would have to run another command to change the directory permissions.

mkdir -p <path>/backup-xxx
mv statistics/* <path>/backup-xxx/.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.