3

Sometimes I need to batch rename files on a macOS system to random or pseudo-random filenames, eight lowercase letters and digits only, e.g. from mcmurdo-station.png to something like a12bc0xy.png. For this, I use CRC32:

for f in *.png; do mv $f $(crc32 $f); done

But there is a drawback that if there are two identical files in the same folder, they have same checksums, and so there will be a filename collision. Of course, usually I don't store identical files in the same folder, but I nevertheless would prefer to avoid such over-automatization while batch renaming.

Another way I have found on the internet is to use base64:

randomname() { head -c8 /dev/urandom | base64 | tr -dc a-z0-9; }
zmv '(*).(*)' '`randomname`.$2'

2024-05-22 edit: Instead of backquotes, it is better to use a dollar sign:

zmv '*(.*)' '$(randomname)$2'

But the resulting filenames are of different lenght: the first test file was renamed from foo.png to 52rud.png, whereas the second - from bar.png to pxg.png.

There is also a solution using shuf from GNU Core Utils: https://unix.stackexchange.com/a/259761, but for now I would prefer to find a way that don't require installing third-party packages.

What is a good and simple alternative?

11
  • 2
    "But the resulting filenames are of different lenght" – </dev/urandom base64 | tr … | head … Commented May 19 at 16:39
  • 2
    base64 encodes on the fly. To generate a hash you need to terminate the input stream. Start with </dev/urandom head -c N (pick your N) and generate CRC32 from this. Commented May 19 at 18:47
  • 2
    I'd suggest that if you're trying to avoid namespace clashes, there's no 'simple' answer, because you need some manner of "memory" to avoid that. Collisions at least means you're not actually losing data that way though, because the file was a dupe in the first place, and you could have just 'hard linked' to replace it.
    – Sobrique
    Commented May 20 at 8:37
  • 2
    What about uuidgen? It was not enough?
    – Hastur
    Commented May 21 at 9:10
  • 2
    I'm quite curious as to : why do you need to rename files to a random filename ? (and therefore unreversible, unless you intend to rename back using their checksums and a list? ) Commented May 21 at 11:27

4 Answers 4

9

You can just rename the files to an incremented number, only process them in random order.

random() REPLY=$RANDOM
n=0; zmv '*(.*)(#qo+random)' '${(l[8][0])$((++n))}$1'

Here using the l[length][pad] left-padding parameter expansion flag, to left-pad the number with 0s to a length of 8.

Example (here with -n for dry-run):

$ n=0; zmv -n '*(.*)(#qo+random)' '${(l[8][0])$((++n))}$1'
mv -- 21.jpg 00000001.jpg
mv -- 2111.jpg 00000002.jpg
mv -- 111.jpg 00000003.jpg
mv -- 2.jpg 00000004.jpg
mv -- 00012.jpg 00000005.jpg
mv -- 211.jpg 00000006.jpg
mv -- 10012.jpg 00000007.jpg
mv -- 11.jpg 00000008.jpg
mv -- 1111.jpg 00000009.jpg
mv -- 1.jpg 00000010.jpg

For a random string with a-z letters and 0-9 digits, you could use a random number in base 36 (in the 0 to 36#ZZZZZZZZ range for 8 digit long ones)

zmodload zsh/mathfunc
zmv '*(.*)' '${(Ll[8][0])$(( [##36] 36#100000000 * rand48() ))}$1'

(adding the L parameter expansion flag to convert to lower case)

Note that though extremely unlikely, it's not guaranteed that there won't be any collision (though zmv will detect it and bail out before doing any renaming if that happens).

$ zmv -n '*(.*)' '${(Ll[8][0])$(( [##36] 36#100000000 * rand48() ))}$1'
mv -- 00012.jpg 60m34z1w.jpg
mv -- 10012.jpg 4ff2b06p.jpg
mv -- 1111.jpg n2mac2r4.jpg
mv -- 111.jpg dwf72mfg.jpg
mv -- 11.jpg 3nef10qd.jpg
mv -- 1.jpg lggtk364.jpg
mv -- 2111.jpg am7sd85t.jpg
mv -- 211.jpg 35n7ebtn.jpg
mv -- 21.jpg eussspd7.jpg
mv -- 2.jpg 9jjgiw08.jpg

To avoid collision, you could define a urand() math function that checks the generated number against the recorded list of previously generated numbers and tries again if seen:

urand() {
  integer n
  while (( seen[$(( n = rand48() * $1 ))]++ )) continue
  (( n ))
}
zmodload zsh/mathfunc
functions -M urand 1

typeset -A seen=()
zmv '*(.*)' '${(Ll[8][0])$(( [##36] urand(36#100000000) ))}$1'
2
  • Many thanks, Stéphane. "For a random string with a-z letters and 0-9 digits, you could use a random number in base 36." — Could you explain how this method is different, in practical point of view, from the method that utilizes base 16, which is mentioned in the question and further discussed in comments?
    – jsx97
    Commented May 20 at 8:48
  • 2
    @jsx97, not sure what you're asking. For one thing, this approach uses characters 0-9a-z as requested while base 16 ones would use only character 0-9a-f (my first one only uses 0-9). It also doesn't rely on anything other than zsh, doesn't fork nor execute anything (except for the mv called by zmv but you could also enable the builtin mv with zmodload zsh/files). It is not locale dependant. Commented May 21 at 17:33
3

First, I suggest that if you've got the same checksum you've probably got the same contents. It may be desirable to preserve duplicates in all but name. In that case use the name as (some of) your input. But if you're getting checksum collisions on different contents, use md5sum instead - it's 128 bits instead of 32 reducing the chance to negligible levels. I've used it for uniqueness checking in the past, in cases where a 32-bit checksum would have an unacceptably high chance of collisions.

Lets assume you do want to be able to handle files with the same contents and different names

I have a different case of deterministically mapping to arbitrary unique values which would adapt - automatically colour coding map plots; I want them to be the same if reloaded. I generate part of the unique value by a simple hash of the file name, and map this to the red channel. The green is generated from the length of the filename. Applying the same approach in this context gives us (note the use of echo for demo purposes).

Note that here I use md5sum because as well as the 128-bit output it can be applied to strings (filenames) with a simple trick.

for filename in *.jpg; do
    newfilename=$(echo -n "$filename" | md5sum | cut -d ' ' -f 1)
    newfilename=$newfilename${#filename}
    echo mv -- "$filename" "$newfilename.jpg"
done

If the new filenames are longer than you'd like, you could replace the cut -d ' ' -f 1 command with cut -c1-10 or however many digits you want in your value - at the expenses of increased chances of collisions of course.

To be honest, I very much doubt you need anything beyond the use of md5sum on the name; I append the length only to complete the analogy with my process, and to demonstrate how you'd do it if sticking with CRC32.

You could work on the file's date instead if you wanted; I specifically don't want to because I didn't want the plot colour to change if I re-downloaded the file. You could also use the file checksum for one part of the name and the filename hash for the other.

BTW, standard caveat - MD5 is no good for security, so don't use it for that.

4
  • Hello, Chris. It doesn't work for me currently; I suppose you tested in on a different operating system, probably Linux, whereas I need it to work on macOS. The first problem is that if I simply execute this code "as is", there is zsh: command not found: md5sum error. Then I have replaced md5sum with crc32 and removed both echo, but still doesn't work.
    – jsx97
    Commented May 20 at 12:06
  • 1
    @jsx97 crc32 only works on file contents, at least on systems I know. The lack of md5sum on zsh surprises me though. The echo at the end is the one to remove but there's an error in there for me to fix first (I had originally copied old to new then deleted old, and combined the lines wrong)
    – Chris H
    Commented May 20 at 12:43
  • 1
    ... from this answer you might try replace md5sum with md5. As I thought it's not zsh that's the issue, but MacOS that uses a different name. And that means I can't test it
    – Chris H
    Commented May 20 at 12:48
  • 1
    Many thanks for the answer, Chris. After replacing md5sum with md5 and removing the second echo, it works.
    – jsx97
    Commented May 20 at 19:22
2

Another approach is to regenerate the random name as long as a file with the first random name exists. Something like this:

randomname() { base64 < /dev/urandom | tr -dC a-z0-9 | head -c4; }
for f in *.png; do
 newName=$(randomname).png
 while [ -e "$newName" ]; do
   newName=$(randomname).png
 done
 mv -- "$f" "$newName"
done
10
  • Thanks, terdon. Are sure there are no errors here? It produces tr: Illegal byte sequence error.
    – jsx97
    Commented May 20 at 11:03
  • @jsx97 hmm. No, on my system (Arch Linux) it runs fine. Are you sure you copied it exactly? Can you run ` tr -dc a-z0-9 < /dev/urandom | base64 | head -c4` manually? Does that work?
    – terdon
    Commented May 20 at 11:21
  • Executing tr -dc a-z0-9 < /dev/urandom | base64 | head -c4 on its own produces the same tr: Illegal byte sequence error. This is on macOS, zsh 5.9 (x86_64-apple-darwin23.0)
    – jsx97
    Commented May 20 at 11:27
  • Ah, sorry, I missed that it was a macOS system. According to the macOS tr man page, I think you need tr -dC a-z0-9 < /dev/urandom | base64 | head -c4 (i.e. capital -C for tr). Does that work as expected?
    – terdon
    Commented May 20 at 11:30
  • 1
    Sigh, sorry @jsx97, I hadn't actually saved the edit. I thought I had but it was open in another tab. I meant to ask if the version you can see now works. My bad.
    – terdon
    Commented May 20 at 12:06
1

Use the mktemp command with template and other flags of your choosing: https://ss64.com/mac/mktemp.html

This command generates a unique random file name that does not already exist, creates an empty file with that name, and prints the name. You'd then be free to rename your file over the top of it.

1
  • And what would the actual implementation look like?
    – tink
    Commented May 26 at 20:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .