2

I run find on a file structure (several hundreds of folders, about 4 layers deep) with commands like: find "/media/$disk/_all" -depth -type d -exec/execdir rename -d 's/ 1000k/ [1000k]/' "{}" \;. I noted repeatedly that some occurrences of folders containing 1000k remain unrenamed, I noted some of them got renamed on second run of the command. Why not all on the first run?

Also, I noted often output has "Can't rename /media/some_original /media/some_renamed: No such file or directory", after that I check and some_original got successfully renamed to some_renamed.

First error in my guess occurs due to find not going through all tree, second is due to find going and invoking exec twice for some/all? entries. I added -depth option later to be sure of "child" folder being renamed before "parent", but it did not help (and in my case I have not noted several levels due to be renamed anyway when I noted the error).

I was not able to find answer via web search. What could be the issue here?

System is Linux Mint 20.1.

Added: I've checked I have not found any links in the structure, using ls -lR /path/to/folder | grep ^l from https://stackoverflow.com/questions/8513133/how-do-i-find-all-of-the-symlinks-in-a-directory-tree.

Disk is mounted like that: type ext4 (rw,nosuid,nodev,relatime,stripe=8191,uhelper=udisks2)

9
  • 1
    renaming files will reorder the directory that may lead to some results not being found.
    – Jasen
    Apr 11, 2021 at 8:07
  • @Jasen, thank you, have not thought about it. I thought about it SQL, nodes wise, like go through all nodes in a folder by id and assumed renaming a node would not change its order/id, cause I'm not sorting by name. Do you know of any bash-using way to fix that? Apr 11, 2021 at 9:30
  • (1) I added -depth option later to be sure of "parent" folder being renamed before "child"-depth does the opposite: deeper directories are processed first. And it's the right thing to do when renaming with find. (2) What is the filesystem? Is it FUSE? See "Simultaneous Access and Race Conditions" in this answer. I think find does its things sequentially, so it shouldn't(?) trigger "simultaneous access" in this context. But if the filesystem is FUSE then maybe the implementation is somewhat buggy in the first place. (3) Can you provide MCVE? Apr 11, 2021 at 9:30
  • @KamilMaciorowski, thank you. Child first - I thought about it, just wrote here incorrectly, fixed now. MCVE - what is it? Could not quickly web-search it. I added mount line, drive is in USB attached box. Apr 11, 2021 at 9:40
  • Minimal complete verifiable example. In this case it could be a sequence of mkdir -p commands to create a directory structure prone to the issue; finally the find command you have already posted. (4) If you manage to find MCVE then also tell us if you can always replicate the problem with it. Or does the problem occur randomly? Apr 11, 2021 at 9:45

1 Answer 1

3

Preliminary notes

  • This answer is an attempt to explain what may have happened, not a firm explanation what happened for sure. This is because:

    • the OP has tried different variants of the command (with or without -depth, with -exec or -execdir), the exact command posted in the body of the question should be taken with a grain of salt;
    • each try possibly modified the directory tree, but there is no log stating what renaming happen because of what exact command;
    • similarly there is no log of errors;
    • after analyzing the problem (in comments and chat) we noticed there are few different issues that generate similar error messages (and therefore it's easy to assume there is just one issue).

    For these reasons an attempt to create a single MCVE (in a form of one exact directory tree and one exact find command) failed. We agreed it will be the best to create an answer that covers possible issues, with future readers in mind. This answer is such answer.

  • There is more than one rename. The syntax in question works with rename being a Perl script associated with Larry Wall and Robin Barker. I assume this is the rename used. The issues covered in this answer are not specific to this particular rename though.


What may have happened

There at few different issues that can generate No such file or directory when you use find with rename.

First, let's simplify the example. Let the directory structure be:

/a/b/c.1/c.2/d

and the command be:

find /a/b -exec rename s/c/X/ {} \;

This command will yield:

find: '/a/b/c.1': No such file or directory

This is because without -depth find processes c.1 before it tries to descend to c.2. When processing c.1, rename renames it to X.1, but find is not aware the name changes. It tries to work with c.1 in order to process deeper files but c.1 no longer exists. This is very similar to using find with rm -r.

The second iteration of the command will see /a/b/X.1/c.2/d and successfully rename c.2 to X.2. Nevertheless it will complain about c.2 not being there. The third iteration will do nothing and complain about nothing.

You need to tell find to process files in a directory before doing anything to the directory itself. This is what -depth is for.


Even with -depth there may still be problems. Again, having this structure:

/a/b/c.1/c.2/d

and running the "fixed" command:

find /a/b -depth -exec rename s/c/X/ {} \;

we will get:

Can't rename /a/b/c.1/c.2/d /a/b/X.1/c.2/d: No such file or directory
Can't rename /a/b/c.1/c.2 /a/b/X.1/c.2: No such file or directory

It's not immediately obvious the messages come from rename but they do. The problem is d is processed first and rename wants to change /a/b/c.1/c.2/d into /a/b/X.1/c.2/d. It's like mv /a/b/c.1/c.2/d /a/b/X.1/c.2/d, you cannot do this if /a/b/X.1/c.2/ does not exist. And later (while processing c.2) you cannot mv /a/b/c.1/c.2 /a/b/X.1/c.2 because X.1 does not exist (yet).

(You may be surprised rename by default tries something that cannot succeed. Note mv /a/b/c.1/c.2/d /a/b/X.1/c.2/d cannot succeed when X.1 doesn't exist. Still this command is sane when X.1/c.2/ already exists along c.1/c.2/. The default behavior of rename is equally sane in this case.)

In our case X.1 appears when c.2 is silently renamed to X.1 but this happens later. The second iteration of the command will complain while processing d, it will manage to rename c.2. The third iteration will do nothing and complain about nothing.

Note if you had /c.0 instead of /a and the command was find /c.0/b … then no rename would succeed and any number of iterations would be futile.

The problem is rename s/c/X/ tries to change the first c in the path, even if this c is in a directory component, not exactly in the filename. There are at least two ways to deal with this.


You can use rename -d. This option makes rename act on the filename only. The following command:

find /c.0/b -depth -exec rename -d s/c/X/ {} \;

will successfully change /c.0/b/c.1/c.2/d into /c.0/b/X.1/X.2/d (so even c.0 is not a problem).

-d is a relatively new option. In general your rename may not support it. In such case use -execdir instead of -exec:

find /c.0/b -depth -execdir rename s/c/X/ {} \;

(In general your find may or may not support -execdir.)

It works because when processing d, the string rename processes is now ./d, not /c.0/b/c.1/c.2/d; similarly when processing c.2, it's ./c.2, not /c.0/b/c.1/c.2. The tool works in the respective parent directory each time, so the relative paths work. By not including the full path in the string, we get the behavior of rename -d without -d.

Well, almost. If you want rename to do something with dots (e.g. change c.2 into c-2 etc.) then the leading dot in ./d or ./c.2 is something you need to deal with. With -d there is no such problem, regardless if you use -exec or -execdir; so use rename -d if you can.


Final note

The command in question includes -depth and rename -d. The OP explicitly stated:

I added -depth option later […]

so probably the lack of -depth was the initial problem. But then:

[adding -detph] did not help

Considering this and noting the error (Can't rename …) comes from rename, I conclude -d was also not always there or I'm missing something. For now I think find -depth along with rename -d should be robust. Assuming it really failed for the OP, I cannot explain this (yet).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.