2

I have few directories whose names looks like the following:

enter image description here

I want to remove the newline from the end recursively.

I checked Recursively rename directories

I also checked Remove newlines in file names

The solution it suggests is:

find -name $'*\n*' -exec rename  $'s|\n| |g' '{}' \;

But in my case find -name $'*\n*' returns nothing. If I remove the $ it can find the directories

% find . -name '*\n*'
./second?
% find . -name '*\r*'
./third?
./first?

However, when I run find . -name '*\n*' -exec rename $'s|\n| |g' '{}' \; it does not rename the directory. I also tried find . -name $'*\n*' -exec rename $'\n' ' ' {} \; from Recursively remove newline in file names. It is also not renaming the directories.

What can I do?

7
  • 2
    On Linux, with GNU tools, find -name $'*\n*' works for me and finds filenames with hard newlines. -name '*\n*' is the same as -name '*n*' and finds names with an n... What shell are you actually running? You tagged this with both bash and zsh, and they should both support $'...'. Are you sure it's the actual newline character you have there, and not something different? What do you get if you run ls --quoting-style=shell-escape (with a recent-ish GNU ls) or printf "%q\n" * (in Bash or Zsh) in the directory with those dirs?
    – ilkkachu
    Commented Apr 11, 2021 at 21:45
  • 1
    Also, for the commands with rename, you need to know if you have the Perl one, or the util-linux one... What does rename --version say? (see also unix.stackexchange.com/a/510583/170373 and links therein)
    – ilkkachu
    Commented Apr 11, 2021 at 21:47
  • @ilkkachu printf "%q\n" * shows first$'\342\200\251' second$'\342\200\251' third$'\342\200\251' Commented Apr 11, 2021 at 22:56
  • @ilkkachu rename --version says /usr/bin/rename using File::Rename version 1.10 Commented Apr 11, 2021 at 22:58
  • 1
    or find . -type d -exec rename 's/[\n\r]|\342\200\251/ /g' {} +
    – cas
    Commented Apr 12, 2021 at 0:42

4 Answers 4

4

You said that

printf "%q\n" * shows first$'\342\200\251' second$'\342\200\251' third$'\342\200\251'

The $'\342\200\251' there isn't the regular newline, it's the Unicode U+2029 PARAGRAPH SEPARATOR, encoded in UTF-8. The output there has the bytes in octal; in hex, they would be e2 80 a9.

That's why find -name $'*\n*' doesn't match it. Without the dollar, -name '*\n*' would be the same as just -name '*n*' matching the letter n in second. The patterns -name takes are just shell glob patterns, where \ makes the next character non-special. E.g. \* matches just the single asterisk, while * matches anything. n isn't special, so \n is just the same as it.

Given we now know what it is, we can get rid of it in the same way we would of a newline. With the Perl rename (e.g. the one using File::Rename), this should just remove them:

find . -depth -name $'*\342\200\251' \
       -execdir rename -v $'s|\342\200\251||g' '{}' +

You need -depth for leaves to be renamed before branches they're on and -execdir (not standard but pretty common) for rename to be called from within the directories containing the files to rename with only the basenames of the files.

An alternative to -execdir¹ is to pass the list of files to rename with -print0² | rename -0 and use rename's -d (with File::Rename 1.10 or newer to act on the base name only:

find . -depth -name $'*\342\200\251' -print0 |
  rename -d -0 -v $'s|\342\200\251||g'

... rename -v 's|\342\200\251||g' should work too, since it's a Perl expression and Perl does interpret backslash escapes itself, too.

The command you had, has rename $'s|\n| |g' which would replace with a space, but since you have the characters at the end of the names, that would also be confusing.


¹ which can be problematic and introduce ACE vulnerabilities on systems where -execdir doesn't prepend a ./ to what {} expands to and with older versions of File::Rename's rename which accept options after non-options.

² -print0 is now standard since the 2024 edition of the POSIX/SUS standard, but if your find still doesn't support it, you can replace it with -exec printf '%s\0' {} + or use -exec rename -d ... {} +.

1
  • 1
    thank you very much. Commented Apr 12, 2021 at 6:45
4

With zsh:

autoload zmv # best in ~/.zshrc
zmv -v $'(**/)(*[\n\u2029]*)(#qD)' $'$1${2//[\u2029\n]}'

To remove both newlines (U+000A) or the paragraph separator character (U+2029).

Or:

zmv -v $'(**/)(*[[:cntrl:]]*)(#qD)' $'$1${2//[[:cntrl:]]}'

to remove all control characters.

Whether U+2029 will be classified as cntrl or not will depend on the system though. It is on Ubuntu 20.04, but not on FreeBSD 12.2 for instance. Run [[ $'\u2029' = [[:cntrl:]] ]] && echo yes to check on your system.

1
  • OP here. This is actually a better solution. But, as I have already accepted an answer, I do not want to change it. Moreover, @ilkkachu taught me to use printf "%q\n" * to find what the real control operator is so I think he deserves some appreciation. Commented Apr 12, 2021 at 12:58
0

Here it turned out to be about removing the U+2029 paragraph separator character (which your file manager happens to render as ) rather than newline.

But to answer the question in the subject, to remove only the newline characters (U+000A), you'd adapt @ilkkachu's answer (using the rename from perl's File::Rename as packaged on most Linux distributions, with the executable sometimes called rename or prename or perl-rename or file-rename or pname) to:

find . -depth -name $'*\n*' -execdir rename -v 's/\n//g' '{}' +
find . -depth -name $'*\n*' -print0 | rename -d -0 -v 's/\n//g'

Or with the one from http://plasmasturm.org/code/rename aka https://github.com/ap/rename (more features but missing the ability to work on the base name only which we need to implement by hand in perl):

find . -depth -name $'*\n*' -print0 |
  rename -0 -v -e 's{[^/]+\z}{$& =~ s/\n//gr}e'

Or @StéphaneChazelas's answer using zsh to:

autoload zmv
zmv -v $'(**/)(*\n*)(#qD)' $'$1${2//\n}'
10
  • Why $PWD instead of .? What shell is that for? In Bourne-like ones expect zsh, $PWD would need to be quoted. Commented Jul 5 at 18:41
  • -iname "*\n*" in most shells and with find implementations that support -iname would match on file names that contain n or N, while perl's s{\n}{_}g would replace newline characters with _. Commented Jul 5 at 18:43
  • What's the builtin rename. I don't know of any shell that has rename builtin. Some systems come with a rename utility preinstalled, but there are a great number of incompatible commands by that name, some perl-based, some not. Commented Jul 5 at 18:44
  • The rename.pl you provide a link to doesn't seem to support a -p option. Commented Jul 5 at 18:46
  • Why the -maxdepth 100? Commented Jul 5 at 18:46
0

rename.pl

For those of you who always prefer find + xargs, here's what you need:

find -iname  $'*\n*' -print0 |  xargs -0 rename.pl -p -v  's{\n}{_}g'

I had this issue on some Windows dirs that I'd made a backup of on Linux. Note you'll need to delete the empty dirs afterwards.

FYI rename.pl is better than the rename that is more widely available in package managers:

    -p/--mkpath/--make-dirs
1
  • Still same problems: find -iname "*\n*" matches on files whose name contains n or N. You need find . -name $'*\n*', no need for xargs as your rename has a -0. Without -depth, you'll end up renaming directories under find's feet, so it won't be able to rename files within. With -depth and that -p, with a $'./dir\n/file\n', you'd end up renaming that to ./dir_/file_ (into a new dir_ directory), and then renaming $'./dir\n' to ./dir_ will fail as ./dir_ already exists. Commented Jul 7 at 10:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .