3

A few files on my server were inadvertently saved with line breaks. They show up as file?name.txt when looking at the files on the server. Using scp to transfer these files to my local machine doesn't work, as I get a "couldn't write file" error. The error causing this has been fixed but I'm not able to access these files. What is the best way to get to these files?

  • Are you looking for a way to rename these files so that they don't have the line breaks in them? Are there existing files (without line breaks in their name) that would be clobbered as a result? (e.g. filename.txt from your example) – Jeff Schaller Jul 22 '17 at 19:47
  • Did either of the answers solve your problem? If so, please use the checkmark to tell the system that the question is Answered. Thank you! – Jeff Schaller Jul 30 '17 at 12:13
4

Newline is as valid as any other character in a file name on Unix. For a shell, that character has a special meaning (it separates commands), so needs to be quoted like any other special characters (like space, ;, |...).

In Bourne-like shells, that can be done with single or double quotes:

ls -d 'a
b'

Or:

ls -d "a
b"

Some shells (ksh93, zsh, bash, mksh, FreeBSD sh at least) also support:

ls -d $'a\nb'

For scp, note that the name of the file is passed to the login shell of the remote user. When you do:

scp file user@host:something

it ends up invoking the user's login shell on host to interpret the:

scp -t something

command line (at least with openssh). So if that shell is Bourne-like, it has to be:

scp 'src/a
b' user@host:"dest/'a
b'"

If you do:

scp 'a
b' user@host:dest/

or

scp -r src user@host:dest/

sshd on the server runs:

scp -t dest/

or

scp -r -t dest/

And the names of the files are not passed on the remote shell command line, but embedded in the data stream between the client and server scp. What I find however (with openssh 7.2p2 at least) is that the newline character ends up transformed to \^J (3 characters).

So you may want to use a different mechanism over ssh that transfers files more reliably like rsync or sftp, or tar:

tar cf - src | gzip | ssh user@host 'cd dest && gunzip | tar xpf -'

That would also deal better with preserving hard links, soft links and other file attribute. You may also need to consider charset conversion if the file names contain non-ASCII characters.

  • Ok I got it working -- rsync worked for me! Thanks! – user2762533 Aug 7 '17 at 19:23
0

To rename existing files that have newlines (assuming that's what they are) in them, with ksh93, zsh, bash or mksh:

for f in *$'\n'*; do mv -i -- "$f" "${f//$'\n'/}"; done

The glob pattern *$'\n'* says to match any/all filenames that have newlines in them; the mv command renames the files by removing all instances of newlines from them.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.