7

As a personal project, I'm trying to write a script to clean up some of the extraneous files that ship with Windows 10 that runs under Cygwin. I'd like the script to be copy and paste-able, which would require it to be robust against different kinds of newlines. Are there any sh-like shells that expose the ability to treat \r as whitespace, possibly when some option is set?

It's a strange thing to do. The answer might boil down to "yes, that is a strange thing to do and there's no way to do it".

4
  • Anything against dos2unix or plain old sed? Dec 31, 2018 at 4:54
  • I'd like the script to be able to "just run" without the line endings being fixed up first. There are some languages/programs like python and perl that are line-ending insensitive. Dec 31, 2018 at 4:58
  • Do you want a shebang line for a script that will run in such a shell, or just identifying a suitable shell that could be invoked directly? Dec 31, 2018 at 5:17
  • @MichaelHomer Both of those options are totally fine. As far as I can tell, bash itself can't be configured to process \r differently. Dec 31, 2018 at 5:25

1 Answer 1

7

This is, in general, not possible for an actual script.

  1. A standalone executable script will have a shebang line that identifies the interpreter to use:

    #!/bin/sh
    

    The problem is that, regardless of the interpreter, the shebang line itself will contain a carriage return character before the end, meaning that the interpreter won't be found (barring the unlikely executable name of literally $'sh\r' - which I think is illegal on Windows).

  2. The other kind of executable script is a plain text file with the execute bit set, which POSIX requires to be interpreted as a script in the POSIX shell command language (when invoked from a compliant shell itself). This shell language, however, as you've found, treats carriage returns as data bytes, so you have the issue you've identified.


You have some options. One is to forego an actual executable script directly, and simply to invoke an interpreter for some other language first. You've identified Perl and Python, and in fact most scripting languages are happy with any contemporary line-ending format. Invoking the script as perl cleanup.pl will not be a problem.

Many administrative tasks are much easier in a conventional shell language, however, and there is one more trick: end every line with a comment.

echo hello #
rm file.ext #
mv /path/other.fl /path/bin #

The # character introduces a comment to the end of the line in all standard shells (in non-interactive contexts). The comment will include the carriage return byte, which is a fine element of a comment, and then end immediately afterwards with the terminating line feed byte.

It's important that there be that space before the hash: a # inside a word does not begin a comment, but is part of the word.

You could run this script directly with a shell:

bash cleanup.sh
dash cleanup.sh
sh cleanup.sh

But if you're going to launch it by typing a command from an existing shell session, those POSIX scripts mentioned earlier will also work: just make the file executable, and then

./cleanup.sh

will run it using whatever your shell considers the right thing to do for a POSIX script.

Since this sounds like a one-off it probably isn't worth the effort of making it executable, so just sh cleanup.sh seems adequate to me.


My honest suggestion for this situation is probably Powershell (or batch at a push), but I understand it's not what you want. WSL is another option: it handles carriage returns in the shebang line automatically, and then you can use comments for the rest.

3
  • 1
    Can you use a command-line arg to hide / eat the \r? like #!/bin/sh -b (Bash's -b "Report the status of terminated background jobs immediately, rather than before the next primary prompt", so it's pretty much harmless for non-interactive use.) I just tried it on Linux with -b^M, and bash says : invalid option, so unfortunately not. Maybe there's an option that can consume a character as an arg, without breaking if it's not there? Dec 31, 2018 at 9:40
  • 1
    Cygwin is funky, though: it's the cygwin library itself that parses the #! line, because the OS proper (Windows) doesn't understand it. It's a user-space emulation of POSIX on top of a non-POSIX OS. So in theory you could just build a version of the cygwin DLL that strips a trailing \r (if present) from a shebang line before processing, maybe more easily than you could compile a custom Linux kernel that did the same. And you don't need to modify the Windows kernel (good thing because it's closed source). Dec 31, 2018 at 9:45
  • 1
    But since windows considers \r\n the usual line ending, shouldn't a hashbang line of #!/bin/sh\r\n look for /bin/sh without the CR? Then you'd only need to have a properly fixed shell in /bin/sh\r. Which should work on Unixen (as long as the script doesn't try to pass arguments on the hashbang line), though it would be hideously ugly.
    – ilkkachu
    Dec 31, 2018 at 10:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.