2

I'm trying to run this open source bash script and it is not working because it is assuming I am running linux when it is parsing inputs. Actually, I'm running cygwin on a Windows machine. There is a spot in the script where it sets the $IFS to \n, which leaves behind a \r which causes errors because it can't match exact strings.

It would be easy to modify my local version of the script, but how would I fix this so it works on any OS? If I could run some command to get the newline characters, then i could assign them to $IFS.


For some additional context, the code that's dealing with newlines is this:

IFS=\n
# Word splitting is intended here
# shellcheck disable=2046
set -- $(printf '%s\n' "$_response" | jq -r '
    .error,
    .access_token,
    .token_type,
    .expires_in,
    .refresh_token,
    .scope')
IFS=$oifs

[ "$1" = "null" ] && ...

That test ends up comparing "null\r" to "null".

8
  • 2
    In Cygwin or any system in scope of this site, the line delimiter should be LF. It sounds like the question should rather be how to process files from foreign / unusual operating systems where the line delimiter is not LF. Note that the behaviour of bash for IFS=$'\r\n' changed in recent releases in that CR (along with all single-byte whitespace characters, not just SPC, TAB, NL like in most shells) is now considered as an IFS-white-space character. Aug 16, 2021 at 6:42
  • @StéphaneChazelas if the OP is running Cygwin then files containing CR/LF line endings must be expected. Similarly running UNIX commands on a Mac one should expect CR line endings. Not convenient, but reality Aug 16, 2021 at 7:35
  • 1
    DanielKaplan can you preprocess the files with dos2unix? Either modifying the files in-place or using it as an input folder. This might be your easiest approach Aug 16, 2021 at 7:38
  • @roaima, AFAIK, cygwin text files are LF-delimited, though cygwin tools might be given to process files from the foreign non-POSIX host OS (Windows) which are CRLF delimited. Just like a Linux-running file server might be given to process CRLF delimited files dumped by Windows clients, or Ubuntu desktop OSes might be given to process CRLF delimited files from a Windows file server. Still in all those cases, the OS (Cygwin, Ubuntu...) is a POSIX system so should have LF delimited text files. Aug 16, 2021 at 7:49
  • 1
    @roaima as far as I know, macOS, which is a UNIX after all, uses the standard LF. The old apple OS used to have CR but that hasn't been the case in what, 20 years or so? I don't think we need to worry about CR line endings any more.
    – terdon
    Aug 16, 2021 at 10:09

2 Answers 2

1

There is a spot in the script where it sets the $IFS to \n [...]

The code you've shown us uses IFS=\n. Unfortunately this is not IFS='\n' and only ends up setting IFS=n. So the code needs fixing anyway

1
  • ugh, sorry. I thought that might be a problem. That was a variable I modified. The script is most likely doing it the right way. Aug 16, 2021 at 20:02
0

Rather than determining what kind of line endings your system uses, another option that might work could be to focus on what kind of line endings your input file uses.

I say this, because it's pretty easy (programmatically) to change the line endings in a file, either as it's written or by post-processing (via dos2unix or similar).

You can use the file command to give some information about the line endings a file is using.

tim@chaos:~/tmp$ file lnx.txt
lnx.txt: ASCII text

tim@chaos:~/tmp$ file win.txt
win.txt: ASCII text, with CRLF line terminators

tim@chaos:~/tmp$ dos2unix -n win.txt win2unix.txt
dos2unix: converting file win.txt to file win2unix.txt in Unix format...

tim@chaos:~/tmp$ file win2unix.txt
win2unix.txt: ASCII text

So, the script that is taking the input files could be set to check for the different line endings, and modify them on the fly, or it could simply take any input files and convert them to unix line endings via dos2unix.

The difference is just \r\n compared to \n. So processing the input stream in the shell script could conceivably catch any CRLF type line endings without having to do any file conversions.

However your shell script is choosing to read files line by line to process them, you could just add an extra sed step.

You could do:

IFS=\n
# Word splitting is intended here
# shellcheck disable=2046
set -- $(printf '%s\n' "$_response" | sed -e s'/\r//g' | jq -r '
    .error,
    .access_token,
    .token_type,
    .expires_in,
    .refresh_token,
    .scope')
IFS=$oifs

[ "$1" = "null" ] && ...

That would remove CRLF if they exist and not change input lines if they don't.

By having the script automatically convert different line endings on the input, then it really doesn't matter what kind of line endings the file has, and your script then becomes much more portable. Or at least, input agnostic.

1
  • Note that IFS=\n is the same as IFS=n, which is unlikely what you want.
    – Kusalananda
    May 2, 2023 at 14:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .