4

I was trying to parse some nginx config

λ tree sites-enabled/ sites-available/
sites-enabled/
├── bank.cwrcoding.com.conf
├── calendar.cwrcoding.com.conf
├── cloud.cwrcoding.com.conf
├── cwrcoding.com.conf
├── drive.cwrcoding.com.conf
├── groups.cwrcoding.com.conf
├── mail.cwrcoding.com.conf
├── sites.cwrcoding.com.conf
├── studentenverwaltung.cwrcoding.com.conf
├── wekan.cwrcoding.com.conf
└── www.cwrcoding.com.conf
sites-available/
├── bank.cwrcoding.com.conf
├── calendar.cwrcoding.com.conf
├── cloud.cwrcoding.com.conf
├── cwrcoding.com.conf
├── drive.cwrcoding.com.conf
├── groups.cwrcoding.com.conf
├── mail.cwrcoding.com.conf
├── sites.cwrcoding.com.conf
├── studentenverwaltung.cwrcoding.com.conf
├── wekan.cwrcoding.com.conf
└── www.cwrcoding.com.conf

The sites-enabled/* files each contain a single line:

include sites-availabe/cwrcoding.com.conf;

When trying to iterate over the sites-enabled/* files, cutting those and trying to read their contents as files, I got some weird error, so I tried a minimalistic working solution, and working my way up from there, but yet the following happens:

λ for enabled in sites-enabled/*
> do
> echo "$(cat "$enabled") |"
> echo ==========
> done
include sites-available/bank.cwrcoding.com.conf; |
==========
 |clude sites-available/calendar.cwrcoding.com.conf;
==========
 |clude sites-available/cloud.cwrcoding.com.conf;
==========
 |clude sites-available/cwrcoding.com.conf;
==========
 |clude sites-available/drive.cwrcoding.com.conf;
==========
 |clude sites-available/groups.cwrcoding.com.conf;
==========
 |clude sites-available/mail.cwrcoding.com.conf;
==========
 |clude sites-available/sites.cwrcoding.com.conf;
==========
 |clude sites-available/studentenverwaltung.cwrcoding.com.conf;
==========
include sites-available/wekan.cwrcoding.com.conf; |
==========
 |clude sites-available/www.cwrcoding.com.conf;
==========

As you can see, for most of the sites the first characters of the cat output are replaced by the text supposed to be after the command substitution.

Can anyone explain what is happening? Or have I found some bug?

If you want to take a look at the files: https://github.com/cwrau/nginx-config

  • This doesn't really solve the question you're asking, but depending on exactly how you want the output formatted, you might want to take a look at the paste program. – David Z Jan 8 '18 at 23:09
9

The problem is that your files have DOS/Windows-style line-endings. As a quick work-around, replace:

echo "$(cat "$enabled") |"

With:

echo "$(tr -d '\r' <"$enabled") |"

Here, tr removes the carriage-return character before the file is displayed, avoiding the problem.

If your files are intended to be used on a Unix system, however, you would be better off removing the carriage-returns from the files themselves using one the dos2unix or similar utilities.

Example

Let's create two DOS-style files:

$ echo 'include sites-availabe/cwrcoding.com.conf;' | unix2dos > sites-enabled/file1
$ echo 'include sites-availabe/cwrcoding.com.conf;' | unix2dos > sites-enabled/file2

Let's run the original command:

$ for enabled in sites-enabled/*; do echo "$(cat "$enabled") |"; echo ==========; done
 |clude sites-availabe/cwrcoding.com.conf;
==========
 |clude sites-availabe/cwrcoding.com.conf;
==========

Note the mangled output.

With tr applied, we receive the output that we expect:

$ for enabled in sites-enabled/*; do echo "$(tr -d '\r' <"$enabled") |"; echo ==========; done
include sites-availabe/cwrcoding.com.conf; |
==========
include sites-availabe/cwrcoding.com.conf; |
==========
  • 1
    Thank you, that's exactly it! Now to remove those pesky carriage-returns :D – Chris Werner Rau Jan 8 '18 at 19:43
  • 2
    Look into dos2unix. – DopeGhoti Jan 8 '18 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.