bash shell script errors with: syntax error near unexpected token `$'do\r''

Question

I have written the script below, but when I run it from the directory where it is (and also the files)

bash xmlimport-magento2.sh

I get this error message:

xmlimport-magento2.sh: line 7: syntax error near unexpected token $'do\r''

mlimport-magento2.sh: line 7: do

#!/bin/bash
if exists *.ZIP
then
    # disable autoindexing
    #php -f "$MAGEPATH"shell/indexer.php -- --mode-manual all
    for file in `ls *.ZIP`
    do
        # Backup zip file
        cp $file /backups
        # Extract XML files
        unzip -P web $file
        # Delete zip file
        rm -f $file
        # rename xml files
        # hier komt procedure die controleert of bestand xxx*basic.xml bestaat en zo ja dan wordt dit bestand hernoemt en daarna geexecuteerd
        if [ -f "*basic.xml" ]
        then
            mv *_BASIC.XML BASIC.XML
            php -f bin/magento import:job:run 1
        fi
        # rename xml files
        # hier komt procedure die controleert of bestand xxx*prices.xml bestaat en zo ja dan wordt dit bestand hernoemt en daarna geexecuteerd
        if [ -f "*prices.xml" ]
        then
            mv *_PRICES.XML PRICES.XML
            php -f bin/magento import:job:run 4
            php -f bin/magento import:job:run 2
        fi
        # rename xml files
        # hier komt procedure die controleert of bestand xxx*stock.xml bestaat en zo ja dan wordt dit bestand hernoemt en daarna geexecuteerd
        if [ -f "*stockinfo.xml" ]
        then
            mv *_STOCKINFO.XML STOCKSTOCK.XML
            php -f bin/magento import:job:run 3
        fi
    done
    # verplaats .XML files naar backup
        cp -f $file ../backups
    # Delete XML files
        rm -f *.XML
fi

Answer 1

First, there's the obvious issue with CRLF newlines, the \r in the error message being a tell-tale sign. See e.g. How to bulk convert all the file in a file system branch between Unix and Windows line break format? on how to fix that.

Then, you have

if exists *.ZIP
then

which might also be DOS-ism, at least I've not heard of a utility called exists.

Also here:

for file in `ls *.ZIP`

is a useless use of ls. What happens is that the shell expands the filename pattern *.ZIP, passes the results to ls, which the... just prints the same names out. Except that you run into issues with word splitting of the result.

Instead of those two, use:

shopt -s nullglob
for file in ./*.ZIP; do
    ...
done

First, the shell is quite capable of listing the matching filenames (it's what you did above with exists *.ZIP). Then, Bash's nullglob makes the pattern expand to nothing if there are no matching files. Otherwise you might get errors about a file literally called *.ZIP not existing. Prefixing the pattern with ./ makes some potential problems with filenames starting with dashes go away.

Also, you used unquoted expansions of $file all the way, i.e. rm -f $file, instead of rm -f "$file". That, too, leads to issues with word splitting, see:

• When is double-quoting necessary?
• Why does my shell script choke on whitespace or other special characters?

Also,

if [ -f "*basic.xml" ]

this doesn't work, because the quotes prevent the pattern from being expanded to actual filenames. It would look for a file literally called *basic.xml. If you're sure there can only be one match, then if [ -f *basic.xml ] could do. If there are more, that would give an error because the test would get too many arguments.

Also, this will probably give an error, unless BASIC.XML is an existing directory.

mv *_BASIC.XML BASIC.XML

Okay, I think the suggestion to write the script in smaller parts, and test them individually (possibly with some debugging output sprinkled in), would be a good idea.