Bash script is not working when using case statement to make decision

Question

I have the following script that will schedule a job based on input from the file "schedule.csv". The file contains 7 fields and the $f7 field contains a flag that either has "NOT_S" for a job that needs to be scheduled and "YES" if the job has already been scheduled.

#!/bin/bash
filename='/home/opc/3A-Lab/schedule.csv'

i=1
while IFS=, read -r f1 f2 f3 f4 f5 f6 f7; do

        case "$f7" in

        NOT_S)
                test  $i -eq 1 && ((i=i+1)) && continue
                echo "/home/opc/3A-Lab/3ALab.sh $f5 start" | at  now
                echo "/home/opc/3A-Lab/3ALab.sh $f5 stop"  | at  now + "$f6" " " "hours"
                ;;
        YES)
                echo "Already Scheduled"
                ;;
esac
done < $filename

echo "something is wrong"

When I run the script, I get the output of "something is wrong" which is something I added a troubleshooting step to alert me that for some reason the logic I have is not working.

What I want is the jobs that have the flag "NOT_S" to be scheduled and the ones that have "YES" to be ignored. Without the decision logic, the jobs will get scheduled as desired. However, the problem with this is that I then have no way of excluding the jobs that have already be scheduled once the script runs again. The file it's reading from is generated by a Google Form and I'm downloading the responses from Google Sheets via wget. I currently have Google Sheets setting the flag in column $f7 so that after a period of time, when the file is downloaded again and the script is run again on the server, it will only process the jobs that have been added to the sheet that have the "NOT_S" flag and ignore the other ones.

What am I doing wrong in my script as to why its not picking up on the value of $f7 and executing the commands to schedule the jobs? Thanks for any help you can provide.

EDIT:

Here is sample of what the schedule.csv file looks like:

Timestamp,Email Address,First Name,Last Name,Workstation Name,Duration (in hours),Schedule Flag
,,,,no workstation,,YES
1/1/2021 14:52:11,[email protected],Jone Doe,no workstation,2,NOT_S
1/1/2021 15:39:48,[email protected],jane,sue,no workstation,2,NOT_S
1/1/2021 15:40:26,fred.flintstone,Fred,Flintsone,no workstation,5,YES
,,,,no workstation,,YES
,,,,no workstation,,
,,,,no workstation,,

So let me explain the file: A student comes to the Google form to book time for a virtual workstation. The form automatically retrieves their email address. Based on their email address, the sheet will populate with a workstation name associated with the student. Then the form asks them how much time do they want to reserve on the workstation. They can select between 1 to 5 hours from a drop down menu. Once that is complete, the form saves their entry onto the google sheet.

The sheet has a formula that will determine that uses the timestamp time +10 minutest to set the flag in field 7 to either NOT_S or YES. Once the 10 minutes are up, the flag gets set to YES. At the time of scheduling, it gets set to NOT_S.

Here is a sample output of the script scheduling the jobs without the conditional logic:

job 13984 at Sat Jan  2 14:41:00 2021
syntax error. Last token seen: hours
Garbled time
job 13985 at Sat Jan  2 14:41:00 2021
job 13986 at Sat Jan  2 16:41:00 2021
job 13987 at Sat Jan  2 14:41:00 2021
job 13988 at Sat Jan  2 16:41:00 2021
job 13989 at Sat Jan  2 14:41:00 2021
job 13990 at Sat Jan  2 19:41:00 2021
job 13991 at Sat Jan  2 14:41:00 2021
syntax error. Last token seen: hours
Garbled time
job 13992 at Sat Jan  2 14:41:00 2021
syntax error. Last token seen: hours
Garbled time
[opc@vm-control-server 3A-Lab]$ 

2nd EDIT:

Here is the output using the commands Gordon suggested:

[opc@vm-control-server 3A-Lab]$ file ./schedule.csv 
./schedule.csv: ASCII text, with CRLF line terminators
[opc@vm-control-server 3A-Lab]$ LC_ALL=C cat -vet ./schedule.csv 
Timestamp,Email Address,First Name,Last Name,Workstation Name,Duration (in hours),Schedule Flag^M$
,,,,no workstation,,YES^M$
1/1/2021 14:52:11,[email protected],Philip,Monroe,no workstation,2,NOT_S^M$
1/1/2021 15:39:48,[email protected],John,Crocket,no workstation,2,NOT_S^M$
1/1/2021 15:40:26,[email protected],Eddie,Reed,no workstation,5,YES^M$
,,,,no workstation,,YES^M$
,,,,no workstation,,^M$
,,,,no workstation,,[opc@vm-control-server 3A-Lab]$ 

Edit 3:

Here is the updated file I'm running the script on:

[opc@vm-control-server 3A-Lab]$ LC_ALL=C cat -vet ./schedule.csv 
Timestamp,Email Address,First Name,Last Name,Workstation Name,Duration (in hours),Schedule Flag^M$
,,,,no workstation,,YES^M$
1/1/2021 14:52:11,[email protected],Philip,Monroe,no workstation,2,NOT_S^M$
1/1/2021 15:39:48,[email protected],John,Crocket,no workstation,2,NOT_S^M$
1/1/2021 15:40:26,[email protected],Fred,Brown,no workstation,5,YES^M$
,,,,no workstation,,YES^M$
,,,,no workstation,,^M$
,,,,no workstation,,[opc@vm-control-server 3A-Lab]$ 

Output of the script. Keep in mind, this running without any decision logic, so it's not taking into account f7 field so it's scheduling all jobs on the sheet.

[opc@vm-control-server 3A-Lab]$ ./final2.sh
job 13993 at Sat Jan  2 15:38:00 2021
syntax error. Last token seen: hours
Garbled time
job 13994 at Sat Jan  2 15:38:00 2021
job 13995 at Sat Jan  2 17:38:00 2021
job 13996 at Sat Jan  2 15:38:00 2021
job 13997 at Sat Jan  2 17:38:00 2021
job 13998 at Sat Jan  2 15:38:00 2021
job 13999 at Sat Jan  2 20:38:00 2021
job 14000 at Sat Jan  2 15:38:00 2021
syntax error. Last token seen: hours
Garbled time
job 14001 at Sat Jan  2 15:38:00 2021
syntax error. Last token seen: hours
Garbled time
[opc@vm-control-server 3A-Lab]$ 

Edit 4:

Olivier's modifications are working great. However, I need to solve another problem that I hope you all can help me with. So the google sheet that I'm collecting the responses from the google form has a formula on it that checks the timestamp of the requested entry + 10 minutes to see if its >= to the now() time and if so, it changes the flag to YES. Well it works great on the form and ther flag gets changed, however, when my cron job pulls the next wget of the file, it doesn't recognize the changes to the f7 field and still has the entry as NOT_S so my script will continue to schedule jobs that have already been scheduled.

What I ultimately want to do is as soon as the job is scheduled, immediately set the flag to YES so that once the file gets read again, it will get ignored.

I was trying to figure out how to use wget to only pull in the changes to the sheets file and not have to download the whole thing each time. I want to set up something where I can write out to a local file on the Linux server and just have the script update that local file with the flag change and only take in the new entries from the google sheet. I would appreciate some help with this. Thanks.

Answer 1

Your input files have ^M (cr) characters before the unix newline (lf). Therefore, your f7 field is not what is expected.

Pass the file through: dos2unix < oldfile > newfile

or : tr -d '\015' < oldfile > newfile

And try again.

Make also sure:

• all input lines have 7 arguments (empty or not, ie have 6 commas): awk -F',' '(NF != 7) { print "line: " NR " has " NF " arguments : " $0 }' /home/opc/3A-Lab/schedule.csv
• add a default line for your case to notify which lines do not correspond to either cases: *) printf "Something is wrong: We have a line with an unexpected f7='%s'\n" "$f7" ;; # before the esac.
• delete the last echo : it would be always displayed, whatever the while does. or just modify it to say : echo "End of the script."
• and the line with hours has an extra empty arguments: change at now + "$f6" " " "hours" into at now + "$f6" hours

So the script becomes:

#!/bin/bash
filename='/home/opc/3A-Lab/schedule.csv'

i=0
while IFS=, read -r f1 f2 f3 f4 f5 f6 f7; do
    i=$(( i + 1 ))
    if [ "$i" = "1" ]; then printf "Bypass first line\n" ; continue ; fi
    if [ -z "$f1" ]; then printf "Bypass line: %s, empty field f1\n" "$i" ; continue ; fi
    case "$f7" in
    NOT_S)
            printf "/home/opc/3A-Lab/3ALab.sh %s start\n" "$f5" | at  now
            printf "/home/opc/3A-Lab/3ALab.sh %s stop\n"  "$f5" | at  now + "$f6" hours
            printf "scheduled start and stop for line: %s\n" "$i"
            ;;
    YES)
            printf "line %s: YES : Already Scheduled\n" "%i"
            ;;
    "")     printf "Warning: I see an empty f7... for line: %s =  '%s,%s,%s,%s,%s,%s,%s'\n" "$i" "$f1" "$f2" "$f3" "$f4" "$f5" "$f6" "$f7"
            ;;
    *)
            printf "Something is wrong: line %s: we see f7='%s'\n No action taken.\n" "$i" "$f7"
            ;;
    esac
done < $filename

printf "End of the Script.\n"

I am also puzzled as to why you don't want to do anything with the first NOT_S line encountered (ie, where i=1) ... But you probably have your reasons.

edit1: changed echo to printf everywhere, good habit to take (printf is portable, echo isn't.) edit2: changed the script following the comment about bypassing title and comments (and the reason for $i)

Answer 2

As has been explained elsewhere, your file's lines are terminated with Windows-style CRLF endings. This means that your last field always ends with a CR character, so your literal NOT_S and YES can never match.

You can either pass the file through a convertor, or more easily just ignore trailing characters on the end of the last field:

case "$f7" in
    NOT_S*) ... ;;
    YES*) ... ;;
esac