0

So I am fairly new to bash scripting, but I want a bash job that will:

Create a post-dated timestamp. (either as a var, or to a file)

Read lines from a file named feeds.txt.

Separate the number string from the whole string into two variables $feedNum and $wholeString.

Then execute a cat command to create or append a new file named $wholeString append the contents of timestamp.txt to that file.

Lastly, execute a wget command to insert $feedNum into the url string, and output the response of the wget to the previously created $wholeString file.

Here's what I have.

feeds.txt looks like (it will eventually be longer):

8147_feed_name.xml
19176_nextfeed_name.xml

And the script I've cobbled together looks like this.

#Changes the directory to the correct location
cd /home/scripts
# Inserts the proper timestamp in the file , postdating for one hour.
date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/timestamp.txt

#Loop Logic
for feedNum in `cat feeds.txt |cut -d _ -f 1`;do
   for  wholeString in `cat feeds.txt`; do
    cat /home/scripts/timestamp.txt >>/home/scripts/tmp/$wholeString ;
    wget https\://some.url.com/reports/uptimes/${feedNum}.xml?\&api_key=KEYKEYKEYKEY\&location=all\&start_date=recent_hour -O ->>/home/scripts/tmp/$wholeString;
done
done

My issue is, that it runs 4 times, and so if I ditch cat and wget and replace them with a simple echo, like this,

#Changes the directory to the correct location
cd /home/scripts
# Inserts the proper timestamp in the file , postdating for one hour.
date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/timestamp.txt

#Loop Logic
for feedNum in `cat feeds.txt |cut -d _ -f 1`;do
   for  wholeString in `cat feeds.txt`; do
    echo $feedNum $wholeString
done
done

I get an output like this, where the top and bottom lines are correct. The middle two are a mix-mash.

8147 8147_feed_name.xml
8147 19176_nextfeed_name.xml
19176 8147_feed_name.xml
19176 19176_nextfeed_name.xml

I understand why it is doing this, but not how to fix it.

  • if $feedNum is unique, you could grep it from the file in the inner loop. Or, reorganize your loop to read feeds.txt line-by-line and cut out the two pieces you want inside one for loop. – Jeff Schaller May 12 '16 at 17:14
1

This should do the trick for splitting the string (though as always, be wary of spaces in your filenames):

#Changes the directory to the correct location
cd /home/scripts
# Inserts the proper timestamp in the file , postdating for one hour.
date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/timestamp.txt

#Loop Logic
while IFS=_ read feedNum xmlFile; do
  echo "$feedNum $xmlFile"
done < feeds.txt

It should be trivial to assemble your wget call from here.

  • Why not skip the cuts and use while IFS=_ read -r feedNum xmlFile; do ... ? – steeldriver May 12 '16 at 18:12
  • Because I didn't think of it at the time? I'll roll that in; thanks. – DopeGhoti May 12 '16 at 18:14
  • ... also might be worth suggesting using a variable (i.e. command substitution rather than redirection to a file) for the date string? – steeldriver May 12 '16 at 18:17
  • I thought of it, but wanted to keep it at least superficially similar to the OP's script to allow it to be more clear where I was going. timestamp=$(date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S") isn't a terrible idea though- why write to disk when you don't need to? – DopeGhoti May 12 '16 at 18:35
0

OP Here, as a different user name. I ended up using the first code you posted DopeGhoti. With some slight modification. Thanks for showing me how to grab a line, and then execute from it rather than trying to re-read the source file and pull for each.

 #Create Timestamp
    date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S" >/home/scripts/sitename/timestamp.txt

    #Loop Logic
    while read line; do
       feedNum="$(echo "$line" | cut -d_ -f1)"
       wholeString="$(echo "$line")"
       cat /home/scripts/sitename/timestamp.txt >>/home/scripts/sitename/$wholeString
       wget https\://my.sitename.com/reports/uptimes/${feedNum}.xml?\&api_key=KEY\&location=all\&start_date=recent_hour -O ->>/home/scripts/sitename/$wholeS$
    done < feeds.txt

Which resulted in the correct output for all 18 feeds we had.

Thank you!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.