1

First of all, yes, locked into csh on a Solaris box, can't do anything about it, sorry.

I have a report batch I was running using a foreach loop. Right now it runs as a single thread and I would like to speed it up with GNU parallel. I have been trying two different approaches but hitting roadblocks on each.

Here is my current version:

if( $#argv <=  1) then
        #Get today's date
        set LAST = `gdate +%Y-%m-%d`
else
        #use date passed in parameter
        set LAST=`echo $2 | tr -d _/`; 
endif

if( $#argv == 0  ) then
        #default to 15 day lookback
        set COUNT = 15
else
        #otherwise use parameter
        set COUNT = $1
endif

@ LCOUNT = $COUNT + 1 #increment by one to exclude $LAST date
#get starting date by subtracting COUNT (now incremented by 1)
set START = "`gdate --date='$LAST -$LCOUNT day' +%Y/%m/%d`";

#loop through dates, generate report string, and pipe to reportcli
foreach i (`seq $COUNT`)
    set REPDATE = "`gdate --date='$START +$i day' +%Y/%m/%d`";
    set FILEDATE = "`gdate --date='$START +$i day' +%Y%m%d`";
    echo "runf reportname.rep -ps "$REPDATE" -pe "$REPDATE" -o report_"$FILEDATE".csv" \ 
       | reportcli <cli params here>
end

So I would like to get this working with parallel, but as you can see I have a boatload of command expansion/substitution going on.

I tried a few different approaches, including making an array of the string passed to the reportcli, but I can't figure out how to get it to play nice.

As I see it, I have two choices:

A) one big line (have to iron out all the quoting problems to get the gdate command substitution to work):

`seq $COUNT` | parallel reportcli <cli params> < "runf reportname.rep -ps \
  `gdate --date='$START +{} day' +%Y/%m/%d` -pe `gdate --date='$START +{} day' +%Y/%m/%d` \
  -o report_`gdate --date='$START +${} day' +%Y%m%d`.csv" 

B) Assemble a csh array beforehand, then try to expand the array (expand with echo?), pipe to parallel

set CMDLIST
foreach i (`seq $COUNT`)
    set REPDATE = "`gdate --date='$START +$i day' +%Y/%m/%d`";
    set FILEDATE = "`gdate --date='$START +$i day' +%Y%m%d`";
    set CMDLIST = ($CMDLIST:q "runf reportname.rep -ps "$REPDATE" -pe "$REPDATE" \
       -o report_"$FILEDATE".csv")
end

I know my array is good because I can do this and get back each element:

foreach j ($CMDLIST:q)
    echo $j
end

but, I'm not sure how to get this to work in csh:

echo $CMDLIST | parallel --pipe "reportcli <cli params here>"

Thanks in advance!!

0

Write a script. Call that from GNU Parallel:

[... set $START and $COUNT ...]

seq $COUNT | parallel my_script.csh $START {}

my_script.csh:

#!/bin/csh

set START = $1
set i = $2

set REPDATE = "`gdate --date='$START +$i day' +%Y/%m/%d`";
set FILEDATE = "`gdate --date='$START +$i day' +%Y%m%d`";
echo "runf reportname.rep -ps "$REPDATE" -pe "$REPDATE" -o report_"$FILEDATE".csv" \ 
   | reportcli <cli params here>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.