1

First of all, yes, locked into csh on a Solaris box, can't do anything about it, sorry.

I have a report batch I was running using a foreach loop. Right now it runs as a single thread and I would like to speed it up with GNU parallel. I have been trying two different approaches but hitting roadblocks on each.

Here is my current version:

if( $#argv <=  1) then
        #Get today's date
        set LAST = `gdate +%Y-%m-%d`
else
        #use date passed in parameter
        set LAST=`echo $2 | tr -d _/`; 
endif

if( $#argv == 0  ) then
        #default to 15 day lookback
        set COUNT = 15
else
        #otherwise use parameter
        set COUNT = $1
endif

@ LCOUNT = $COUNT + 1 #increment by one to exclude $LAST date
#get starting date by subtracting COUNT (now incremented by 1)
set START = "`gdate --date='$LAST -$LCOUNT day' +%Y/%m/%d`";

#loop through dates, generate report string, and pipe to reportcli
foreach i (`seq $COUNT`)
    set REPDATE = "`gdate --date='$START +$i day' +%Y/%m/%d`";
    set FILEDATE = "`gdate --date='$START +$i day' +%Y%m%d`";
    echo "runf reportname.rep -ps "$REPDATE" -pe "$REPDATE" -o report_"$FILEDATE".csv" \ 
       | reportcli <cli params here>
end

So I would like to get this working with parallel, but as you can see I have a boatload of command expansion/substitution going on.

I tried a few different approaches, including making an array of the string passed to the reportcli, but I can't figure out how to get it to play nice.

As I see it, I have two choices:

A) one big line (have to iron out all the quoting problems to get the gdate command substitution to work):

`seq $COUNT` | parallel reportcli <cli params> < "runf reportname.rep -ps \
  `gdate --date='$START +{} day' +%Y/%m/%d` -pe `gdate --date='$START +{} day' +%Y/%m/%d` \
  -o report_`gdate --date='$START +${} day' +%Y%m%d`.csv" 

B) Assemble a csh array beforehand, then try to expand the array (expand with echo?), pipe to parallel

set CMDLIST
foreach i (`seq $COUNT`)
    set REPDATE = "`gdate --date='$START +$i day' +%Y/%m/%d`";
    set FILEDATE = "`gdate --date='$START +$i day' +%Y%m%d`";
    set CMDLIST = ($CMDLIST:q "runf reportname.rep -ps "$REPDATE" -pe "$REPDATE" \
       -o report_"$FILEDATE".csv")
end

I know my array is good because I can do this and get back each element:

foreach j ($CMDLIST:q)
    echo $j
end

but, I'm not sure how to get this to work in csh:

echo $CMDLIST | parallel --pipe "reportcli <cli params here>"

Thanks in advance!!

1 Answer 1

0

Write a script. Call that from GNU Parallel:

[... set $START and $COUNT ...]

seq $COUNT | parallel my_script.csh $START {}

my_script.csh:

#!/bin/csh

set START = $1
set i = $2

set REPDATE = "`gdate --date='$START +$i day' +%Y/%m/%d`";
set FILEDATE = "`gdate --date='$START +$i day' +%Y%m%d`";
echo "runf reportname.rep -ps "$REPDATE" -pe "$REPDATE" -o report_"$FILEDATE".csv" \ 
   | reportcli <cli params here>
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .