0

I have an issue with the script below. It uses a few sql statements in a for loop to set variables and produces a log file. The current db produces +- 1200 lines of output.

The original version is with the commented echo line, it adds one line to a text file (report.csv) for each loop (one write operation to disk).

The new intended version would append the data to an array (dcimlog), and then do only one write operation for the whole dataset in newreport.csv.

My issue is that somehow the array is not appended correctly by the child processes, so newreport.csv only contains the first line done before the for loop, and is not appended by any child.

The order of the lines is not important, but the speed of execution is, hence the "do () &" -- which I believe might be the reason for the failure...

Any idea how I can get this to work?

N=$(nproc)
dcimlog=();dcimlog=("${dcimlog[@]}" "some stuff") # works
for i in "${sqlanswer[@]}"; do
    ( echo somework #get the data from sql
    #   echo "$DDeviceID;$NBPort;$PortNumber;$CConnectedDevice;$Connectedport;$CabinetLocation;$HeightDevice;$PositionDevice;$ZoneName;$DataCenterName" >> report.csv
    dcimlog=("${dcimlog[@]}" "$DDeviceID;$NBPort;$PortNumber;$CConnectedDevice;$Connectedport;$CabinetLocation;$HeightDevice;$PositionDevice;$ZoneName;$DataCenterName") # does not work
) &
    if [[ $(jobs -r -p | wc -l) -gt $N ]]; then
            wait -n
    fi
done
wait
echo "${dcimlog[@]}" > newreport.csv
Improve this question
4
  • 1
    Not sure I understand correctly, but variable / array assignments in a subshell won't be passed back to the parent shell. The echo redirected to the report file should work from the subshell, so why not leave it in?
    – RudiC
    Commented Oct 20, 2018 at 13:40
  • Because with the echo in the loop, I open the file for 1200 consecutives writes and close, instead of one, if I managed to get to echo everything at once
    – user1747036
    Commented Oct 20, 2018 at 14:33
  • 3
    Why don't you echo, and redirect the entire loop's stdout to the file? Like for FN in *; do (echo $BASHPID, $$, $FN) & done >result
    – RudiC
    Commented Oct 20, 2018 at 14:54
  • Thank you @RudiC , that worked like a charm! If you create an "anwser" for the question, I will select it as solver.
    – user1747036
    Commented Oct 20, 2018 at 15:15

1 Answer 1

Reset to default
1

Think about redirecting the entire loop's stdout to the report.csv file, and, within, just echo as you did before. Like

for i in "${sqlanswer[@]}"; do
    ( echo somework #get the data from sql
      echo "$DDeviceID;$NBPort;$PortNumber;$CConnectedDevice;$Connectedport;$CabinetLocation;$HeightDevice;$PositionDevice;$ZoneName;$DataCenterName"
    ) &
  done > report.csv
Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .