I have a CSV with a list of users, and would like to add a column with a one-time-use randomly generated password, unique to each user.

My script works... but then it just keeps going indefinitely adding rows. If I move the code to set the variable out of the loop, it works just fine, but then every user gets the same password.

How do I get this to terminate on the last row?

#add column to csv
{ echo `head -1 $ORIG_FILE`",One Time Password" ; tail -n +2 $ORIG_FILE | \
  while read x ; OneTimePass=$(openssl rand -base64 14 | head -c 6) ; do echo "$x,$OneTimePass" ; done ; } > $NEW_FILE
  • 1
    Welcome to the site. Please edit your post to include an (anonymized) example of the CSV file before the operation, with the corresponding desired result of the operation. Also, consider including the undesired result that your code produces at the moment.
    – AdminBee
    Jul 29 at 14:37
  • Always paste your script into https://shellcheck.net, a syntax checker, or install shellcheck locally. Make using shellcheck part of your development process.
    – waltinator
    Jul 31 at 0:50

You have the syntax of your while...do...done loop wrong. You are running this:

while read x ; OneTimePass=$(openssl rand -base64 14 | head -c 6) ; do ...

The while keyword's expected format is explained in help while:

$ help while
while: while COMMANDS; do COMMANDS; done
    Execute commands as long as a test succeeds.
    Expand and execute COMMANDS as long as the final command in the
    `while' COMMANDS has an exit status of zero.
    Exit Status:
    Returns the status of the last command executed.

Here, you are giving it two commands: read x ; and OneTimePass=$(openssl rand -base64 14 | head -c 6), and the second command always works, it doesn't have any ending since you can always re-run the openssl command. This is why the while loop never exits and you just keep writing more and more lines. What you're after is this:

while read x; do

Here's a working version of your script with some other improvements such as proper quoting and avoiding CAPS for your variable names:


#add column to csv

printf "%s,%s\n" "$(head -1 "$orig_file")" "One Time Password" > "$new_file"
tail -n +2 "$orig_file" |
  while read -r x; do
    OneTimePass="$(openssl rand -base64 14 | head -c 6)"
    printf '%s,%s\n' "$x" "$OneTimePass"
  done >> "$new_file"

Personally, I would avoid hardcoding the file names since that makes the script harder to use and less versatile. I would instead take the input file name as an argument, and then print to standard output so you can choose whatever output file you want:


#add column to csv

printf "%s,%s\n" "$(head -1 "$orig_file")" "One Time Password"
tail -n +2 "$orig_file" |
  while read -r x; do
    OneTimePass="$(openssl rand -base64 14 | head -c 6)"
    printf '%s,%s\n' "$x" "$OneTimePass"

You can then run it like this:

foo.sh new-users2.csv > Output.csv 

I tested with this input file:

$ cat new-users2.csv 

Which results in:

$ foo.sh new-users2.csv 
name,age,One Time Password
  • You are awesome. Thank you so much! This was helpful on a number of levels.
    – Mr S
    Jul 29 at 15:03
  • 2
    You're very welcome @MrS! If this or any other answer here solved your issue, please take a moment and accept it by clicking on the checkmark on the left. That is the best way to express your thanks on the Stack Exchange sites.
    – terdon
    Jul 29 at 15:06

Using Miller, and cheekily borrowing @terdon's example file

$ mlr --csv put -S '
    $["One Time Password"] = substr(system("openssl rand -base64 5"),0,5)
' new-users2.csv 
name,age,One Time Password

(I changed the openssl rand parameter from 14 to 5 bytes since you should need no more than 36 bits to generate six base64 characters. Change it back if it is important for some reason.)

You can do a similar thing with awk's system() function but handling of the CSV structure and headers is messier. Perhaps neater with awk would be to use getline ex.:

$ awk -F, '
    BEGIN {OFS=FS; cmd = "openssl rand -base64 5"} 
    NR==1 {$(NF+1) = "One Time Password"} 
    NR >1 {cmd | getline var; close(cmd); $(NF+1) = substr(var,1,6)} 
' new-users2.csv
name,age,One Time Password

Unlike the Miller version, this will only work for "simple" CSV (it won't handle embedded "," characters within fields for example).

  • 1
    @rowboat thanks can't believe I didn't spot that - should be better now I think? Jul 30 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.