4

I have a CSV file that contains usernames and passwords. The file looks something like this:

user1,password1
user2,password2
user3,password3

I need to loop through each line to grab the username and password, use those variables, and then grab the next set of usernames and passwords, replace the content of the variables with the new ones, etc.

I've been searching for the best way to do this, but I'm not very familiar scripting and I'm getting lost. I've used awk to grab both individually, but I'm struggling to figure out how to use awk within a while loop. And I'm reading that might not be a great approach.

3
  • 2
    You need to be very explicit about your requirements. What dows "replace the content of the variables" mean? Do you need to modify the file, or is the new value only relevant for the duration of the running program? Commented Jul 2, 2020 at 20:22
  • 1
    Can the passwords contain commas? If yes, how would they be encoded if at all? Commented Jul 3, 2020 at 15:06
  • The right answer all depends on what you mean by use those variables - if you post some concise, testable sample input and expected output then we can help you.
    – Ed Morton
    Commented Jul 4, 2020 at 13:11

3 Answers 3

8

I don't see any need for awk here:

#!/bin/bash

while IFS=, read -r user pass; do
  # something with "$user" "$pass"
done < path/to/file.csv
2
  • Perfect thanks! I tried read, but I couldn't figure out how to separate by the comma. Thanks again!
    – user420837
    Commented Jul 2, 2020 at 19:40
  • If use those variables from the question means call a tool or move a file or similar that requires manipulating files or processes than that's not a terrible approach but if it means change value x to y or similar that just requires manipulating text then that would be a terrible approach (see why-is-using-a-shell-loop-to-process-text-considered-bad-practice) so hopefully the OP understands the difference,
    – Ed Morton
    Commented Jul 4, 2020 at 13:07
8

Assuming you have some pretty good passwords:

user1,",3 ""e`$^~´"
user2,""")& Eu`id`"
user3,ThisIsAlsoAGoodPasswordBecauseItIsLong

Then you need something that can parse CSV ("" inside " is a ").

cat user+password.csv | parallel --csv do_stuff {1} {2}
7
  • One of these days, we'll find something that parallel can't do... ;P More seriously though, does parallel really need you to cat the file? That feels like a UUoC, is there no other way?
    – terdon
    Commented Jul 2, 2020 at 20:08
  • @terdon You know there are plenty of other ways :)
    – Ole Tange
    Commented Jul 2, 2020 at 20:12
  • Funnily enough, those passwords are no longer secure because they are available online. also, is the do_stuff a function? A command/alias? Or something else? Commented Jul 3, 2020 at 9:02
  • @IsmaelMiguel True. It works directly if it is a command. If it a function or an alias you use env_parallel instead of parallel.
    – Ole Tange
    Commented Jul 3, 2020 at 14:59
  • @IsmaelMiguel, the password also ends up being passed as argument to a shell and to do_stuff, which at least in the case of the shell, means it will be exposed in the output of ps. Commented Jul 3, 2020 at 15:22
1

If the file is a proper CSV file, you can use ksh93 instead of bash which has support for parsing CSVs:

#! /usr/bin/env ksh93
while IFS=, read -rS user password; do
  something with "$user" and "$password"
done

On an input like

user,"a""b
c,d"

That would correctly set $user to user and $password to a"b<LF>c,d.

2
  • How do you enter a password containing newline?
    – Ole Tange
    Commented Jul 23, 2020 at 6:18
  • @OleTange, that very much depends on the interface that inputs those passwords if any. For instance, if the password is retrieved as an environment variable, that can be PASSWORD=$'a"b\nc,d' some-command. With basic HTTP authentication, newline would be base64 encoded like any other character. In any case, that's hardly relevant to this Q&A. Commented Jul 23, 2020 at 6:38

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .