I am parsing a very large csv file and creating many sub-files with its contents according to the value of the first column. So far, what I have done is manually declare an array:

declare -a arr=(
"Value 1"
"Value 2"

and then I loop through this array and create the mini-csv files with the information I care about, also changing the name of the output csv files to swap spaces with underscores:

for i in "${arr[@]}"
    NAME=${i// /_}
    echo $header > "$outputpath/$NAME.csv"
    grep "^$i," "$fixed_file" | cut -d ',' -f 4,6 >> "$outputpath/$NAME.csv"

Is there a way to automate the first step as well? i.e., Can I loop through the original csv file and populate arr with all unique values of its column 1?


One way:

declare -A arr
while IFS=, read -r one rest
done < input.csv

Then loop over the keys:

for i in "${!arr[@]}"
  • I'm getting an error: Value Set Name: syntax error in expression (error token is "Set Name") -- I believe it's because the name of the first column is "Value Set Name" – nvergos Oct 3 '16 at 16:48
  • I can't reproduce that error; what code are you using? – Jeff Schaller Oct 3 '16 at 19:04
  • The column header is "Value Set Name" and the individual strings of that column are strings that include spaces most of the time, i.e., "Diabetic Retinal Screening" for the first 10 lines, "Acute Inpatient" for another 50 lines, etc. What I am aiming for is for the script to generate an array (arr) with each one of these unique strings as elements... – nvergos Oct 3 '16 at 19:58
  • This answer creates that array; do you get that error while building the array, or when using it? – Jeff Schaller Oct 3 '16 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.