0

I have a .CSV file that I need to extract numbers from. Example of file:

#File of numbers
1,2,3,4,5
6,7,8,9,10

I want to use awk (or another utility) to extract the first number and second number and store them in a variable to perform calculations within my bash script. I want to exclude the first line that has # at the beginning.

Once I perform the calculation I want to extract the third number to continue calculations and repeat until I have gotten all numbers. I don't know how to start this and would appreciate some help. I know that I will need to put the awk script inside a loop to get number three, four, etc. but I'm confused with how to write the awk script.

  • 3
    If you're going to "perform calculations" then it's probably best to do that within awk - rather than extracting them back to the shell and doing so there – steeldriver Mar 15 '18 at 22:53
  • I understand that but the requirements of the assignments are to have two shell scripts interacting with each other (which I've figured out) but I need to use awk or some other utility to extract the numbers. – Nasir Mar 15 '18 at 23:26
  • 1
    The shell is a beautifully functional utility orchestrator. It is a horribly inefficient programming language. Use it for what it's good for—calling other utilities and making them work together. Don't try and do any actual work with Bash itself. (In other words: listen to @steeldriver and do your calculations in awk or, if you're feeling rebellious, in bc.) – Wildcard Mar 16 '18 at 0:01
  • What do you want these calculations to do? It may be better to exclude awk and bash altogether and use an actual programming language such as python for your needs. – Nasir Riley Mar 16 '18 at 1:35
2

Using a real csv parser in a :

#!/bin/bash

csvcut -c 1,2 -d, file.csv |
while IFS=, read -r c1 c2; do
    echo $((c1 * c2)) # do some arithmetic with variables c1 & c2
done

Check csvkit

  • csvkit, not cvskit (couldn't offer edit becuase less than six characters...) – user1404316 Mar 16 '18 at 10:46
  • Thanks guy, post edited accordingly. Not worst time to upvote if i'ts interesting for you ;) – Gilles Quenot Mar 16 '18 at 19:01
1

The only way to solve this by going back and forth between awk and the shell is to let AWK scripts generate the shell code for the next step. But this sounds way too complicated to me. I also would recommend moving everything into awk (or would perl also be allowed here?)

  • I agree, it is very complicated. The assignment requires this though. I don't have to necessarily use AWK but I thought it would be the easiest utility to get the job done. The assignment requires script 1 to get the numbers from the .txt file and then store them into a separate file which script 2 then calculates and prints the results. Any suggestions on how to approach this? @Wolfgang Liebich – Nasir Mar 16 '18 at 13:57
  • The next-simplest approach would be to split the line in bash (as mentioned below), and then maybe use bc to run your calculation. – Wolfgang Liebich Mar 19 '18 at 7:05
0

This is pretty straightforward in bash and other shells:

while IFS=, read a b c d e ; do
  your_stuff
  done < your_csv_file
  1. IFS is the input field separator. set it for the read command only to be a comma.

  2. read five variables at a time.

  3. Do your stuff with your brand new shell variables $a $b $c $d $e.

  4. You must define the input re-direction at the end of the while loop, after the done keyword.

  • What about input like foo, "bar,base", xxx ? BOOOOM ! And now you hope a real csv parser ;) – Gilles Quenot Mar 16 '18 at 20:27
  • @GillesQuenot : But this isn't some theoretical classroom open-ended question dealing with unknown input. The OP knows the exact format of the input and has told us it is csv numeric, and has given us an example. – user1404316 Mar 16 '18 at 20:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.