5

I have a csv file that contains 3 fields per line.

firstname,lastname,url

I'm trying to access the url via the following pipeline:

grep theName file.csv | cut -d, -f 3

then I want to add another pipe and use the results of the cut command in a curl command like so:

grep theName file.csv | cut -d, -f 3 | curl > result.txt

problem is, when i do the above, the curl command throws an error, i assume because curl doesn't have an argument?

how can I use the result of cut to curl the resulting url? Thanks in advance. =)

  • By the way, this assumes your CSV fields have no internal commas. Be careful. – msh210 Nov 16 '16 at 11:25
  • Be aware that URLs may contain commas. – Stig Hemmer Nov 17 '16 at 8:42
7

Leverage command substitution, $():

curl "$(grep ... | cut -d, -f 3)"

Here $() will be substituted by the STDOUT of the command inside $() i.e. grep ... | cut -d, -f 3, as this is done by the shell first so the curl command would be finally:

curl <the_url>
  • Duh. i should have known that and it didn't even occur to me. Well done. this works perfectly. It won't let me mark this correct yet. I'll try to remember and come back to mark your answer as correct. – wdowling Nov 16 '16 at 5:14
  • @wdowling, should be long enough now. – Wildcard Nov 16 '16 at 5:30
  • I know that few users use anything other than bash these days, but this is a bashism. The portable answer is curl `grep ... | cut -d, -f 3` - although I much prefer the xargs solution, it's more flexible in general. – reinierpost Nov 16 '16 at 8:55
  • @reinierpost No, $() is defined by POSIX. – heemayl Nov 16 '16 at 8:58
  • OK, thanks, it's a POSIXism then. I'm not sure how many non-POSIX-compliant shs are still out there. I just checked and dash and ash both support it. – reinierpost Nov 16 '16 at 8:59
5

Another solution without substitution:

grep theName | cut -d, -f 3 | xargs curl > result.txt
  • This answer works just as well as the other answer and i like that the flow doesn't change. My only question is, is xargs supported cross platform? I routinely go back and forth between my linux box and a mac, so I always want to make sure i don't get in the habit of doing something that I can't do on one machine. Thanks for your quick response! – wdowling Nov 16 '16 at 5:16
  • 1
    @wdowling, it's portable but still a bad habit to get into. See the last paragraph of this answer. – Wildcard Nov 16 '16 at 5:34
  • xargs is very widely available. – reinierpost Nov 16 '16 at 8:54
  • xargs is better than command substitution in case your input has funny characters - it has options to deal with them, e.g. I like to provide -d'\n'. – reinierpost Nov 16 '16 at 8:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.