3

Attempting to shorten a bash script that uses curl to fetch multiple API calls takes something like this:

curl --user $USER:$PASS https://api.example.com/foo -o 'foo.json'
curl --user $USER:$PASS https://api.example.com/bar -o 'bar.json'
curl --user $USER:$PASS https://api.example.com/baz -o 'baz.json'

And use it in this form:

curl --user $USER:$PASS https://api.example.com/{foo,bar,baz} -o '#1.json'

The issue is that curl is fetching foo, bar and baz but is not assigning the output to foo.json, bar.json and baz.json. It is literally creating #1.json and piping output to stdout. Has been tried with single, double, and no quotes, all same result. This is being run inside a bash script, although the curl command behaves the same way when entered directly on the command line. Is this an OS X syntax issue?

  • 2
    What is it you think the #1 syntax is supposed to do? Where does that come from? Anyway you're almost certainly better off running three different curl invocations in a loop. – Celada Apr 17 '16 at 21:10
  • 1
    The #1 syntax is described in the curl man page, if you're curious. – larsks Apr 17 '16 at 21:11
  • @larsks yes I see it how, it's part of cURL's pesky built-in globbing. Thanks for educating both myself and the OP. – Celada Apr 17 '16 at 21:25
9

Your problem is that the {...} expression is also valid shell syntax. For example, run:

echo file/{one,two}

And you get:

file/one file/two

So when you run:

curl --user $USER:$PASS https://api.example.com/{foo,bar,baz} -o '#1.json'

The {foo,bar,baz} is getting interpreted by your shell, and curl actually receives the command line:

  curl --user youruser:secret https://api.example.com/foo https://api.example.com/bar https://api.example.com/baz -o '#1.json'

Since curl doesn't see the {...} expression, you don't get the magical handling for #1. The solution is simply to enclose the URL in single quotes:

curl --user $USER:$PASS 'https://api.example.com/{foo,bar,baz}' -o '#1.json'

The single quotes inhibit any shell expansion of the string.

  • Wow, and here I was thinking this question was just based on a misunderstanding of how shell expansion works. Well done. – Wildcard Apr 17 '16 at 21:27
  • Brilliant. Always a small detail that's overlooked causing confusing problems. Thanks. – Chase Apr 17 '16 at 22:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.