44

Is it possible to use curl and post binary data without passing in a file name? For example, you can post a form using binary via --data-binary:

curl -X POST --data-binary @myfile.bin http://foo.com

However, this requires that a file exists. I was hoping to be able to log HTTP calls (such as to rest services) as the text of the curl command to reproduce the request. (this greatly assists debugging these services, for example)

However, logging curl commands that reference a file would not be useful, so I was hoping I could actually log the raw binary data, presumably base64 encoded, and yet still allow you to copy and paste the logged curl command and execute it.

So, is it possible to use curl and post binary data without referencing a file? If so, how would that work? What would an example look like?

Improve this question
2
  • If you're not logging it to a file then where are you logging it to?
    – slm ♦
    May 30, 2013 at 0:07
  • @slm, I think you misunderstand. I am logging my client-side rest calls to a log file in plain text (as the verbatim equivalent curl command). However, I do not want to have to reference some random file for each binary curl command I log. I want the entire text of the curl command to be fully self-contained so that someone could copy that text into a terminal and run it to reproduce the call.
    – Kirk Woll
    May 30, 2013 at 0:09

3 Answers 3

Reset to default
60

You can pass data into curl via STDIN like so:

echo -e '...data...\n' | curl -X POST --data-binary @- http://foo.com

The @- tells curl to pull in from STDIN.

To pipe binary data to curl (for example):

echo -e '\x03\xF1' | curl -X POST --data-binary @- http://foo.com
Improve this answer
6
  • 1
    But this still does not allow the binary data to be part of the text of the curl command. So is that not possible?
    – Kirk Woll
    May 30, 2013 at 13:50
  • What you're asking for doesn't sound feasible. The data to the curl command can either be piped into curl or pulled into via the @<filename> facility. To my knowledge there isn't any other method.
    – slm ♦
    May 30, 2013 at 14:58
  • 1
    Actually, your answer is perfect -- your example should just use binary instead. :) i.e. echo -e '\x03\xF1' | curl -X POST --data-binary @- http://foo.com Where \x03\xF1 is the raw binary data as hexadecimal, which is pretty much exactly what I want.
    – Kirk Woll
    May 30, 2013 at 16:31
  • 3
    I tried this: cat 1.jpg | curl -X POST --data-binary @- http://foo.com. Worked like a charm. To validate, I also tried: cat 1.jpg | md5sum and md5sum 1.jpg. They both returned the same value.
    – dimitarvp
    May 30, 2014 at 18:53
  • 1
    -X POST is not necessary, since the --data-binary @- part already tells curl to automatically use POST as its method (from man curl: "Sends the specified data in a POST request to the HTTP server"). It can therefore be omitted.
    – Yeti
    Oct 28, 2021 at 11:28
1

Adding to this answer, the echo command appends a newline to its output by default. This adds a \n to the end of your binary data, therefore curl will receive that character too as input.

To avoid that, you can use the printf command or the -n switch like so:

printf '\x03\xF1' | curl -X POST --data-binary @- http://foo.com

or

echo -en '\x03\xF1' | curl -X POST --data-binary @- http://foo.com

This way no newline will be appended to the curl input and the bytes being fed to curl will be exactly those you pass to echo.

Improve this answer
-1

Not sure why, but the exact command line that slm suggested didn't work for me. With a slight modification, the following worked:

echo -e '...data...\n' | curl -s -T - sftp://[email protected]/~/test.txt
Improve this answer
2
  • 2
    The reason why is that SFTP is an entirely different protocol from HTTP with very different capabilities, and both the man page and help message say --data-binary only applies to HTTP (and HTTPS).
    – dave_thompson_085
    Oct 10, 2017 at 2:05
  • That makes sense. I was combining several online sources to get that working so I missed the part of the question which made it HTTP specific. I'll leave my answer here in case someone else needs it.
    – Richard Nienaber
    Oct 10, 2017 at 15:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .