4

I just saw an example of using echo to put multiple things into the pipeline (for jq).

echo '{"A": {"a": 1}}' '{"A": {"b": 2}}' '{"B": 3}' |\
  jq --slurp 'reduce .[] as $item ({}; . * $item)'

If I understand it right, this seems to put three objects into the pipeline on the one end, and on the other, jq receives them and puts them into an array and reduce it with a merge operator.

This is surprising to me, and I am not sure how it works to have multiple things in the stdin.

More specifically, how does the system know the input is three files and where each file begins/ends?

If I just do the echo part echo '{"A":1}' '{"A": {"b": 2}}' '{"B": 3}', the output does not seem to have any visible delimiter between the three objects:

{"A":1} {"A": {"b": 2}} {"B": 3}

Does the single-quotes have some special effects that tells the next filter (jq) the boundaries? Or how does jq know the boundaries?

(This is with Ubuntu 24.04, jq 1.7.1)

4
  • 2
    Don't get confused: a JSON file is the serialized version of an object, but inherently it's just a string. And as such is processed on the command line. jq deserializes the string and processes the resulting object internally.
    – kos
    Commented Aug 12 at 8:30
  • 1
    @kos, though since the shell only sets up the pipeline, it doesn't need to know anything about what goes in there. It doesn't even see the data. (Though of course the whole subject is a bit hypothetical since there are no objects there.)
    – ilkkachu
    Commented Aug 12 at 9:25
  • @ilkkachu Well I rephrased OP's question according to what (to me) seemed to be their interpretation. They seemed (per the title and per the question) to be under the impression that echo (and by extension the shell) is performing some magic to feed more than one file / multiple JSON objects into a pipeline. Obviously that doesn't represent a correct or fully-encompassing model of how things work, at all, just the model I thought they had in their mind.
    – kos
    Commented Aug 12 at 9:44
  • 1
    jq doesn't see 3 files, just one file with 3 JSON objects
    – slebetman
    Commented Aug 13 at 6:30

1 Answer 1

31

It doesn't do any of that. There are no objects; there's just a "stream" of characters.

echo prints all arguments it gets, separated by spaces. If you do echo 'a' 'b' 'c' then the output will be a b c; a single stream

Similarly your echo statement just produces the output {"A": {"a": 1}} {"A": {"b": 2}} {"B": 3}. Again, just a single stream of characters.

The jq command knows enough about JSON to know those characters mean something (a set of separate JSON objects), but to echo and the pipeline it's just a stream of meaningless characters.

You'd get exactly the same result if you did echo '{"A": {"a": 1}} {"A": {"b": 2}} {"B": 3}'. Or even something like echo '{"A":' '{"a":' "1}}" '{"A": {"b": 2}} {"B": 3}'

Also note that you can produce output from multiple commands to the pipeline, and the result is still just the concatenation of all the bytes involved. E.g. here, foobar gets joined into a single line:

% (printf "foo"; printf "bar\nxxx\n") | head -1
foobar
1
  • 7
    TL;DR: jq divides the stream of bytes into objects because jq recognizes braces ({ and }) as delimiters. Commented Aug 11 at 21:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .