168

Is it possible to combine output from these two commands?

node ~/projects/trunk/index.js 
python ~/projects/trunk/run.py run

Neither command exits so I'm not sure how to do this.

3
  • 4
    If the programs don't finish, presumably they write output continuously? What do yo want to do with their output? Interleave lines, ...? Why do you want to do this?
    – vonbrand
    Commented Feb 14, 2013 at 0:37
  • 2
    The node command doesn't output much, but it still needs to run. The python one outputs all requests, I want to capture both and watch them both in the same shell window.
    – chovy
    Commented Feb 14, 2013 at 5:23
  • The output of both of those commands is already being written to the same stream (they each inherit the same file descriptor for stdout). So you don't have to do anything. Commented Nov 20, 2019 at 16:06

9 Answers 9

211

You can combine two commands by grouping it with { } :

{ command1 & command2; }

so far, you can redirect the group to a file (last ; before } is mandatory), and the space between the open and closing bracket too.

{ command1 & command2; } > new_file

if you want to separate STDOUT and STDERRin two files :

{ command1 & command2; } > STDOUT_file 2> STDERR_file

If you don't want to run the first command in the background, use this form :

{ command1; command2; }

or

{ command1 && command2; }

to run the second command only if the first is a success

16
  • 5
    Doesn't matter that they programs don't finish. 'tail -f' doesn't "finish" either, but this still works and combines the outputs of both programs. Works for more than two commands as well. ^c to quit kills only one of the grouped commands. You'll have to kill the other's manually, though.
    – SuperMagic
    Commented Feb 14, 2013 at 4:58
  • 5
    Seems like you lack the last ; before }, it's mandatory ! Commented Dec 2, 2014 at 18:27
  • 6
    Be warned: This doesn't preserve whole lines! You'll get unreliable outputs as lines get split up part way and mixed up among each other. You can try this with { yes {1..20} & yes {1..20}; } | grep -v '^1 2 3' which ideally won't print anything if lines aren't broken.
    – antak
    Commented Jan 17, 2017 at 1:41
  • 32
    I would rather use && instead of &! command1 & command2 - this runs command1 in background and starts command2 immediately, thus running both commands in parallel and messing up the output. command1 && command2 - this runs command1 (in foreground) and then, if command1 succedded, runs command2.
    – DUzun
    Commented Apr 5, 2018 at 11:04
  • 6
    @DUzun OP said neither command exits, so with your solution, the second command will never run
    – Zoey Hewll
    Commented May 31, 2018 at 2:25
105

More generally, it's possible to use either a subshell or command grouping, and redirect the output of the whole group at once.

Code:

( command1 ; command2 ; command3 ) | cat

{ command1 ; command2 ; command3 ; } > outfile.txt

The main difference between the two is that the first one splits of a child process, while the second one operates in the context of the main shell. This can have consequences regarding the setting and use of variables and other environment settings, as well as performance.

Don't forget that the closing bracket in command grouping (and functions) must be separated from the contents by either a semicolon or a newline. This is because "}" is actually a command (keyword) of its own, and must be treated like one.

7
  • 2
    Redirection from ( ) works fine too.
    – muru
    Commented Jan 1, 2015 at 12:49
  • 4
    } isn't a command at all. It's a reserved word. Same goes for {. I usually write such lists like so: { command1;command2;} > outfile.txt. You can add spaces after the semicolons but it's not necessary. The space after { is necessary, though.
    – Wildcard
    Commented May 6, 2016 at 2:45
  • 1
    Be warned: This doesn't preserve whole lines! You'll get unreliable outputs as lines get split up part way and mixed up among each other. You can try this with ( yes {1..20} & yes {1..20}; ) | grep -v '^1 2 3' which ideally won't print anything if lines aren't broken. (H/t to @antak).
    – Ole Tange
    Commented Nov 15, 2017 at 7:36
  • 3
    Sometimes you want to run command2 only if command1 succeded: ( command1 && command2 && command3 ) | cat
    – DUzun
    Commented Apr 5, 2018 at 11:08
  • i prefer the round brackets () as with the curly brackets {} it runs as a background progress and then you have to deal with the output from that. Also pipe to cat ` | cat` is a nicer alternative then ` > /dev/stdout`
    – DarkMukke
    Commented Jun 19, 2018 at 10:36
15

Most of the solutions so far deal badly with the partial line problem. Assume for a second that the programs are:

cmd1() {
    perl -e 'while(1) { print "a"x3000_000,"\n"}'
}
export -f cmd1
cmd2() {
    perl -e 'while(1) { print "b"x3000_000,"\n"}'
}
export -f cmd2

When running those in parallel you want the output to have full lines of as followed by full lines of bs. What you do not want is as and bs mixing on the same line (tr -s ab replaces repeating as with a single a, so it is easier to see what happens):

# This is bad - half lines are mixed
$ (cmd1 & cmd2 ) | tr -s ab
bababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababa
ababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababab

If you instead use GNU Parallel, you get nice clean full lines with either as or bs but never mixed:

$ parallel --line-buffer ::: cmd1 cmd2 | tr -s ab
a
a
b
b
b
b
a

Newer versions of GNU Parallel even avoids filling up your disk: The above can run forever.

3
  • $ parallel --line-buffer ::: ls who | tr -s ab Academic tradition requires you to cite works you base your article on. When using programs that use GNU Parallel to process data for publication please cite: O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login: The USENIX Magazine, February 2011:42-47. This helps funding further development; AND IT WON'T COST YOU A CENT. If you pay 10000 EUR you should feel free to use GNU Parallel without citing. To silence this citation notice: run 'parallel --citation'. Seriously!? Imagine if every utility did this Commented Mar 30, 2022 at 23:29
  • I assume you are no longer a meager student. 10k Euro is steep, but I am happy to send you 100 Euro to remove the text and submit patch upstream. Commented Mar 30, 2022 at 23:31
  • @peterkarasev Read: git.savannah.gnu.org/cgit/parallel.git/tree/doc/…
    – Ole Tange
    Commented Mar 30, 2022 at 23:53
7

For the special case of combining multiple BASH command outputs onto one line, here's a recipe to run each command in turn, removing any newlines between their outputs.

(echo 'ab' && echo 'cd' && echo 'ef') | tr -d '\n'
>>> abcdef

As a real-world example, the code below will embed an ASCII message between two fixed strings of bytes (forming a print command, in this case)

#   hex prefix           encode a message as hex    hex suffix    | strip newline | hex to binary | (then, for example, send the binary over a TCP connection)
(echo '1b40' && echo "Test print #1" | xxd -p && echo '1d564103') | tr -d '\n'    | xxd -r -p     | nc -N 192.168.192.168 9100

(Note: this method only works if the commands exit. For combining stdout from commands that don't exit, see other answers.)

2
  • (1) Please show the (expected) output of your second command. (2) Please show how the OP would use this technique to solve his problem. Commented Jul 11, 2019 at 19:28
  • 1
    1) The output of the second command is non-ascii binary, so it wouldn't be useful to show it. 2) OP likely solved his specific problem between 2013 and now. This question is now effectively a reference on combining the stdout of multiple Bash commands, so I believe a technique for combining them on one line is a useful "recipe" to be mentioned here (since I came here looking for it and couldn't find it).
    – Luke
    Commented Jul 15, 2019 at 5:34
5

The accepted answer’s output might not come in the right order, do not use it if your sources are async (e.g. cat or curl)

Use this instead:

cat <(command1) <(command2) <(command3) > output.txt

Real life example:

cat \
<(curl https://stackoverflow.com/humans.txt) \
<(echo -e "\n\nDownloaded from https://stackoverflow.com") \
> output.md

Produces

There's a bunch of humans behind Stack Overflow and the network of Stack Exchange sites. If you'd like to join us check out https://stackoverflow.com/company/work-here

Downloaded from https://stackoverflow.com
2

I ended up doing this, the other suggestions did not work, as the 2nd command was either killed or never executed.

alias app () {
    nohup python ~/projects/trunk/run.py run 1>/tmp/log 2>&1 &
    echo $! > /tmp/api.pid
    nohup node ~/projects/trunk/index.js 1>/tmp/log 2>&1 &
    echo $! > /tmp/client.pid
    tail -f /tmp/log
}
5
  • 1
    Note: this may cause I/O errors if the two processes try to write to the file "at the same time".
    – Djizeus
    Commented Nov 16, 2016 at 15:06
  • 2
    can specify 2 different log files and do tail -f *.log although I've never seen this as a problem with 2 different processes writing to the same log file.
    – chovy
    Commented Nov 16, 2016 at 20:25
  • @chovy: could u write your issue as question here ... it is useful Commented Nov 22, 2016 at 17:58
  • 1
    Be warned: This doesn't preserve whole lines! You'll get unreliable outputs as lines get split up part way and mixed up among each other. You can try this with command1 = yes {1..20} command2 = yes {1..20} and pipe the combined output through | grep -v '^1 2 3' which ideally won't print anything if lines aren't broken. (H/t to @antak).
    – Ole Tange
    Commented Nov 15, 2017 at 7:38
  • On top of that your disk may run full if the amount of data is big.
    – Ole Tange
    Commented Nov 15, 2017 at 7:39
1

Try this:

paste $(node ~/projects/trunk/index.js) $(python ~/projects/trunk/run.py run) > outputfile
5
  • 1
    what does 'paste' do?
    – chovy
    Commented Feb 14, 2013 at 5:24
  • @chovy, see here: techrepublic.com/article/… Not sure if it'll work in this context though.
    – FixMaker
    Commented Feb 14, 2013 at 10:59
  • 2
    I don't think paste is appropriate here, as it is meant to put columns next to eachother
    – Bernhard
    Commented Feb 15, 2013 at 22:06
  • @Bernhard indeed. But it wasn't specified in the req's Commented Feb 16, 2013 at 3:04
  • @frogstarr78 I think it is highly unlikely that this is what he wants, but you are right, it is not specified.
    – Bernhard
    Commented Feb 16, 2013 at 10:24
1

Since you're already using node, you may want to try concurrently

Run multiple commands concurrently. Like npm run watch-js & npm run watch-less but better.

1

Parallel vs Serial execution: It is important to highlight the two different solutions:

  1. Parallel Execution, use e.g. '&' as in (cmd1)&(cmd2)
  2. Serial Execution, use e.g. '&&' as in (cmd1)&&(cmd2)
  3. Serial Execution, where you want the second task to be executed even if the first one fails, use e.g. ';' as in (cmd1;cmd2)

Proof / example for (1) vs (2):

$ (echo p1;sleep 6;echo p1-end) && (echo p2;sleep 6;echo p2-end)
p1
p1-end
p2
p2-end
$ (echo p1;sleep 6;echo p1-end) & (echo p2;sleep 6;echo p2-end)
[1] 104629
p1
p2
p1-end
p2-end

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .