89

Is it possible to combine output from these two commands?

node ~/projects/trunk/index.js 
python ~/projects/trunk/run.py run

Neither command exits so I'm not sure how to do this.

  • 3
    If the programs don't finish, presumably they write output continuously? What do yo want to do with their output? Interleave lines, ...? Why do you want to do this? – vonbrand Feb 14 '13 at 0:37
  • 2
    The node command doesn't output much, but it still needs to run. The python one outputs all requests, I want to capture both and watch them both in the same shell window. – chovy Feb 14 '13 at 5:23
  • The output of both of those commands is already being written to the same stream (they each inherit the same file descriptor for stdout). So you don't have to do anything. – William Pursell Nov 20 at 16:06
115

You can combine two commands by grouping it with { } :

{ command1 & command2; }

so far, you can redirect the group to a file (last ; before } is mandatory):

{ command1 & command2; } > new_file

if you want to separate STDOUT and STDERRin two files :

{ command1 & command2; } > STDOUT_file 2> STDERR_file
  • 3
    Doesn't matter that they programs don't finish. 'tail -f' doesn't "finish" either, but this still works and combines the outputs of both programs. Works for more than two commands as well. ^c to quit kills only one of the grouped commands. You'll have to kill the other's manually, though. – SuperMagic Feb 14 '13 at 4:58
  • 5
    Seems like you lack the last ; before }, it's mandatory ! – Gilles Quenot Dec 2 '14 at 18:27
  • 2
    Be warned: This doesn't preserve whole lines! You'll get unreliable outputs as lines get split up part way and mixed up among each other. You can try this with { yes {1..20} & yes {1..20}; } | grep -v '^1 2 3' which ideally won't print anything if lines aren't broken. – antak Jan 17 '17 at 1:41
  • 8
    I would rather use && instead of &! command1 & command2 - this runs command1 in background and starts command2 immediately, thus running both commands in parallel and messing up the output. command1 && command2 - this runs command1 (in foreground) and then, if command1 succedded, runs command2. – DUzun Apr 5 '18 at 11:04
  • 2
    @DUzun OP said neither command exits, so with your solution, the second command will never run – Zoey Hewll May 31 '18 at 2:25
53

More generally, it's possible to use either a subshell or command grouping, and redirect the output of the whole group at once.

Code:

( command1 ; command2 ; command3 ) | cat

{ command1 ; command2 ; command3 ; } > outfile.txt

The main difference between the two is that the first one splits of a child process, while the second one operates in the context of the main shell. This can have consequences regarding the setting and use of variables and other environment settings, as well as performance.

Don't forget that the closing bracket in command grouping (and functions) must be separated from the contents by either a semicolon or a newline. This is because "}" is actually a command (keyword) of its own, and must be treated like one.

  • 2
    Redirection from ( ) works fine too. – muru Jan 1 '15 at 12:49
  • 3
    } isn't a command at all. It's a reserved word. Same goes for {. I usually write such lists like so: { command1;command2;} > outfile.txt. You can add spaces after the semicolons but it's not necessary. The space after { is necessary, though. – Wildcard May 6 '16 at 2:45
  • 1
    Be warned: This doesn't preserve whole lines! You'll get unreliable outputs as lines get split up part way and mixed up among each other. You can try this with ( yes {1..20} & yes {1..20}; ) | grep -v '^1 2 3' which ideally won't print anything if lines aren't broken. (H/t to @antak). – Ole Tange Nov 15 '17 at 7:36
  • 3
    Sometimes you want to run command2 only if command1 succeded: ( command1 && command2 && command3 ) | cat – DUzun Apr 5 '18 at 11:08
  • i prefer the round brackets () as with the curly brackets {} it runs as a background progress and then you have to deal with the output from that. Also pipe to cat ` | cat` is a nicer alternative then ` > /dev/stdout` – DarkMukke Jun 19 '18 at 10:36
2

I ended up doing this, the other suggestions did not work, as the 2nd command was either killed or never executed.

alias app () {
    nohup python ~/projects/trunk/run.py run 1>/tmp/log 2>&1 &
    echo $! > /tmp/api.pid
    nohup node ~/projects/trunk/index.js 1>/tmp/log 2>&1 &
    echo $! > /tmp/client.pid
    tail -f /tmp/log
}
  • 1
    Note: this may cause I/O errors if the two processes try to write to the file "at the same time". – Djizeus Nov 16 '16 at 15:06
  • 2
    can specify 2 different log files and do tail -f *.log although I've never seen this as a problem with 2 different processes writing to the same log file. – chovy Nov 16 '16 at 20:25
  • @chovy: could u write your issue as question here ... it is useful – Abdennour TOUMI Nov 22 '16 at 17:58
  • 1
    Be warned: This doesn't preserve whole lines! You'll get unreliable outputs as lines get split up part way and mixed up among each other. You can try this with command1 = yes {1..20} command2 = yes {1..20} and pipe the combined output through | grep -v '^1 2 3' which ideally won't print anything if lines aren't broken. (H/t to @antak). – Ole Tange Nov 15 '17 at 7:38
  • On top of that your disk may run full if the amount of data is big. – Ole Tange Nov 15 '17 at 7:39
2

Try this:

paste $(node ~/projects/trunk/index.js) $(python ~/projects/trunk/run.py run) > outputfile
  • 1
    what does 'paste' do? – chovy Feb 14 '13 at 5:24
  • @chovy, see here: techrepublic.com/article/… Not sure if it'll work in this context though. – FixMaker Feb 14 '13 at 10:59
  • I don't think paste is appropriate here, as it is meant to put columns next to eachother – Bernhard Feb 15 '13 at 22:06
  • @Bernhard indeed. But it wasn't specified in the req's – frogstarr78 Feb 16 '13 at 3:04
  • @frogstarr78 I think it is highly unlikely that this is what he wants, but you are right, it is not specified. – Bernhard Feb 16 '13 at 10:24
2

Most of the solutions so far deal badly with the partial line problem. Assume for a second that the programs are:

cmd1() {
    perl -e 'while(1) { print "a"x3000_000,"\n"}'
}
export -f cmd1
cmd2() {
    perl -e 'while(1) { print "b"x3000_000,"\n"}'
}
export -f cmd2

When running those in parallel you want the output to have full lines of as followed by full lines of bs. What you do not want is as and bs mixing on the same line (tr -s ab replaces repeating as with a single a, so it is easier to see what happens):

# This is bad - half lines are mixed
$ (cmd1 & cmd2 ) | tr -s ab
bababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababa
ababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababababab

If you instead use GNU Parallel, you get nice clean full lines with either as or bs but never mixed:

$ parallel --line-buffer ::: cmd1 cmd2 | tr -s ab
a
a
b
b
b
b
a

Newer versions of GNU Parallel even avoids filling up your disk: The above can run forever.

1

For the special case of combining multiple BASH command outputs onto one line, here's a recipe to run each command in turn, removing any newlines between their outputs.

(echo 'ab' && echo 'cd' && echo 'ef') | tr -d '\n'
>>> abcdef

As a real-world example, the code below will embed an ASCII message between two fixed strings of bytes (forming a print command, in this case)

#   hex prefix           encode a message as hex    hex suffix    | strip newline | hex to binary | (then, for example, send the binary over a TCP connection)
(echo '1b40' && echo "Test print #1" | xxd -p && echo '1d564103') | tr -d '\n'    | xxd -r -p     | nc -N 192.168.192.168 9100

(Note: this method only works if the commands exit. For combining stdout from commands that don't exit, see other answers.)

  • (1) Please show the (expected) output of your second command. (2) Please show how the OP would use this technique to solve his problem. – Scott Jul 11 at 19:28
  • 1) The output of the second command is non-ascii binary, so it wouldn't be useful to show it. 2) OP likely solved his specific problem between 2013 and now. This question is now effectively a reference on combining the stdout of multiple Bash commands, so I believe a technique for combining them on one line is a useful "recipe" to be mentioned here (since I came here looking for it and couldn't find it). – Luke Jul 15 at 5:34
0

Since you're already using node, you may want to try concurrently

Run multiple commands concurrently. Like npm run watch-js & npm run watch-less but better.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.