134

I am using ffmpeg to get the meta info of an audio clip. But I am unable to grep it.

    $ ffmpeg -i 01-Daemon.mp3  |grep -i Duration
    FFmpeg version SVN-r15261, Copyright (c) 2000-2008 Fabrice Bellard, et al.
      configuration: --prefix=/usr --bindir=/usr/bin 
      --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib
      --mandir=/usr/share/man --arch=i386 --extra-cflags=-O2 
      ...

I checked, this ffmpeg output is directed to stderr.

$ ffmpeg -i 01-Daemon.mp3 2> /dev/null

So I think that grep is unable to read error stream to catch matching lines. How can we enable grep to read error stream?

Using nixCraft link, I redirected standard error stream to standard output stream, then grep worked.

$ ffmpeg -i 01-Daemon.mp3 2>&1 | grep -i Duration
  Duration: 01:15:12.33, start: 0.000000, bitrate: 64 kb/s

But what if we do not want to redirect stderr to stdout?

5
  • 2
    I believe that grep can only operate on stdout (Although I can't find the canonical source to back that up), which means that any stream needs to be converted to stdout first. Oct 26, 2010 at 18:20
  • 10
    @Stefan: grep can only operate on stdin. It's the pipe created by the shell that connects grep's stdin to the other command's stdout. And the shell can only connect an stdout to an stdin. Oct 26, 2010 at 19:16
  • Whoops, you're right. I think that's what I really meant to say , I just didn't think it through. Thanks @Giles. Oct 27, 2010 at 17:37
  • Do you want it to still print stdout?
    – Mikel
    Feb 8, 2011 at 2:06
  • Here is the ZSH way: unix.stackexchange.com/questions/265061/…
    – acorello
    Jan 6, 2021 at 15:04

11 Answers 11

121

If you're using bash why not employ anonymous pipes, in essence shorthand for what phunehehe said:

ffmpeg -i 01-Daemon.mp3 2> >(grep -i Duration)

9
  • 3
    +1 Nice! Bash only, but way cleaner than the alternatives.
    – Mikel
    Feb 8, 2011 at 1:43
  • 19
    If you want the filtered redirected output on stderr again, add a >&2, e.g. command 2> >(grep something >&2)
    – tlo
    May 6, 2016 at 9:16
  • 4
    Please explain how this works. 2> redirects stderr to file, I get that. This reads from file >(grep -i Duration). But the file is never stored? What is this technique called so I can read more about it? May 23, 2018 at 10:57
  • 6
    @MarkoAvlijaš Look up process substitution. "Process substitution can also be used to capture output that would normally go to a file, and redirect it to the input of a process."
    – wisbucky
    Jan 4, 2019 at 0:57
  • 4
    Do note that the target command of process substitution runs asynchronously. As a consequence stderr lines that get through the grep filter, may not appear at the place you would expect in the rest of the output, yes even on your next command prompt.
    – db-inf
    Sep 9, 2020 at 11:28
56

None of the usual shells (even zsh) permits pipes other than from stdout to stdin. But all Bourne-style shells support file descriptor reassignment (as in 1>&2). So you can temporarily divert stdout to fd 3 and stderr to stdout, and later put fd 3 back onto stdout. If stuff produces some output on stdout and some output on stderr, and you want to apply filter on the error output leaving the standard output untouched, you can use { stuff 2>&1 1>&3 | filter 1>&2; } 3>&1.

$ stuff () {
  echo standard output
  echo more output
  echo standard error 1>&2
  echo more error 1>&2
}
$ filter () {
  grep a
}
$ { stuff 2>&1 1>&3 | filter 1>&2; } 3>&1
standard output
more output
standard error
11
  • 2
    This approach works. Unfortunately, in my case, if a non-zero return value is returned, it gets lost - the value returned is 0 for me. This may not always happen, but it happens in the case I'm currently looking at. Is there any way to save it? Apr 26, 2016 at 22:08
  • 2
    @FaheemMitha Not sure what you're doing, but maybe pipestatus would help Apr 26, 2016 at 23:15
  • 1
    @FaheemMitha, also set -o pipefail may be helpful here, depending on what you want to do with the error status. (For instance if you have set -e turned on to fail on any errors, you probably want set -o pipefail also.)
    – Wildcard
    Apr 26, 2016 at 23:21
  • @Wildcard Yes, I have set -e turned on to fail on any errors. Apr 26, 2016 at 23:33
  • The rc shell allows piping stderr. See my answer below.
    – Rolf
    Sep 4, 2019 at 18:55
29

This is similar to phunehehe's "temp file trick", but uses a named pipe instead, allowing you to get results slightly closer to when they are output, which can be handy for long-running commands:

$ mkfifo mypipe
$ command 2> mypipe | grep "pattern" mypipe

In this construction, stderr will be directed to the pipe named "mypipe". Since grep has been called with a file argument, it won't look to STDIN for its input. Unfortunately, you will still have to clean up that named pipe once you are done.

If you are using Bash 4, there is a shortcut syntax for command1 2>&1 | command2, which is command1 |& command2. However, I believe that this is purely a syntax shortcut, you are still redirecting STDERR to STDOUT.

2
25

Gilles and Stefan Lasiewski's answers are both good, but this way is simpler:

ffmpeg -i 01-Daemon.mp3 2>&1 >/dev/null | grep "pattern"

I am assuming you don't want ffmpeg's stdout printed.

How it works:

  • pipes first
    • ffmpeg and grep are started, with ffmpeg's stdout going to grep's stdin
  • redirections next, left to right
    • ffmpeg's stderr is set to whatever its stdout is (currently the pipe)
    • ffmpeg's stdout is set to /dev/null
7
  • 1
    This description is confusing to me. The last two bullets make me think "redirect stderr to stdout", then "redirect stdout (with stderr, now) to /dev/null". However, this isn't what this is really doing. Those statements seem to be reversed.
    – Steve
    Sep 24, 2015 at 15:15
  • 2
    @Steve There's no "with stderr" in the second bullet. Have you seen unix.stackexchange.com/questions/37660/order-of-redirections ?
    – Mikel
    Sep 24, 2015 at 15:30
  • No, I mean that's my interpretation of how you described it in English. The link you provided is very useful, though.
    – Steve
    Sep 24, 2015 at 17:50
  • Why was it needed to be redirected to /dev/null ? Sep 30, 2019 at 8:59
  • @KanwaljeetSingh So stdout goes away leaving us with only stderr, otherwise grep will operate on the mixed stream of stdout and stderr produced by ffmpeg
    – aularon
    Nov 21, 2019 at 6:25
10

See below for the script used in these tests.

Grep can only operate on stdin, so therefore you must convert the stderr stream in a form that Grep can parse.

Normally, stdout and stderr are both printed to your screen:

$ ./stdout-stderr.sh
./stdout-stderr.sh: Printing to stdout
./stdout-stderr.sh: Printing to stderr

To hide stdout, but still print stderr do this:

$ ./stdout-stderr.sh >/dev/null
./stdout-stderr.sh: Printing to stderr

But grep won't operate on stderr! You would expect the following command to suppress lines which contain 'err', but it does not.

$ ./stdout-stderr.sh >/dev/null |grep --invert-match err
./stdout-stderr.sh: Printing to stderr

Here's the solution.

The following Bash syntax will hide output to stdout, but will still show stderr. First we pipe stdout to /dev/null, then we convert stderr to stdout, because Unix pipes will only operate on stdout. You can still grep the text.

$ ./stdout-stderr.sh 2>&1 >/dev/null | grep err
./stdout-stderr.sh: Printing to stderr

(Note that the above command is different then ./command >/dev/null 2>&1, which is a very common command).

Here's the script used for testing. This prints one line to stdout and one line to stderr:

#!/bin/sh

# Print a message to stdout
echo "$0: Printing to stdout"
# Print a message to stderr
echo "$0: Printing to stderr" >&2

exit 0
1
  • If you switch the redirections around, you don't need all the braces. Just do ./stdout-stderr.sh 2>&1 >/dev/null | grep err.
    – Mikel
    Feb 8, 2011 at 1:45
5

When you pipe the output of one command to another (using |), you are only redirecting standard output. So that should explain why

ffmpeg -i 01-Daemon.mp3 | grep -i Duration

doesn't output what you wanted (it does work, though).

If you don't want to redirect error output to standard output you can redirect error output to a file, then grep it later

ffmpeg -i 01-Daemon.mp3 2> /tmp/ffmpeg-error
grep -i Duration /tmp/ffmpeg-error
3
  • Thanks. it does work, though, you mean it is working on your machine? Secondly, as you pointed out using pipe we can only redirect stdout. I am interested in some command or bash feature that will let me redirect stderr. (but not the temp file trick) Oct 26, 2010 at 4:07
  • 1
    @Andrew I mean, the command works the way it has been designed to work. It just doesn't work the way you want it to :)
    – phunehehe
    Oct 26, 2010 at 4:25
  • I don't know of any way that can redirect error output of a command to standard input of another. Would be interesting if someone can point that out.
    – phunehehe
    Oct 26, 2010 at 4:31
5

You may swap the streams. This would enable you to grep the original standard error stream while still getting the output that originally went to standard output in the terminal:

somecommand 3>&2 2>&1 1>&3- | grep 'pattern'

This works by first creating a new file descriptor (3) open for output and setting it to the standard error stream (3>&2). Then we redirect standard error to standard output (2>&1). Finally standard output is redirected to the original standard error, and the new file descriptor is closed (1>&3-).

In your case:

ffmpeg -i 01-Daemon.mp3 3>&2 2>&1 1>&3- | grep -i Duration

Testing it:

$ ( echo "error" >&2; echo "output" ) 3>&2 2>&1 1>&3- | grep "error"
output
error

$ ( echo "error" >&2; echo "output" ) 3>&2 2>&1 1>&3- | grep -v "error"
output
6
  • Convoluted as a solution but great for demonstrating how redirection operators work! Also to show how to swap FDs.
    – grin
    May 30, 2021 at 10:20
  • This is not perfect, as all the resulting text is being output to STDOUT.
    – Trolzen
    Oct 20, 2021 at 2:22
  • @Trolzen The question did not specify what to do with the original standard output stream. If you want to discard it, redirect it to /dev/null rather than hooking it up to the standard error stream. My answer is built around the idea that one may still want to see the standard output; hence, I suggest swapping the streams.
    – Kusalananda
    Oct 20, 2021 at 8:14
  • @Kusalananda Yes, you're right, the question didn't specify that. Hence I said "not perfect", but not "wrong". And I'm talking not about discarding the initial output or not, I'm talking about keeping original STDOUT and STDERR separate. And there's a solution for that.
    – Trolzen
    Oct 20, 2021 at 13:48
  • @Trolzen In that case, I'm not entirely sure what it is that you want to say. I mean, it's nice of you to make me aware of that other answer. If that is a better answer for what you are doing, then, by all means, use that answer instead.
    – Kusalananda
    Oct 20, 2021 at 14:09
4

bash can redirect stdout to stdin stream via regular pipe - | It also can redirect both stdout and stderr to stdin by |&

1

I like to use the rc shell in this case.

First install the package (it's less than 1MB).

This is an example of how you would discard stdout and pipe stderr to grep in rc:

find /proc/ >[1] /dev/null |[2] grep task

You can do it without leaving Bash:

rc -c 'find /proc/ >[1] /dev/null |[2] grep task'

As you may have noticed, the syntax is straightforward, which makes this my preferred solution.

You can specify which file descriptor you want piped inside brackets, just after the pipe character.

Standard file descriptors are numbered as such:

  • 0 : Standard input
  • 1 : Standard ouptut
  • 2 : Standard error
0

A variation on the bash subprocess example:

echo two lines to stderr and tee stderr to a file, grep the tee and pipe back out to stdout

(>&2 echo -e 'asdf\nfff\n') 2> >(tee some.load.errors | grep 'fff' >&1)

stdout:

fff

some.load.errors (eg stderr):

asdf
fff
-1

try this command

  • create file with random name
  • send output to the file
  • send content of the file to pipe
  • grep

ceva --> just a variable name (english = something)

ceva=$RANDOM$RANDOM$RANDOM; ffmpeg -i qwerty_112_0_0_record.flv 2>$ceva; cat $ceva | grep Duration; rm $ceva;
2
  • 1
    I would use /tmp/$ceva, better than littering the current directory with temporary files - and you are assuming that the current directory is writable.
    – Rolf
    Sep 4, 2019 at 18:48
  • If you are going to create a temp file use mktemp.
    – Adam Shand
    Sep 1, 2021 at 0:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.