96

I have piped a line in bash script and want to check if the pipe has data, before feeding it to a program.

Searching I found about test -t 0 but it doesn't work here. Always returns false. So how to be sure that the pipe has data?

Example:

echo "string" | [ -t 0 ] && echo "empty" || echo "fill"

Output: fill

echo "string" | tail -n+2 | [ -t 0 ] && echo "empty" || echo "fill"

Output: fill

Unlike Standard/canonical way to test whether foregoing pipeline produced output? the input needs to be preserved to pass it to the program. This generalizes How to pipe output from one process to another but only execute if the first has output? which focuses on sending email.

2

19 Answers 19

69

There's no way to peek at the content of a pipe using commonly available shell utilities, nor is there a portable way to read a character from the pipe then put it back. The only way to know that a pipe has data is to read a byte, and then you have to get that byte to its destination.

So do just that: read one byte; if you detect an end of file, then do what you want to do when the input is empty; if you do read a byte then fork what you want to do when the input is not empty, pipe that byte into it, and pipe the rest of the data.

first_byte=$(dd bs=1 count=1 2>/dev/null | od -t o1 -A n | tr -dc 0-7)
if [ -z "$first_byte" ]; then
  # stuff to do if the input is empty
else
  {
    printf "\\$first_byte"
    cat
  } | {
    # stuff to do if the input is not empty
  }      
fi

The ifne utility from Joey Hess's moreutils runs a command if its input is not empty. It usually isn't installed by default, but it should be available or easy to build on most unix variants. If the input is empty, ifne does nothing and returns the status 0, which cannot be distinguished from the command running successfully. If you want to do something if the input is empty, you need to arrange for the command not to return 0, which can be done by having the success case return a distinguishable error status:

ifne sh -c 'do_stuff_with_input && exit 254'
case $? in
  0) echo empty;;
  254) echo success;;
  *) echo failure;;
esac

test -t 0 has nothing to do with this; it tests whether standard input is a terminal. It doesn't say anything one way or the other as to whether any input is available.

Here with 254 chosen arbitrarily, avoiding 255 as commands doing some exit(-1) (which usually result in $? being 255) are relatively common (-exec(-2) less so).

4
  • On systems with STREAMS based pipes (Solaris HP/UX), I believeve you can use the I_PEEK ioctl to peek at what's on a pipe without consuming it. Mar 29, 2019 at 21:01
  • 1
    @StéphaneChazelas unfortunately there's no way to peek data from a pipe/fifo on *BSD, so no perspective to implement a portable peek utility which could return the actual data from a pipe, not just how much of it there is. (in 4.4 BSD, 386BSD etc pipes were implemented as socket pairs, but that was gutted in later versions of *BSD -- though they kept them bi-directional).
    – user313992
    Mar 29, 2019 at 22:07
  • From my brief testing it looks like this will hang if the pipe is open but there is no data available, as is common when running the script from a terminal. Oct 9, 2023 at 18:31
  • @NathanArthur Yes, it will. That's part of the requirement. If you open a pipe and there's no data on it now, how do you know whether it's because it's an empty pipe or it's because you opened it too early and the other side hasn't started writing yet? The only way to know is to wait until either the writing side writes something, or the writing side closes the pipe. Oct 9, 2023 at 18:50
29

A simple solution is to use ifne command (if input not empty). In some distributions, it is not installed by default. It is a part of the package moreutils in most distros.

ifne runs a given command if and only if the standard input is not empty

Note that if the standard input is not empty, it is passed through ifne to the given command

2
  • 4
    As of 2017, it's not there by default in Mac or Ubuntu. Feb 3, 2017 at 1:32
  • I installed ifne in FreeBSD 11.3 (more specifically, a FreeBSD 11.3 jail in FreeNAS 11.3) using port freshports.org/sysutils/moreutils. I then copied the ifne binary to $HOME/bin in FreeNAS 11.3 so that I could run it from scripts in FreeNAS home directory. Feb 4, 2020 at 8:41
26

check if file descriptor of stdin (0) is open or closed:

[ ! -t 0 ] && echo "stdin has data" || echo "stdin is empty"
4
  • When you pass some data and you want to check if there is some, you pass the FD anyway so this is also not a good test.
    – Jakuje
    Jan 31, 2018 at 15:40
  • 19
    [ -t 0 ] checks if fd 0 is opened to a tty, not whether it is closed or open.
    – user313992
    Feb 1, 2019 at 7:52
  • 1
    @mosvy could you please elaborate on how that would affect using that solution in a script? Are there cases when it doesn't work?
    – JepZ
    Apr 22, 2019 at 20:39
  • 3
    @JepZ huh? ./that_script </dev/null => "stdin has data". Or ./that_script <&- to have the stdin really closed.
    – user313992
    Apr 22, 2019 at 20:59
22

If you like short and cryptic one-liners:

$ echo "string" | grep . && echo "fill" || echo "empty"
string
fill
$ echo "string" | tail -n+2 | grep . && echo "fill" || echo "empty"
empty

I used the examples from the original question. If you don't want the piped data use -q option with grep

Some commenters rightfully noted that this version will ignore new lines. My answer is focused on the original question. From this point of view a newline symbol is an empty string. Using grep ^, as suggested by @EvgEnZh solves this corner case, but I am not sure it is what the requestor wanted. Please, add this variant in comments, it is a useful correction.

6
  • 2
    grep . is exactly what I was looking for, and it's so simple. Thanks!
    – claymation
    May 3, 2020 at 7:29
  • What is the diffrence betwee the two examples? (i.e., the one with tail and the one without).
    – user
    Dec 5, 2020 at 18:49
  • 1
    grep consumes the pipe and blocks when it becomes empty.
    – EvgenKo423
    Feb 10, 2021 at 16:00
  • 1
    good catch, for text with color: echo "" | grep --color=never . || echo "empty"
    – Atika
    Feb 19, 2021 at 16:57
  • @user I used examples from the question. In reality 'echo "string" | tail -n+2' is the same as 'echo ""'
    – yashma
    Feb 23, 2021 at 5:00
16

Old question, but in case someone comes across it as I did: My solution is to read with a timeout.

while read -t 5 line; do
    echo "$line"
done

If stdin is empty, this will return after 5 seconds. Otherwise it will read all the input and you can process it as needed.

In these situations where -t is supported, you can test for input before reading any data with -t0. (also specifying -u0 for STDIN because awslinux needs it, whereas ubuntu assumes it)

if [ $(read -u0 -t0) ]; then ....

And you can start reading as normal afterward

from help read:

If TIMEOUT is 0, read returns immediately, without trying to read any data, returning success only if input is available on the specified file descriptor.

2
11

One easy way to check if there's data available for reading in Unix is with the FIONREAD ioctl.

I cannot think of any standard utility doing just that, so here is a trivial program doing it (better than the ifne from moreutils IMHO ;-)).

fionread [ prog args ... ]

If there's no data available on stdin, it will exit with status 1. If there's data, it will run prog. If no prog is given, it will exit with status 0.

This should work with most kinds of file descriptors, not just pipes.

fionread.c

#include <unistd.h>
#include <poll.h>
#include <sys/ioctl.h>
#ifdef __sun
#include <sys/filio.h>
#endif
#include <err.h>

int main(int ac, char **av){
        int r; struct pollfd pd = { 0, POLLIN };
        if(poll(&pd, 1, -1) < 0) err(1, "poll");
        if(ioctl(0, FIONREAD, &r)) err(1, "ioctl(FIONREAD)");
        if(r < 1) return 1;
        if(++av, --ac < 1) return 0;
        execvp(*av, av);
        err(1, "execvp %s", *av);
}
14
  • Does this program actually work? Don't you need to wait for a POLLHUP event as well to handle the empty case? Does that work if there are multiple file descriptions on the other end of the pipe? Mar 29, 2019 at 19:33
  • Yes it works. POLLHUP is only returned by poll, you should use POLLIN to wait for a POLLHUP. It doesn't matter how many open handles are to any end of the pipe.
    – user313992
    Mar 29, 2019 at 20:15
  • See unix.stackexchange.com/search?q=FIONREAD+user%3A22565 for how to run FIONREAD from perl (more commonly available than compilers) Mar 29, 2019 at 20:45
  • 2
    @DerekMahar Because a) it's much smaller b) much faster c) it's a single process, it execs through to the command directly.
    – user313992
    Feb 4, 2020 at 9:11
  • 2
    @DerekMahar d) it doesn't change the nature of the stdin passed to cmd; eg. if the stdin is a socket, cmd would still be able to call getpeername(2) on it to see who's at the end of the connection. As mentioned by Stéphane Chazelas, you can trivially do this in perl, python, ruby, etc. Not in the shell or with any standard utilities, though.
    – user313992
    Feb 4, 2020 at 9:27
6

In bash:

read -t 0 

Detects if an input has data (without reading anything). Then you can read the input (if the input is available at the time the read is executed):

if     read -t 0
then   read -r input
       echo "got input: $input"
else   echo "No data to read"
fi

Note: Understand that this depends on timing. This detects if input already has data only at the time read -t runs.

For example, with

{ sleep 0.1; echo "abc"; } | read -t 0; echo "$?"

the output is 1 (read failure, i.e.: empty input). The echo writes some data but is not very fast to start and write its first byte, thus, read -t 0 will report that its input is empty, since program has not written anything yet.

2
  • 1
    github.com/bminor/bash/blob/… - here is source of how bash detects that something is in file descriptor. Jun 12, 2019 at 13:27
  • 4
    @PavelPatrin That does not work. As clearly seen from your link, bash will either do a select() or an ioctl(FIONREAD), or neither of them, but not both, as it should for it to work. read -t0 is broken. Do not use it
    – user313992
    Jun 12, 2019 at 18:51
4

You may use test -s /dev/stdin (in an explicit subshell) as well.

# test if a pipe is empty or not
echo "string" | 
    (test -s /dev/stdin && echo 'pipe has data' && cat || echo 'pipe is empty')

echo "string" | tail -n+2 | 
    (test -s /dev/stdin && echo 'pipe has data' && cat || echo 'pipe is empty')

: | (test -s /dev/stdin && echo 'pipe has data' && cat || echo 'pipe is empty')
2
  • 10
    Doesn't work for me. Always says pipe is empty. Dec 15, 2014 at 16:53
  • 4
    Works on my Mac, but not on my Linux box.
    – peak
    Jul 5, 2016 at 17:45
4

If you need this functionality for use in xargs and you have GNU xargs, you can use xargs -r. Ex. curl won't run here:

echo -n '' | xargs -r curl 
1
  • There's a lot of answers here on how to do this on non GNU xargs
    – CervEd
    Sep 20, 2022 at 9:12
2

The following approach worked like a charm for me:

function _read_from_stdin() {
  less <&0 2>/dev/null
}

It reads from STDIN and returns its contents (if any). All you need to do afterwards is to check if contents is empty:

stdin_contents="$(_read_from_stdin)"

if [[ ! "${stdin_contents}" ]] ; then
  echo "STDIN is empty!"
else
  echo "Have something in STDIN: ${stdin_contents}"
fi
2

Full POSIX solution

Compile the following file containing a call to poll() into a binary called has_data:

#include <poll.h>
int main() {
    struct pollfd fds = { 0, POLLIN, 0};
    poll(&fds, 1, -1);
    return fds.revents & POLLIN ? 0 : 1;
}

gcc -o has_data has_data.c

Usage

with a pipeline

cat only runs if has_data succeeds

echo foobar | ( has_data && cat ) | ...

with a file descriptor

N is the number of the file descriptor opened in the current Shell Execution Environment

if has_data <&N; then
    echo "Has data"
fi

with a filename

if has_data <filename; then
    echo "Has data"
fi

There is a problem with this approach when filename is a FIFO, though. According to the documentation of close:

When all file descriptors associated with a pipe or FIFO special file are closed, any data remaining in the pipe or FIFO shall be discarded.

This means that when has_data closes the FIFO file and the file is not opened by another process, the data yet to be read will be lost.

1

Another hack that you can try is using timeout to read from /dev/stdin like this:

timeout 1s cat /dev/stdin > /tmp/input
if [ $? -eq 124 ] && ! [ -s /tmp/input ]
then
  echo "No input provided."
else
  echo "Input provided"
  # Optionally, you can move or read from /tmp/input...whatever your program needs to do.
fi
1

Use grep

chris@SR-ENG-P18 /cygdrive/c/Projects
$ ( ! echo -n "" | grep -q '.' && echo "empty" || echo "fill" )
empty

chris@SR-ENG-P18 /cygdrive/c/Projects
$ ( ! echo -n "string" | grep -q '.' && echo "empty" || echo "fill" )
fill

grep will return 1 (i.e. error) when nothing is matched. You want the opposite thus the not operator !.

Be sure to use echo -n "" in your testing because echo "" will output a newline.

3
  • grep will consume the pipe and block when it becomes empty. Also there is a grep answer already.
    – EvgenKo423
    Feb 10, 2021 at 16:16
  • @EvgenKo423, that answer makes no attempt to explain why this works. 1 for false and 0 for true is a subtlety that needs to be explained.
    – shrewmouse
    Feb 10, 2021 at 16:50
  • Please note that if you have some minor improvements for a question or answer, you should use Edit.
    – EvgenKo423
    Feb 10, 2021 at 17:12
1

I find the following is the most elegant, I use it in scripts:

function isempty {
    ifne false
}

function isnotempty {
    ifne -n false
}

It uses ifne from moreutils, as others already suggested.

0

This seems to be a reasonable ifne implementation in bash if you're ok with reading the whole first line

ifne () {
        read line || return 1
        (echo "$line"; cat) | eval "$@"
}


echo hi | ifne xargs echo hi =
cat /dev/null | ifne xargs echo should not echo
1
  • 6
    read will also return false if the input is non-empty but contains no newline character, read does some processing on its input and may read more than one line unless you call it as IFS= read -r line. echo can't be used for arbitrary data. May 27, 2015 at 8:34
0

This works for me using read -rt 0

example from original question, with no data:

echo "string" | tail -n+2 | if read -rt 0 ; then echo has data ; else echo no data ; fi
2
  • 2
    no, that doesn't work. try with { sleep .1; echo yes; } | { read -rt0 || echo NO; cat; } (false negative) and true | { sleep .1; read -rt0 && echo YES; } (false positive). In fact, bash's read will be fooled even by fds opened in write-only mode: { read -rt0 && echo YES; cat; } 0>/tmp/foo. The only thing it seem to do is a select(2) on that fd.
    – user313992
    Feb 1, 2019 at 8:08
  • 2
    ... and select will return a fd as "ready" if a read(2) on it would not block, no matter if it will return EOF or an error. Conclusion: read -t0 is broken in bash. Don't use it.
    – user313992
    Feb 1, 2019 at 8:38
0

Another approach is reformulating the problem as a pure problem of running a command with the input from a stream and massaging that stream to fit your use case. Ie, you want a command to operate on a stream of data and this data may come from stdin and/or another stream(s).

AND

If its stdin and another stream it should be pretty straightforward using cat

echo bar |
  cat - <(echo biz) |
  cat -A
# output: bar$biz$

This will pipe the contents of stdin and the output of the process substitution echo bar to the stdin of cat -A.

OR

Or on the other hand is a bit trickier. Let's say you want to run a command with input from stdin but fallback to something else if nothing was passed to stdin.

In this case an approach is reformulate the problem as combining streams (including a delimiter sequence say two null-bytes) and sinking the stream IFF stdin is non-empty by using sed and matching the delimiter in the combined stream.

echo bar |
  cat - <(printf '\x0\x0\n') <(echo biz) |
  sed \
    -e '1{/\x0\x0/s@@@;N;s@\n@@}' \
    -e '1!{/\x0\x0/s@@@;q}' | # stdin was non-empty, sink the rest
    cat -A
# output: bar$


< /dev/null |
  cat - <(printf '\x0\x0\n') <(echo biz) |
  sed \
    -e '1{/\x0\x0/s@@@;N;s@\n@@}' \
    -e '1!{/\x0\x0/s@@@;q}' | # stdin was non-empty, sink the rest
    cat -A
# output: biz$

We can combine this approach with xargs -I% for an additional fallback. It uses process substitution but other than that it should be pretty POSIX compatible.

#!/bin/bash

# This script takes a list of paths from stdin
# IFF it's empty the list of files is the existing file 'foo'
# Each file and/or directory in this list is passed to stat
# IFF this list is empty we run stat on the current directory

combined_stream() {
  if [ -t 0 ]; then
    cat <(printf '\x0\x0\n') <(find foo -type f 2> /dev/null)
  else
    cat - <(printf '\x0\x0\n') <(find foo -type f 2> /dev/null)
  fi
}

combined_stream |
  sed \
    -e '1{/\x0\x0/s@@@;N;s@\n@@}' \
    -e '1!{/\x0\x0/s@@@;q}' | # stdin was non-empty, sink the rest
  xargs -I% stat % |
  grep . || stat .
0

Is more difficult to be achieved on C Shell, but I made it.

#!/bin/tcsh -f

while 1
  head -c1 | sed p | \
  ( exit ( ! `head -c1 | wc -c` ) && \
  tail -c1 ) || exit
end

Also achievable on Bourne Shells, though I think may not be the best solution.

#!/bin/sh

while :
do
  head -c1 | sed p | \
  ( ! ( exit `head -c1 | wc -c` ) && \
  tail -c1 ) || exit
done
-1

Simplest ternary

doSomethingAndCheckTruth && echo 'yes' || echo 'no'

The right part (after the &&) will not run unless the left part is complete, so you only check truth upon successfully running something else, saving computation.

Example

using homebrew and grep we check if the bat program is installed

  • If true you will see "yes"
  • If false you will see "no"

I added the -q to suppress the grepped string output here, so you only see "yes" or "no"

brew list | grep -q bat && echo 'yes' || echo 'no'

Tested with bash and zsh

2
  • Multiple answers here already mention grep -q and .. && .. || ... What does this answer add?
    – muru
    Mar 18, 2021 at 1:19
  • The Simplest ternary format I've seen so far Mar 18, 2021 at 3:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .