3

My script works as below:

./script file1 file2 file3 ... -x -st -i

or

cat filelist.txt | ./script -x -st -i

or

./script <<< "$(cat filelist) -x -st -i

can someone help with common function that sets array from file names in case files are given by positional arguments or piped or redirection

How can i cover all above cases in single shell script?

Tried:

input="$(</dev/stdin)"
Arr_tmp+=("$@")
if ! echo "${Arr_tmp[@]}" | grep 'file_regex'; then  Array+=( $(echo "$input") ); else Array+=("$(echo "$@" | grep 'file_regex')"); fi

or


input="$(</dev/stdin)"
IFS=$'\n' read -ra Array -d '' <<< "$(echo $input)"
if [[ "${#Array[@]}" == 0 ]]; then 
Arr_tmp+=("$@")
fi

But nothing working

Problem with above is when no stdin is piped script struck at read. Is there a way to prevent script from reading and fallback to argumemts if no stdin is piped??

2 Answers 2

7

First, expecting options after non-option arguments is bad practice.

The standard command line interface is to have options before non-option arguments, and allow an optional -- to mark the end of options so you can accept non-option arguments that happen to start with -:

myscript -c -ofile.out -- -file1-.in -file2-.in

Same as:

myscript -co file.out -- -file1-.in -file2-.in

Where, as an example -o is an option that takes an argument, -c one that doesn't, -- marks the end of options, and -file1-.in, -file2-.in are two non-option arguments, taken as such despite starting with - because they are after the -- end-of-option argument.

You can use the standard getopts shell builtin to process options with that standard convention.

On Linux, you can also use the getopt utility from util-linux that handles standard options, but also GNU-style long-options or options that take optional arguments, and allows options after non-option arguments (unless $POSIXLY_CORRECT is in the environment), but still honours -- as the end-of-option marker. getopt will take care of reordering the arguments correctly with options before non-options.

To handle input to be given either via command line or stdin, you'd do:

while getopts...; do
  # process options
  ...
done
# option processing done.
shift "$((OPTIND - 1))"
# now the positional parameters contain non-option arguments

if (( $# )); then
  args=("$@")
else
  # no non-option arguments, read them one per line from stdin
  readarray -t args
fi

The ((...)) is a kshism (also supported by bash), readarray a bashism. In sh, you'd do:

if [ "$#" -eq 0 ]; then
  # no non-option arguments, read them from stdin
  IFS='
' # split on newline
  set -o noglob
  set -- $(cat) # read and split+glob with glob disabled
fi

Where the result is stored in the positional parameters ($1, $2... "$@") instead of the $args array in the bash example above. Beware that in that case, empty lines are discarded.

Also note that file paths can contain any byte value except 0, and that includes the newline character, so you can't pass an arbitrary list of file paths one per line.

Above, we only ready from stdin if no non-option argument is passed on the command line. That's what most text utilities do. For example grep -e regexp -- file1 file2 searches for the regexp in those files, but grep -e regexp searches in stdin.

If you wanted to allow input to be provided both via arguments or stdin, you'd have to always read from stdin:

readarray -t args
args+=("$@")

To combine both into $args.

When the user doesn't want to pass any via stdin, they can always do:

myscript arg1 arg2 < /dev/null

To pass the list interactively when stdin is a terminal, you'd type the file list and end it with Ctrl+d.

You might be tempted to do:

if [ -t 0 ]; then
  args=()
else
  readarray -t args
fi
args+=("$@")

To only read stdin if it's not a tty device, but I'd argue it's a bad idea as then; users would easily lose sight of the fact the script may consume stdin, which would create probem when they use it in a context where stdin is redirected to a file other than /dev/null, such as in things like while IFS= read -r...; do myscript ...; done < some-file.

As suggested by @ilkkachu, you could also add an option to take input from stdin or from an arbitrary file so the user can decide when and when not to read stdin.

Another option is to use xargs which is the tool to convert an input stream into a list of arguments. For instance, with GNU xargs:

xargs -d '\n' -a filelist.txt myscript -x -st -i file1 file2

Would call myscript with file1, file2 and the contents of each line of filelist.txt as arguments. Beware though that if the list is bing, it may be broken down into several invocations of myscript each taking a subset of the lines in filelist.txt, but each of them also getting file1 and file2 each time.

0
0

To avoid waiting for input when there's no pipe, test it before reading:

[ -p /dev/stdin ] && read PIPEIN

Tested on Ubuntu's Bash 5.

5
  • This does not appear to work reliably. For example, it seems to always fail in bash on FreeBSD.
    – Kusalananda
    Apr 28, 2023 at 12:59
  • I know you have, but on FreeBSD, the pipe is implemented using a socket, not a pipe, so the code doesn't work.
    – Kusalananda
    Apr 28, 2023 at 13:42
  • @Kusalananda I don't have FreeBSD, so if it's a socket, I guess, you could just try and replace -p with -S.
    – Multifix
    Apr 28, 2023 at 13:54
  • @Kusalananda, ksh93 may use socketpairs on FreeBSD like it does on Linux, but I doubt bash or any other shell does. It's expect /dev/stdin there to be a character device there though like on most systems except Linux (where it's instead a symlink to /proc/selfd/fd/0 itself a magic symlink to the file open on fd 0) Apr 28, 2023 at 18:22
  • @StéphaneChazelas It is indeed a character special file, but I suspect that's not the complete story, as it's a character special file on macOS too, but the behaviour on macOS is equivalent to that of Linux rather than to that of FreeBSD. There is obviously something here that I don't understand (and don't have the capacity to take in just this moment).
    – Kusalananda
    Apr 28, 2023 at 18:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .