How to parse each line of a text file as an argument to a command?

Question

I'm looking to write a script that takes a .txt filename as an argument, reads the file line by line, and passes each line to a command. For example, it runs command --option "LINE 1", then command --option "LINE 2", etc. The output of the command is written to another file. How do I go about doing that? I don't know where to start.

Answer 1

Another option is xargs.

With GNU xargs:

xargs -a file -I{} -d'\n' command --option {} other args

{} is the place holder for the line of text.

Other xargs generally don't have -a, -d, but some have -0 for NUL-delimited input. With those, you can do:

< file tr '\n' '\0' | xargs -0 -I{} command --option {} other args

On Unix-conformant systems (-I is optional in POSIX and only required for UNIX-conformant systems), you'd need to preprocess the input to quote the lines in the format expected by xargs:

< file sed 's/"/"\\""/g;s/.*/"&"/' |
  xargs -E '' -I{} command --option {} other args

However note that some xargs implementations have a very low limit on the maximum size of the argument (255 on Solaris for instance, the minimum allowed by the Unix specification).

Answer 2

Use while read loop:

: > another_file  ## Truncate file.

while IFS= read -r line; do
    command --option "$line" >> another_file
done < file

Another is to redirect output by block:

while IFS= read -r line; do
    command --option "$line"
done < file > another_file

Last is to open the file:

exec 4> another_file

while IFS= read -r line; do
    command --option "$line" >&4
    echo xyz  ## Another optional command that sends output to stdout.
done < file

If one of the commands reads input, it would be a good idea to use another fd for input so the commands won't eat it (here assuming ksh, zsh or bash for -u 3, use <&3 instead portably):

while IFS= read -ru 3 line; do
    ...
done 3< file

Finally to accept arguments, you can do:

#!/bin/bash

file=$1
another_file=$2

exec 4> "$another_file"

while IFS= read -ru 3 line; do
    command --option "$line" >&4
done 3< "$file"

Which one could run as:

bash script.sh file another_file

Extra idea. With bash, use readarray:

readarray -t lines < "$file"

for line in "${lines[@]}"; do
    ...
done

Note: IFS= can be omitted if you don't mind having line values trimmed of leading and trailing spaces.

Answer 3

Keeping precisely to the question:

#!/bin/bash

# xargs -n param sets how many lines from the input to send to the command

# Call command once per line
[[ -f $1 ]] && cat $1 | xargs -n1 command --option

# Call command with 2 lines as args, such as an openvpn password file
# [[ -f $1 ]] && cat $1 | xargs -n2 command --option

# Call command with all lines as args
# [[ -f $1 ]] && cat $1 | xargs command --option

Answer 4

The best answer I found is:

for i in `cat`; do "$cmd" "$i"; done < $file

EDIT:

... four years later ...

after several down votes and some more experience I'd recommend the following now:

xargs -l COMMAND < file

Answer 5

    sed "s/'/'\\\\''/g;s/.*/\$* '&'/" <<\FILE |\
    sh -s -- command echo --option
all of the{&}se li$n\es 'are safely shell
quoted and handed to command as its last argument
following --option, and, here, before that echo
FILE

OUTPUT

--option all of the{&}se li$n\es 'are safely shell
--option quoted and handed to command as its last argument
--option following --option, and, here, before that echo

Answer 6

ed file.txt
%g/^/s// /
2,$g/^/-,.j
1s/^/command/
wq
chmod 755 file.txt
./file.txt

Take all the lines of a file and pass them as arguments to a single command i.e.,

command line1 line2 line3 ....

If you need the --option flag to precede each line change the second command to:

%g/^/s// --option /