A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Hot answers tagged

144

The difference is in what data the target program is accepting. If you just use a pipe, it receives data on STDIN (the standard input stream) as a raw pile of data that it can sort through one line at a time. However some programs don't accept their commands on standard in, they expect it to be spelled out in the arguments to the command. For example touch ...


89

$ echo 192.168.1. | xargs -I{} grep {} *.txt Example Sample files: $ cat {1..3}.txt 192.168.1 192.168.1 192.168.1 Example run: $ echo 192.168.1. | xargs -I{} grep {} *.txt 1.txt:192.168.1. 2.txt:192.168.1. 3.txt:192.168.1.


73

To expand on the answers already provided, xargs can do one cool thing that is becoming increasingly important in today's multicore and distributed computing landscape: it can parallel process jobs. For example: $ find . -type f -name '*.wav' -print0 |xargs -0 -P 3 -n 1 flac -V8 will encode *.wav => *.flac, using three processes at once (-P 3).


60

Some versions of sort have a -z option, which allows for null-terminated records. find folder1 folder2 -name "*.txt" -print0 | sort -z | xargs -r0 myCommand Additionally, you could also write a high-level script to do it: find folder1 folder2 -name "*.txt" -print0 | python -c 'import sys; sys.stdout.write("\0".join(sorted(sys.stdin.read().split("\0"))))' |...


58

You are almost there. In your last command, you can use -I to do the ls correctly -I replace-str Replace occurrences of replace-str in the initial-arguments with names read from standard input.  Also, unquoted blanks do not terminate input items; instead the separator is the newline character.  Implies -x and -L 1. So, with find . -type ...


57

Use find as usual and delimit your lines with NUL. GNU sort can handle these with the -z switch: find . -print0 | sort -z | xargs -r0 yourcommand


49

Putting multiple jobs in the background is a good way of using the multiple cores of a single machine. parallel however, allows you to spread jobs across multiple servers of your network. From man parallel: GNU parallel is a shell tool for executing jobs in parallel using one or more computers. The typical input is a list of files, a list of ...


43

Look at Stephane's answer for the best method, take a look at my answer for reasons not to use the more obvious solutions (and reasons why they are not the most efficient). You can use the -I option of xargs: find /tmp/ -ctime -1 -name "x*" | xargs -I '{}' mv '{}' ~/play/ Which works in a similar mechanism to find and {}. I would also quote your -name ...


31

You may benefit from the -p or -t flags. xargs -p or xargs --interactive will print out the command to be executed and then prompt for input (y/n) to confirm before executing the command. % cat list one two three % ls list % cat list | xargs -p -I {} touch {} touch one ?...y touch two ?...n touch three ?...y % ls list one three xargs -t or xargs --...


29

You can use standard globbing on the rm command: rm -- *\ * This will delete any file whose name contains a space; the space is escaped so the shell doesn't interpret it as a separator. Adding -- will avoid problems with filenames starting with dashes (they won’t be interpreted as arguments by rm). If you want to confirm each file before it’s deleted, add ...


27

On Linux, the maximum amount of space for command arguments is 1/4th of the amount of available stack space. So, a solution is to increase the amount of space available for the stack. Short version: run something like ulimit -s 65536 Longer version: The default amount of space available for the stack is something like 8192 KB. You can see the amount of ...


27

You can use -I to define a place-holder which will be replaced with each value of the arguments fed to xargs. For example, ls -1 | xargs -I '{}' echo '{}' will call echo once per line from ls's output. You'll often see '{}' used, presumably because it's the same as find's place-holder. In your case, you also need to pre-process file's output to extract ...


27

Just write {} two times in your command. The following would work: $ echo test | xargs -I {} echo {} {} test test Your problem is how the commands are nested. Lets look at this: echo test | xargs -I {} echo {} && echo {} bash will execute echo test | xargs -I {} echo {}. If it runs successfully, echo {} is executed. To change the nesting, you ...


26

Safely piping file names to xargs requires that your find supports the -print0 option and your xargs has the corresponding option to read it (--null or -0). Otherwise, filenames with unprintable characters or backslashes or quotes or whitespace in the name may cause unexpected behavior. On the other hand, find -exec {} + is in the POSIX find spec, so it is ...


26

Yes it can, with the unimaginatively named cat command: $ cat *csv > all.csv cat does what it says on the bottle, it conCATenates its input and prints to standard output. The command above will give an error if a file called all.csv already exists in the target directory: $ cat *csv > all.csv cat: all.csv: input file is output file You can safely ...


26

If you want that last 10 lines: tail myFile.txt | tr '\n' '\0' | xargs -r0i myCmd {} arg1 arg2 But with GNU xargs, you can also set the delimiter to newline with: tail myFile.txt | xargs -ri -d '\n' myCmd {} arg1 arg2 (-0 is short for -d '\0'). Portably, you can also simply escape every character: tail myFile.txt | sed 's/./\\&/g' | xargs -I{} ...


25

No, you can't. From the xargs sources at savannah.gnu.org: if (WEXITSTATUS (status) == CHILD_EXIT_PLEASE_STOP_IMMEDIATELY) error (XARGS_EXIT_CLIENT_EXIT_255, 0, _("%s: exited with status 255; aborting"), bc_state.cmd_argv[0]); if (WIFSTOPPED (status)) error (XARGS_EXIT_CLIENT_FATAL_SIG, 0, _("%s: stopped by signal %d"), bc_state....


25

Try exporting function, then calling it in a subshell: showword() { echo $1 } export -f showword echo This is a sample message | xargs -d' ' -t -n1 -P2 bash -c 'showword "$@"' _


24

xargs is particularly useful when you have a list of filepaths on stdin and want to do something with them. For example: $ git ls-files "*.tex" | xargs -n 1 sed -i "s/color/colour/g" Let's examine this step by step: $ git ls-files "*.tex" tex/ch1/intro.tex tex/ch1/motivation.tex .... In other words, our input is a list of paths that we want to do ...


23

This can be done from find directly using -exec: find . -name "*.xml" -type f -exec xmllint --output '{}' --format '{}' \; What's passed to -exec will be invoked once per file found with the template parameters {} being replaced with the current file name. The \; on the end of the find command just terminates the line. The use of xargs isn't really ...


23

That && is not part of the xargs command, it's a completely separate invocation. I think you'll want to explicitly execute a subshell: echo 95 | xargs -n1 -I_percent -- sh -c '[ _percent -ge 95 ] && echo "No Space on disk _percent% full -- remove old backups please"' Note also I'm using _percent instead of {} to avoid extra quoting ...


23

While you already know how you should solve your current problem, I'll still answer about xargs. xargs puts the string it got in the end of command, while in your case you need that string before the last argument of cp. Use -I option of xargs to construct the command. Like this: ls /source/path/*pattern* | xargs -I{} cp -u {} /destination/path In this ...


23

I believe that you can’t use -I that way.  But you can get the effect / behavior you want by saying: echo {1..8} | xargs -n2 sh -c 'echo "the number $1 comes before $2"' sh This, essentially, creates an ad hoc one-line shell script, which xargs executes via sh -c.  The two values that xargs parses out of the input are passed to this “script”.  The shell ...


22

Because Vim is invoked from inside the pipeline, the stdin is connected to the previous pipeline's output, not the terminal. As an interactive command, Vim needs to receive its input from the terminal. Better avoid the pipe, e.g. via $ vim $(grep -rl test .) or from inside Vim: :args `grep -rl test .`


21

if you want xargs to ignore quotes one of the good soultion can be the use of xargs flag xargs -0 Directly from Man page OPTIONS OPTIONS -0, --null Input items are terminated by a null character instead of by whitespace, and the quotes and backslash are not special (every character is taken literally). Disables the end of ...


21

You might want to chain calls to find (once, when you learned, that it is possible, which might be today). This is, of course, only possible as long as you stay in find. Once you pipe to xargs it's out of scope. Small example, two files a.lst and b.lst: cat a.lst fuddel.sh fiddel.sh cat b.lst fuddel.sh No trick here - simply the fact that both contain "...


21

GNU head and tail since coreutils version 8.25 have a -z option for that. With older versions or for non-GNU systems, you can try and swap \0 and \n: find ... -print0 | tr '\0\n' '\n\0' | head | tr '\0\n' '\n\0' Note that some head implementations can't cope with NUL characters (and they're not required to by POSIX), but where find supports -print0, ...


18

Below are a dozen or so examples of how you can take a file such as this: $ cat k.txt 1 2 3 and convert it to this format: 1,2,3 You can use this command to create the above file if you'd like to play along: $ cat <<EOF > k.txt 1 2 3 EOF The examples below are split into 2 groups. Ones that "work" and ones that "almost" work. I leave these ...


18

"with names read from standard input" means that xargs takes the data coming in on its standard input, splits it up, and uses it to run the command given in its arguments. By default, it splits on blanks or newlines, and runs echo with as many arguments at a time as possible. The -0 option in your example instructs xargs to split its input on null bytes ...


18

If you're using GNU xargs, there's --process-slot-var: --process-slot-var=environment-variable-name Set the environment variable environment-variable-name to a unique value in each running child process. Each value is a decimal integer. Values are reused once child processes exit. This can be used in a rudimentary load distribution scheme, for ...


Only top voted, non community-wiki answers of a minimum length are eligible