Questions tagged [xargs]
xargs is a command that executes another command and generates its arguments from stdin
661
questions
0
votes
1
answer
28
views
How to create multiple directories and populate them with files
I want to make some directory trees and populate each end of the tree with a text file with random content.
Here is what I tried so far:
#!/bin/bash
set -e
echo "Generating test folders"
...
- 673
0
votes
1
answer
12
views
xargs combination of variable and piping
Hello I am building a bash script, I would like to use xargs to do a call of my_command in parallel, issue is that with the command below, the variable $input_file is not available.
command is looking ...
- 101
1
vote
1
answer
38
views
Pass multiple rows from awk to xargs
I want an easy way to monitor the resource usage of processes matching certain keywords.
So I do something like:
ps -aux | grep XX | awk 'FNR == 2 {print $2}' | xargs -o -I {} top -p {}
(pgrep won't ...
- 384
2
votes
3
answers
317
views
xargs ignore blank line when using -d'\n'
I have a file input that may have blank lines, I want any empty lines skipped.
I am trying to find a solution that does not require using grep / tr and piping the output to xargs
I use the following ...
- 41
4
votes
2
answers
258
views
The command `grep -Ilri foo . | xargs grep -i bar` works only if the resulting path has no space
To find all the files that have the word foo in it, and also the word bar in it, we can use
grep -Ilri foo . | xargs grep -i bar
(which is case insensitive and exclude binary files)... however, if ...
- 2,781
0
votes
1
answer
169
views
Need to capture exit code of pylint command using find in Github actions
I am trying to implement a python linter using pylint. But I am getting the score of each python file and also displaying the suggestion to improve the score but I am also looking to terminate the ...
0
votes
0
answers
47
views
Difference between xargs and GNU parallel
I need to run an executable a large number of times, each time with two command line arguments. I've used to use xargs for this purpose, but lately I've been made aware of the existence of GNU ...
- 123
0
votes
1
answer
64
views
Why basename don't work here [duplicate]
Why when I do this (I know, it's stupid) it returns the full path, not only the filename as expected?
ls -l | awk '{print $9}' | xargs -I% find ./my_dir -type f -name "%" -exec echo $(...
- 101
2
votes
4
answers
273
views
How to distribute files per directories based on names of these files?
I have camera pictures stored in files with names
192.168.33.16_01_20220723150356205_MOTION_DETECTION.jpg
I want to move each file into separate directory per date
I found that command
find | awk '{...
- 2,719
1
vote
0
answers
87
views
How to circumvent the "xargs: argument line too long" problem if one wants to send very long arguments to a command?
Say I have an executable script process_image that performs actions on a base 64 encoded image. I am storing every image in a file images_file line by line. Every line of images_file is a base 64 ...
- 51
1
vote
1
answer
88
views
Does xargs -P run background processes? How is it different from using an &?
I have a file processes.txt as follows:
process_1.sh
process_2.sh
process_3.sh
I will run the following on the terminal:
cat processes.txt | xargs -L1 -P3 sh
How would this be different if I had a ...
- 51
0
votes
2
answers
36
views
tar processing files multiple times with find -newer
I'm trying to use tar(1) to create an archive of files newer than a specific file (fileA). However, when I use find(1) to obtain the list of files to pass to tar, some files are listed multiple times:
...
- 609
0
votes
1
answer
62
views
"xargs" not sending argument to "cd" via stdio
I am seeing "xargs" behaviour that seems to be quite unexpected, in my experience. I use "ls" to find a matching folder name and sending it to "cd" using "xargs&...
- 225
0
votes
1
answer
33
views
Define rsync source files as last arguments
Say I am using find to define source files for an rsync-operation, making use of special capabilities I only know from these two commands:
find /path/to/data -mtime -7 -mtime +5 -exec rsync -aAXu {} /...
- 11.3k
0
votes
1
answer
97
views
Cannot source file using find, fzf, and xargs
I am trying to use fzf to select and activate different virtual environments. However, I am receiving the follow error:
xargs: source: No such file or directory
My command comprises three parts: find |...
- 167
1
vote
3
answers
109
views
What is the difference in how linux implements commands which accept input via pipe vs. command arguments?
Although there are many similar questions, I cannot find one that directly answers my question. I understand the basics of OS memory management, processes, communication between processes, etc. ...
- 167
0
votes
1
answer
120
views
xargs sed cmd error xargs: gsed: No such file or directory
I am trying to replace the below line in myfiles.hcl file
"-var-file=${get_parent_terragrunt_dir()}/variables/eu-west-2.tfvars",
with
"-var-file=${get_parent_terragrunt_dir()}/...
- 101
3
votes
3
answers
666
views
Remove all dnf modules currently installed
I have a CentOS 8.0 system with quite a few modules installed I don't want. I want to get rid of them to force an upgrade to CentOS 9.0, which I know is not officially supported but it is not actually ...
- 1,820
0
votes
3
answers
200
views
Find number of lines between a pattern inside specific filetype
I have a notes.json file inside each directory. The following command counts length of each notes.json file and returns a sorted output by each file's number of lines.
find . -name notes.json | xargs ...
- 7
3
votes
1
answer
81
views
Safely redirect output to a single file with xargs parallel mode
I have a bunch of bzipped JSON file that I read with xargs in parallel, do some light processing with jq and redirect the output to a file as follows:
# Number of workers is one less than the number ...
- 129
0
votes
1
answer
187
views
Dynamically generate arguments for python script input
I call a python script with some command line arguments like:
python3 script.py --run 1 --filepath "this/file/dir"
I now try to parse the arguments from a config file with:
grep -v '^#' ${...
0
votes
2
answers
83
views
xargs running more than one command?
Is it possible to use xargs to run more than one command? The only related information I found was this question, but the answer was not related to xargs.
I'm running a simple
du / -ah | sort -r -n | ...
- 132
0
votes
2
answers
192
views
Piping find to xargs works in terminal, but not in shell script [duplicate]
I'm trying to write a Bash shell script that copies all the files in the current directory, minus a few exceptions, into another directory. The script builds a command, stores it in a variable, and ...
- 101
0
votes
1
answer
109
views
Deleting files using xargs prints lots of No such file or directory (but seems to work...)
I have a script that deletes files older than N minutes. The script basically runs:
find /some-folder/* -mmin +59 | xargs rm -rf
But for some reason it prints tons of: No such file or directory but ...
- 891
0
votes
1
answer
132
views
How to print domain name in the curl output
I am using xargs and curl command to get the response from the list of domains in the input file.
xargs -n 1 curl -sS < input.txt 2>&1 | tee curltest
It is hard to find which domain out is ...
- 717
0
votes
0
answers
24
views
pass output as grep pattern [duplicate]
I wish to pass each line of a file as a grep pattern to be searched in data.txt file.
Thus, i need cat allsearchstrings.txt | grep <should be lines of allsearchstrings.txt> data.txt
cat ...
- 429
-1
votes
1
answer
27
views
xargs -I# showing unexpected behavior - why?
I'd like to zip a whole bunch of folders which are in a certain directory, creating one zip file per folder. To do that, I use the following command:
$ find -maxdepth 1 -mindepth 1 -type d | xargs -I@ ...
- 2,287
0
votes
0
answers
38
views
turf-cli + xargs: argument line too long
I'm using turf-cli (it can be installed with sudo npm install -g turf-cli) for creating some geospatial data. turf-cli is a wrapper for the turf.js library just so we can use its functionalities from ...
- 319
0
votes
1
answer
44
views
Why pipe doesn't work with upower -e
I've just found this command:
upower -e
that displays a list of files that can be used with upower -i to display plugged device status.
So my first try was using:
upower -e | xargs upower -i
but it ...
- 9,030
0
votes
1
answer
126
views
Command substitution with xargs inputs [duplicate]
I'm trying to write a simple command that will create symlinks for all files in a specific directory (Yes, exactly like lndir but I cannot use it so I try to mimic it).
I tried the following using ...
- 133
4
votes
1
answer
207
views
execvp: /bin/bash: Argument list too long on Ubuntu 20.04 but OK on Ubuntu 18.04 (all using Windows 10 subsysem)
I bought a new windows 10 PC and installed WSL 2 using (as admin)
wsl --install
This installed WSL 2 with Ubuntu 20.04 LTS. I then copied my files from the old PC which was using same windows 10 pro ...
- 789
0
votes
4
answers
114
views
Returning filename and number of pages for each pdf document in a directory
CONTEXT: Catalina MacOS: A set of .pdf files is in a directory. Some of the filenames have spaces (problematic with xargs.
From the command line: the goal is to return for each .pdf file, the ...
- 1,134
0
votes
2
answers
28
views
Edit a particular string in a file based on another file
Hello I have a file called users. In that file i have a list of users for example
user1
user2
user3
Now i have another file called searches where there is a specific string called owner = user for ...
2
votes
1
answer
72
views
Why do I need to use the -I parameter in `compgen -G ... | xargs basename`?
I ran into a situation where I was piping the output of compgen -G to xargs basename and could not get it to work until I added the xargs -I parameter as seen below. Here is a script demonstrating ...
- 123
0
votes
1
answer
458
views
How to tell `xargs` to ignore empty stdout?
How can this code be changed so as not to issue an error, in other words not execute the code after xargs if stdout is empty.
Terminal:
$ cat << EOF > dummy.sh
#! /usr/bin/env bash
[[ -f &...
- 501
0
votes
1
answer
51
views
Can one pipe compound commands?
My question is: can/how does one pipe compound commands ({ list; })? See Exhibit B. Exhibit A given for comparison's sake.
Exhibit A:
$ cat << EOF | xargs -n 1 echo
foo
bar
qux
EOF
...
- 501
3
votes
3
answers
523
views
Use xargs and kill in script with optional signal
A script for terminating a server on a certain port:
#!/bin/bash
PORT="$1"
SIGNAL="$2"
if [ "${SIGNAL}" != "" ]; then SIGNAL=" -${SIGNAL}"; fi
lsof ...
- 1,511
2
votes
4
answers
271
views
bash Split a list of file and run a command on each group
I am a super beginner with bash and coding in general so bear with me. I have a list of files (>1000) and I need to convert them to another format. However I need to run the command on 40 files a ...
- 21
0
votes
2
answers
281
views
How to pass output of ps x | grep rclone to kill command?
This is the output:
$ ps x | grep rclone
7111 ? Sl 0:00 rclone mount xxx
7112 ? Sl 0:00 rclone mount xxx
7113 ? Sl 10:16 rclone mount xxx
9843 pts/1 S+ ...
- 135
0
votes
0
answers
46
views
when to use -I in xargs
I have difficulties understanding how to use xargs especially using flag -I. I have two html files in my current directory
$ ls
$ hello world.html hi.html junk
I know if I want to copy these ...
- 1
0
votes
1
answer
78
views
Is there a command encapsulating while+shift...?
My question pertains to writing a script that is likely to be used in conjunction with xargs. I find it cumbersome to combine an iteration step and a loop, when the second can be made into a command. ...
- 501
0
votes
1
answer
61
views
How to write a script such that I can pipe it to find?
I have a script that expects files as arguments, and for each executes a set of instructions. How should I write it to be able to pipe it to find? In the example below, I haven't used find as such. I ...
- 501
1
vote
0
answers
75
views
xargs nano leads to Received SIGHUP or SIGTERM while xargs less is OK
Scenario:
$ cat t0.txt
t1.txt
$ cat t1.txt
xxx
$ sed -n 1p t0.txt | xargs less
# OK
$ sed -n 1p t0.txt | xargs nano
Received SIGHUP or SIGTERM
dvl-linux64 $ nano --version
GNU nano, version 2.5.3
...
- 407
0
votes
1
answer
152
views
parse command output for use in next command
I would like to take each line of the output of one gcloud command for use in another gcloud command.
gcloud container clusters list |grep jesse (alias gccl)
output:
jesse-gke1 us-eastx-...
- 3
0
votes
0
answers
84
views
Share variables in xargs
I have a function, say, xfunc, with something like this
xfunc() {
# Infinite loop
while :; do
# Do some actions
echo Hello
# Counter??
done
}
The question is how ...
- 17
0
votes
1
answer
247
views
Running script in parallel using xargs and cat
I'm inserting a lot of CSV files into database. I want to do it in parallel, for example run 4 processes. Right now I do it with the script like this:
find . -name "*.csv" | xargs -n 1 -P 4 ....
- 115
-1
votes
3
answers
187
views
How to concatenate several file with * in the path using xargs or parallel
I have a command that fetch a path to a file for each line in a CSV file :
awk -F, 'NR>0 {print "/mnt/datagenetique/ANALYSIS/Infectiologie/COVID-WGS/Analyse/" $2 "_*/dragen-covidseq/&...
- 509
21
votes
2
answers
2k
views
Why does 'top' fail to run via 'xargs'?
I am trying to run top with multiple PIDs using -p option and xargs. However, top fails to run with error top: failed tty get:
$ pgrep gvfs | paste -s -d ',' | xargs -t top -p
top -p 1598,1605,1623,...
- 1,596
1
vote
3
answers
458
views
GNU Parallel vs. xargs vs. Python tools - Sending 10000 reqs/second [closed]
I have 1000000 different requests (different request I mean it has different query param. It is just a GET request, and no payload. Size of request and response are in order of KBs only. no images or ...
- 119
-1
votes
1
answer
897
views
Curl Parallel requests using links source file
I have this script to go through a list of URLs and the check return codes using Curl.
Links file goes like this:
https://link1/...
https://link2/...
https://link200/...
(...)
The script:
INDEX=0
DIR=...
- 101