Questions tagged [xargs]
xargs is a command that executes another command and generates its arguments from stdin
644
questions
0
votes
3
answers
151
views
Find number of lines between a pattern inside specific filetype
I have a notes.json file inside each directory. The following command counts length of each notes.json file and returns a sorted output by each file's number of lines.
find . -name notes.json | xargs ...
- 7
3
votes
1
answer
27
views
Safely redirect output to a single file with xargs parallel mode
I have a bunch of bzipped JSON file that I read with xargs in parallel, do some light processing with jq and redirect the output to a file as follows:
# Number of workers is one less than the number ...
- 129
0
votes
1
answer
24
views
Dynamically generate arguments for python script input
I call a python script with some command line arguments like:
python3 script.py --run 1 --filepath "this/file/dir"
I now try to parse the arguments from a config file with:
grep -v '^#' ${...
0
votes
2
answers
61
views
xargs running more than one command?
Is it possible to use xargs to run more than one command? The only related information I found was this question, but the answer was not related to xargs.
I'm running a simple
du / -ah | sort -r -n | ...
0
votes
2
answers
33
views
Piping find to xargs works in terminal, but not in shell script [duplicate]
I'm trying to write a Bash shell script that copies all the files in the current directory, minus a few exceptions, into another directory. The script builds a command, stores it in a variable, and ...
- 101
0
votes
1
answer
52
views
Deleting files using xargs prints lots of No such file or directory (but seems to work...)
I have a script that deletes files older than N minutes. The script basically runs:
find /some-folder/* -mmin +59 | xargs rm -rf
But for some reason it prints tons of: No such file or directory but ...
- 871
0
votes
1
answer
22
views
How to print domain name in the curl output
I am using xargs and curl command to get the response from the list of domains in the input file.
xargs -n 1 curl -sS < input.txt 2>&1 | tee curltest
It is hard to find which domain out is ...
- 717
0
votes
0
answers
24
views
pass output as grep pattern [duplicate]
I wish to pass each line of a file as a grep pattern to be searched in data.txt file.
Thus, i need cat allsearchstrings.txt | grep <should be lines of allsearchstrings.txt> data.txt
cat ...
- 297
-1
votes
1
answer
18
views
xargs -I# showing unexpected behavior - why?
I'd like to zip a whole bunch of folders which are in a certain directory, creating one zip file per folder. To do that, I use the following command:
$ find -maxdepth 1 -mindepth 1 -type d | xargs -I@ ...
- 2,191
0
votes
0
answers
23
views
turf-cli + xargs: argument line too long
I'm using turf-cli (it can be installed with sudo npm install -g turf-cli) for creating some geospatial data. turf-cli is a wrapper for the turf.js library just so we can use its functionalities from ...
- 319
0
votes
1
answer
32
views
Why pipe doesn't work with upower -e
I've just found this command:
upower -e
that displays a list of files that can be used with upower -i to display plugged device status.
So my first try was using:
upower -e | xargs upower -i
but it ...
- 8,730
0
votes
1
answer
36
views
Command substitution with xargs inputs [duplicate]
I'm trying to write a simple command that will create symlinks for all files in a specific directory (Yes, exactly like lndir but I cannot use it so I try to mimic it).
I tried the following using ...
- 133
4
votes
0
answers
67
views
execvp: /bin/bash: Argument list too long on Ubuntu 20.04 but OK on Ubuntu 18.04 (all using Windows 10 subsysem)
I bought a new windows 10 PC and installed WSL 2 using (as admin)
wsl --install
This installed WSL 2 with Ubuntu 20.04 LTS. I then copied my files from the old PC which was using same windows 10 pro ...
- 759
0
votes
4
answers
58
views
Returning filename and number of pages for each pdf document in a directory
CONTEXT: Catalina MacOS: A set of .pdf files is in a directory. Some of the filenames have spaces (problematic with xargs.
From the command line: the goal is to return for each .pdf file, the ...
- 1,104
0
votes
2
answers
26
views
Edit a particular string in a file based on another file
Hello I have a file called users. In that file i have a list of users for example
user1
user2
user3
Now i have another file called searches where there is a specific string called owner = user for ...
2
votes
1
answer
30
views
Why do I need to use the -I parameter in `compgen -G ... | xargs basename`?
I ran into a situation where I was piping the output of compgen -G to xargs basename and could not get it to work until I added the xargs -I parameter as seen below. Here is a script demonstrating ...
- 123
0
votes
1
answer
100
views
How to tell `xargs` to ignore empty stdout?
How can this code be changed so as not to issue an error, in other words not execute the code after xargs if stdout is empty.
Terminal:
$ cat << EOF > dummy.sh
#! /usr/bin/env bash
[[ -f &...
- 481
0
votes
1
answer
47
views
Can one pipe compound commands?
My question is: can/how does one pipe compound commands ({ list; })? See Exhibit B. Exhibit A given for comparison's sake.
Exhibit A:
$ cat << EOF | xargs -n 1 echo
foo
bar
qux
EOF
...
- 481
3
votes
3
answers
455
views
Use xargs and kill in script with optional signal
A script for terminating a server on a certain port:
#!/bin/bash
PORT="$1"
SIGNAL="$2"
if [ "${SIGNAL}" != "" ]; then SIGNAL=" -${SIGNAL}"; fi
lsof ...
- 1,413
2
votes
4
answers
134
views
bash Split a list of file and run a command on each group
I am a super beginner with bash and coding in general so bear with me. I have a list of files (>1000) and I need to convert them to another format. However I need to run the command on 40 files a ...
- 21
0
votes
2
answers
61
views
How to pass output of ps x | grep rclone to kill command?
This is the output:
$ ps x | grep rclone
7111 ? Sl 0:00 rclone mount xxx
7112 ? Sl 0:00 rclone mount xxx
7113 ? Sl 10:16 rclone mount xxx
9843 pts/1 S+ ...
- 135
0
votes
0
answers
36
views
when to use -I in xargs
I have difficulties understanding how to use xargs especially using flag -I. I have two html files in my current directory
$ ls
$ hello world.html hi.html junk
I know if I want to copy these ...
- 1
0
votes
1
answer
39
views
Is there a command encapsulating while+shift...?
My question pertains to writing a script that is likely to be used in conjunction with xargs. I find it cumbersome to combine an iteration step and a loop, when the second can be made into a command. ...
- 481
0
votes
1
answer
40
views
How to write a script such that I can pipe it to find?
I have a script that expects files as arguments, and for each executes a set of instructions. How should I write it to be able to pipe it to find? In the example below, I haven't used find as such. I ...
- 481
0
votes
0
answers
35
views
xargs nano leads to Received SIGHUP or SIGTERM while xargs less is OK
Scenario:
$ cat t0.txt
t1.txt
$ cat t1.txt
xxx
$ sed -n 1p t0.txt | xargs less
# OK
$ sed -n 1p t0.txt | xargs nano
Received SIGHUP or SIGTERM
dvl-linux64 $ nano --version
GNU nano, version 2.5.3
...
- 355
0
votes
1
answer
86
views
parse command output for use in next command
I would like to take each line of the output of one gcloud command for use in another gcloud command.
gcloud container clusters list |grep jesse (alias gccl)
output:
jesse-gke1 us-eastx-...
- 3
0
votes
0
answers
43
views
Share variables in xargs
I have a function, say, xfunc, with something like this
xfunc() {
# Infinite loop
while :; do
# Do some actions
echo Hello
# Counter??
done
}
The question is how ...
0
votes
1
answer
104
views
Running script in parallel using xargs and cat
I'm inserting a lot of CSV files into database. I want to do it in parallel, for example run 4 processes. Right now I do it with the script like this:
find . -name "*.csv" | xargs -n 1 -P 4 ....
- 115
-1
votes
3
answers
92
views
How to concatenate several file with * in the path using xargs or parallel
I have a command that fetch a path to a file for each line in a CSV file :
awk -F, 'NR>0 {print "/mnt/datagenetique/ANALYSIS/Infectiologie/COVID-WGS/Analyse/" $2 "_*/dragen-covidseq/&...
- 499
21
votes
2
answers
2k
views
Why does 'top' fail to run via 'xargs'?
I am trying to run top with multiple PIDs using -p option and xargs. However, top fails to run with error top: failed tty get:
$ pgrep gvfs | paste -s -d ',' | xargs -t top -p
top -p 1598,1605,1623,...
- 1,577
1
vote
3
answers
229
views
GNU Parallel vs. xargs vs. Python tools - Sending 10000 reqs/second [closed]
I have 1000000 different requests (different request I mean it has different query param. It is just a GET request, and no payload. Size of request and response are in order of KBs only. no images or ...
- 119
-1
votes
1
answer
209
views
Curl Parallel requests using links source file
I have this script to go through a list of URLs and the check return codes using Curl.
Links file goes like this:
https://link1/...
https://link2/...
https://link200/...
(...)
The script:
INDEX=0
DIR=...
- 81
0
votes
3
answers
156
views
How to get the json data used in each curl statement while using xargs and maps it to its corresponding result?
I have text file that has arguments of curl command. This is how the file looks
'https://example.com/tl/' -X POST -H 'Content-Type: application/json' --data-raw '{"email":"username2&...
- 119
0
votes
2
answers
71
views
How to make "ls -1" to print out in quotation marks in combination with xargs?
I do an ls command like this to list multiple directories. Please be aware of the blank space in the folder X y.
ls -1d 2021*/"X y"/foobar
Output is
'20211121-161518-801/X y/foobar'
'...
- 474
1
vote
2
answers
73
views
How to individually process each path from a list of paths output from ripgrep
I'm on Linux Ubuntu 18.04 and 20.04.
Ripgrep (rg) can output a list of paths to files containing matches like this:
# search only .txt files
rg 'my pattern to match' -g '*.txt' -l
# long form
rg 'my ...
- 1,378
3
votes
3
answers
237
views
How to use find with paths that are listed in a file while ensuring that spaces are taken care of?
I have a list of paths that are in a file called pathlist.txt. It looks like so:
/home/abc/dirA
/home/abc/dirB
/home/abc/dir with space
/home/abc/dirX
I need to find all files in each of those paths. ...
- 8,454
1
vote
1
answer
81
views
reading filenames with newlines
Now we're all familiar with not using:
find . -print | xargs cmd
but using
find . -print0 | xargs -0 cmd
To cope with filenames containing e.g. newline, but what about a line I have in a script:
...
- 53
0
votes
0
answers
12
views
stacking multiple `find -execute` invocations [duplicate]
i love find -execute but sometimes i would like to do more complex operations like running find -execute in all matching folders.
(e.g. run a script on all header files in a subdirectory tree for all ...
- 5,961
0
votes
1
answer
177
views
use two column file as variables inside xargs command
I'm trying to rename some files while learning awk / xargs. I managed a command which gives me a file with two columns, the first one being the name of the file I want to rename, the second one being ...
- 103
2
votes
1
answer
298
views
why one would use xargs without params?
I'm debugging a complex shell script and there is next code:
echo $data | xargs -n1 | sort -u | sed 's/.*/%&%/' | xargs | sed 's/ /,/g;s/%/"/g'`
what use may have xargs without any command ...
- 215
0
votes
2
answers
123
views
find, xarg, Awk Problem
I have the command:
awk 'BEGIN{print "Name, Number"}/value/{print FILENAME, "," $8}' *.txt >> out.csv
Which works perfectly to go through my txt files in the directory, ...
- 1
-1
votes
1
answer
250
views
Script n curl POSTs from n lines of txt file & conditionally save output for HTTP request response "200 OK"
Given this cURL "POST" request:
$ curl -i -s -k -X $'POST' \
-H $'Host: api.host.it' \
-H $'Content-Length: 205' \
-H $'Sec-Ch-Ua: \"Chromium\";v=\"93\", \" Not;A ...
- 5
0
votes
1
answer
195
views
Rename Multiple Files using xargs without using a loop
I have a folder with files
a1.cpp
a2.cpp
a3.cpp
a4.cpp
a5.cpp
a6.cpp
I wish to change extension of files ending in the range 2 to 5 to .java
How do I rename those files without using a loop and with ...
- 1
0
votes
1
answer
150
views
git clone repos with xargs and jq to a subfolder with the name of .owner.login (part of .full_name)
Goal
Clone repos into subdirectories named .owner.login (argument in Github's REST API) with jq, git clone, and xargs.
Preface
I took a code somewhere that allowed me to clone repos with jq, git ...
- 103
1
vote
3
answers
68
views
How to copy the n newest files to a specific directory?
Using
find $dirname -printf "%T@ %Tc %p\n" | sort -n | tail -n 15
I can find the n (in my example n = 15) newest file in a directory structure below $dirname. Now I want to copy (or link) ...
- 16.7k
2
votes
2
answers
290
views
listing directory contents with xargs and grep
I have a directory with lots of json and pdf files that are named in a pattern. I am trying to filter the files on name with the following pattern \d{11}-\d\.(?:json|pdf) in the command. For some ...
- 135
0
votes
2
answers
213
views
One-liner to run commands from a file in parallel using xargs
I have a script like this:
#!/bin/csh
command 1 \
-f \"input1\" \
-l input2 -other_swithes1
command 2 \
-f \"input1\" \
-m input2 \
-l input3 -other_swithes1
...
- 9
0
votes
3
answers
143
views
Why does xargs work with some commands but not others?
I can run this command and it works:
find . -type f -print0 | xargs md5sum
This prints the md5sum for each file.
But for other commands it seems to pass all the files to a single invocation of the ...
- 474
2
votes
2
answers
2k
views
How do I pass the contents of a multiline file as an argument?
I'm trying to run a script that takes a -t argument. This argument stands for text, and the value -- in theory -- is allowed to be multiline. On the command line, I assume a Here Document would work, ...
- 599
0
votes
1
answer
48
views
Oracle Linux 6.7 - Nohup: Argument list too long
There are 1 million directories and some files in directories (.pdf, .csv etc.) under the path I have specified below:
/data/upl/repo/exp/
I want all files and folders older than 1 year to be deleted,...
- 23