32
votes
Accepted
How to live update `wc -l`?
You can’t use wc -l for this, but you can produce a running count of lines seen using other tools, for example AWK:
<streaming ls> | awk '{ printf "%d\r", NR } END { print NR }'
This ...
- 402k
27
votes
Accepted
Significance of `[...] & wait $!`
There are two side effects of running a command asynchronously in a non-interactive shell that make A and A & wait "$!" different:
SIGINT and SIGQUIT are ignored for A. If you press ...
- 512k
25
votes
Accepted
What is the difference between `cat EOF` and `cat EOT` and when should I use it?
There is no difference, and no particular meaning to those two strings, or any others. It's just an arbitrary terminator and you can use almost any string you like.
Of course, the data itself can't ...
- 130k
25
votes
How to live update `wc -l`?
You could use pv to gives you some progress report:
cmd | pv -lbtr | wc -l
-l for line-based (reports the number of lines instead of bytes).
-b to report the number bytes (well lines here because of -...
- 512k
22
votes
Accepted
What are curly quotes and can I use them in my code?
Curly quotes, or “smart quotes” are used in typography and "straight quotes" will often be automatically changed to curly quotes by various word processing programs. They are sometimes ...
- 35.4k
22
votes
Accepted
What does a hyphen do next to the argument position in bash shell script? Like ${1-}
A hyphen in a parameter expansion allows a default value to be specified. So ${1-} means “the value of the first parameter if it is set, and the empty string otherwise”.
That doesn’t seem particularly ...
- 402k
21
votes
Accepted
Space not taken as an argument separator by shell script (could someone please explain that small file difference ?)
c2 a0 is the UTF-8 encoding of the non-breaking space character. It usually looks like a regular space, but isn't recognized as whitespace by the shell.
In a few keymaps, something like AltGr+Space, ...
- 130k
19
votes
Accepted
How do I perform rm -v !(*.yaml) in a bash script?
!(x) is a Korn shell glob operator. bash supports a subset of ksh's globs operators including that one but only after shopt -s extglob¹
So:
shopt -s extglob
rm -f -- !(*.yaml)
Will remove all the non-...
- 512k
19
votes
Accepted
Use the 'cp' command in a Bash script and exclude a specific directory
You should enable the extglob option, it isn't enabled by default in scripts:
#!/bin/bash
shopt -s extglob
cp -var test/!(test2) testbkp
Also verify that the script is indeed running under bash.
- 3,507
17
votes
Accepted
Can a script be run as a subshell?
(. ./myscript.sh)
is the right way (the standard form is ., not source, and . with no directory specified searches on the PATH).
Doing this using a shebang would require having a reliable way of ...
- 402k
16
votes
Accepted
How can I loop this command in a bash script until the repsonse contains the string "connection successful"
It is rarely necessary to store the output from grep when you just want to test whether some text matches a pattern. Instead, just invoke grep with its -q option and act on its exit status:
#!/bin/sh
...
- 316k
15
votes
Accepted
Executable vs Library in Bash Script
At least when it comes to bash scripts, what exactly is the difference between an executable and a library of bash scripts?
I haven't heard the phrase "library" used much for shell scripts, ...
- 130k
15
votes
Start 100 process at a time in bash script
With zsh instead of bash:
autoload -Uz zargs
zargs -P100 -I{} -- {1..1000} -- foo {}
But if you have GNU xargs, you can also do (in zsh, ksh93 or bash):
xargs -I{} -P100 -a <(echo {1..1000}) foo {}...
- 512k
14
votes
Accepted
bash script produces different output than when commands run in bash prompt
Extended globs like +(...) aren't enabled in Bash by default, you'll need to explicitly use shopt -s extglob in the script to enable them.
Your interactive shell probably has extglob enabled (in some ...
- 130k
14
votes
Understanding "side effect", or multiple commands within one command?
The : is called the null command. You can find its documentation in man bash:
: [arguments]
No effect; the command does nothing beyond expanding arguments and performing any specified redirections....
- 232k
13
votes
Accepted
Should I use pwd or tilde plus (~+)?
~+ doesn't appear to be standard, only ~username and the tilde alone are defined. And e.g. Dash and Busybox don't support ~+. The command pwd, as well as the shell variable $PWD are standard, though, ...
- 130k
13
votes
Is it OK to check indirectly in a Bash if statement for exit codes if they are multiple?
$? expands to the exit status of the last command, so in:
if [ $? -eq 0 ]; then
ncxy 0 "$vpos_l"; prin…
elif [ $? -eq 1 ]
Upon running the second [ command (assuming the first [ ...
- 512k
13
votes
Accepted
How can I assign the output of a command to different variables in each loop iteration?
In your case, I would use an associative array for that:
declare -A rules
for chain in http https ssh
do
rules[$chain]=$(iptables -nvxL $chain | tail -1 | awk '{print $2}')
done
You can then ...
- 21.3k
12
votes
How to print to stdout after exec >/dev/null
Any echo would print to stdout. It's just that your stdout now points to /dev/null. Point being that the original stdout is in no way special, or more "true" than the stdout you have after a ...
- 130k
12
votes
Accepted
When to use "|| true" in Bash?
|| true may be used to suppress immediate exit on error when the errexit shell option is set; that option is enabled with set -e, either by set -e explicitly in the script, or by invoking the script ...
- 406
12
votes
Accepted
Replace all periods with hyphens recursively
A period is special in regular expressions, it matches any character. To match a period literally, backslash it:
rename 's/\./-/g'
With -execdir with some find implementations including GNU find, ...
- 45.2k
12
votes
Bash: Run a command repeatedly till it suceeds
until will do that for you:
until cmd1; do sleep 1; done
This will run cmd1 until it succeeds, waiting one second between each run.
If you want to avoid running forever, you can add a maximum number ...
- 402k
12
votes
Accepted
How can I collapse three statements into one?
Simply test the exit status of the last command in the pipeline directly,
if date -d "$date" | grep -F "Fri"
then
…
fi
I've double-quoted the variable $date to protect it from ...
- 104k
12
votes
run commands in subdirectories concurrently
Run each subshell command in the background and then wait for them all to complete
for d in ./*/
do
( cd "$d" && gh repo sync ) &
done
wait
- 104k
11
votes
Accepted
How do I loop over the first several files in directory?
Glob qualifiers
Glob qualifiers can replace most uses of ls or find to enumerate files. They're a unique feature of zsh.
For example, $(ls /some/path/*.txt | head -2) (enumerate files in lexicographic ...
11
votes
What are the ideas behind the absence of quote removal and escaping in variable expansion?
This behavior makes passing arbitrary arguments using one variable very difficult [...]
Perhaps. But having the results of expansions go through all the usual command line processing would make it ...
- 130k
11
votes
Accepted
In POSIX scripting, is x=$y always equivalent to x="$y"?
Yes, x=$y and x="$y" are guaranteed to be the same in a POSIX shell. If you or some other reader of your code are unsure where double quotes must be used (see When is double-quoting ...
- 316k
11
votes
I need to have each line of a file run in a subshell of its own
GNU Parallel is built for this:
parallel --joblog my.log < myfile.sh
--sshlogin/--sshloginfile, --jobs N, --tag, --linebuffer, and possibly --nonall may also be useful. You would have to divulge a ...
- 33.1k
11
votes
Accepted
run commands in subdirectories concurrently
GNU Parallel does this excellently. I've been using this structure daily for a long time:
parallel --wd '{}' 'gh repo sync' ::: ./*/.git/..
Similar test on my own system:
$ parallel --wd '{}' 'pwd' ::...
- 49.9k
11
votes
Accepted
What is difference between these two declarations of associative arrays in Bash?
declare -A is the only reason your first declaration is treated as an associative array. Your second declaration is handled as an indexed array. When manipulating indexed arrays, indices are treated ...
- 402k
Only top scored, non community-wiki answers of a minimum length are eligible