All Questions
25,156
questions
370
votes
6
answers
382k
views
Why does my shell script choke on whitespace or other special characters?
… or an introductory guide to robust filename handling and other string passing in shell scripts.
I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some ...
265
votes
5
answers
66k
views
Why is using a shell loop to process text considered bad practice?
Is using a while loop to process text generally considered bad practice in POSIX shells?
As Stéphane Chazelas pointed out, some of the reasons for not using shell loop are conceptual, reliability, ...
- 148k
268
votes
9
answers
61k
views
Why *not* parse `ls` (and what to do instead)?
I consistently see answers quoting this link stating definitively "Don't parse ls!" This bothers me for a couple of reasons:
It seems the information in that link has been accepted wholesale with ...
- 56.9k
685
votes
4
answers
301k
views
Why is printf better than echo?
I have heard that printf is better than echo. I can recall only one instance from my experience where I had to use printf because echo didn't work for feeding some text into some program on RHEL 5.8 ...
- 12.1k
173
votes
1
answer
72k
views
When is double-quoting necessary?
The old advice used to be to double-quote any expression involving a $VARIABLE, at least if one wanted it to be interpreted by the shell as one single item, otherwise, any spaces in the content of $...
- 14.2k
266
votes
4
answers
46k
views
Security implications of forgetting to quote a variable in bash/POSIX shells
If you've been following unix.stackexchange.com for a while, you
should hopefully know by now that leaving a variable
unquoted in list context (as in echo $var) in Bourne/POSIX
shells (zsh being the ...
- 504k
416
votes
3
answers
201k
views
What are the shell's control and redirection operators?
I often see tutorials online that connect various commands with different symbols. For example:
command1 | command2
command1 & command2
command1 || command2
command1 && command2
...
- 228k
211
votes
8
answers
36k
views
Why is looping over find's output bad practice?
This question is inspired by
Why is using a shell loop to process text considered bad practice ?
I see these constructs
for file in `find . -type f -name ...`; do smth with ${file}; done
and
for ...
- 77.5k
159
votes
5
answers
226k
views
How can we run a command stored in a variable?
$ ls -l /tmp/test/my\ dir/
total 0
I was wondering why the following ways to run the above command fail or succeed?
$ abc='ls -l "/tmp/test/my dir"'
$ $abc
ls: cannot access '"/tmp/test/my': No ...
- 95.8k
109
votes
1
answer
18k
views
Why does my regular expression work in X but not in Y?
I wrote a regular expression which works well in a certain program (grep, sed, awk, perl, python, ruby, ksh, bash, zsh, find, emacs, vi, vim, gedit, …). But when I use it in a different program (or on ...
475
votes
5
answers
83k
views
Why not use "which"? What to use then?
When looking for the path to an executable or checking what would happen if you enter a command name in a Unix shell, there's a plethora of different utilities (which, type, command, whence, where, ...
- 504k
137
votes
1
answer
98k
views
Understanding the -exec option of `find`
I find myself constantly looking up the syntax of
find . -name "FILENAME" -exec rm {} \;
mainly because I don't see how exactly the -exec part works. What is the meaning of the braces, the ...
- 1,497
1530
votes
10
answers
471k
views
What is the exact difference between a 'terminal', a 'shell', a 'tty' and a 'console'?
I think these terms almost refer to the same thing, when used loosely:
terminal
shell
tty
console
What exactly does each of these terms refer to?
- 34.1k
136
votes
4
answers
14k
views
Why are there so many different ways to measure disk usage?
When I sum up the sizes of my files, I get one figure. If I run du, I get another figure. If I run du on all the files on my partition, it doesn't match what df claims is used. Why are there so many ...
503
votes
9
answers
373k
views
Execute vs Read bit. How do directory permissions in Linux work?
In my CMS, I noticed that directories need the executable bit (+x) set for the user to open them. Why is the execute permission required to read a directory, and how do directory permissions in Linux ...
- 6,093
512
votes
15
answers
301k
views
Turn off buffering in pipe
I have a script which calls two commands:
long_running_command | print_progress
The long_running_command prints progress but I'm unhappy with it. I'm using print_progress to make it nicer (namely, I ...
- 5,808
104
votes
4
answers
125k
views
Understanding "IFS= read -r line"
I obviously understand that one can add value to internal field separator variable. For example:
$ IFS=blah
$ echo "$IFS"
blah
$
I also understand that read -r line will save data from stdin to ...
- 7,132
404
votes
17
answers
598k
views
How to do integer & float calculations, in bash or other languages/frameworks?
Using echo "20+5" literally produces the text "20+5".
What command can I use to get the numeric sum, 25 in this case?
Also, what's the easiest way to do it just using bash for floating
point? For ...
- 40.3k
234
votes
15
answers
512k
views
How to run a specific program as root without a password prompt?
I need to run something as sudo without a password, so I used visudo and added this to my sudoers file:
MYUSERNAME ALL = NOPASSWD: /path/to/my/program
Then I tried it out:
$ sudo /path/to/my/...
- 38.3k
139
votes
4
answers
71k
views
Is there a ".bashrc" equivalent file read by all shells?
Is ~/.bashrc the only place to specify user specific environment variables, aliases, modifications to PATH variable, etc?
I ask because it seems that ~/.bashrc seems to be bash-only, but other ...
- 24.4k
116
votes
3
answers
356k
views
How can I assign the output of a command to a shell variable?
I want to assign the result of an expression
(i.e., the output from a command)
to a variable and then manipulate it – for example,
concatenate it with a string, then echo it.
Here's what I've got:
#!/...
- 1,308
94
votes
3
answers
173k
views
Open a window on a remote X display (why "Cannot open display")?
Once upon a time,
DISPLAY=:0.0 totem /path/to/movie.avi
after ssh 'ing into my desktop from my laptop would cause totem to play movie.avi on my desktop.
Now it gives the error:
No protocol ...
- 1,311
20
votes
2
answers
10k
views
Brackets in if condition: why am I getting syntax errors without whitespace?
I am using the below script to move two days back when script runs at starting two days of the year and also check first and second days of every month and move two days back.
if [$month="01"...
- 341
529
votes
7
answers
296k
views
Difference between Login Shell and Non-Login Shell?
I understand the basic difference between an interactive shell and a non-interactive shell. But what exactly differentiates a login shell from a non-login shell?
Can you give examples for uses of a ...
- 7,109
137
votes
7
answers
25k
views
Redirecting stdout to a file you don't have write permission on
When you attempt to modify a file without having write permissions on it, you get an error:
> touch /tmp/foo && sudo chown root /tmp/foo
> echo test > /tmp/foo
zsh: permission denied:...
- 90.1k
187
votes
9
answers
147k
views
How can I disown a running process and associate it to a new screen shell?
I have a running program on a SSH shell. I want to pause it and be able to unpause its execution when I come back.
One way I thought of doing that was to transfer its ownership to a screen shell, ...
- 3,635
33
votes
3
answers
29k
views
Why is my variable local in one 'while read' loop, but not in another seemingly similar loop?
Why do I get different values for $x from the snippets below?
#!/bin/bash
x=1
echo fred > junk ; while read var ; do x=55 ; done < junk
echo x=$x
# x=55 .. I'd expect this result
x=1
cat ...
- 32k
251
votes
11
answers
209k
views
Allow setuid on shell scripts
The setuid permission bit tells Linux to run a program with the effective user id of the owner instead of the executor:
> cat setuid-test.c
#include <stdio.h>
#include <unistd.h>
int ...
- 90.1k
202
votes
5
answers
94k
views
$VAR vs ${VAR} and to quote or not to quote
I can write
VAR=$VAR1
VAR=${VAR1}
VAR="$VAR1"
VAR="${VAR1}"
the end result to me all seems about the same. Why should I write one or the other? are any of these not portable/POSIX?
- 56.8k
426
votes
16
answers
83k
views
In Bash, when to alias, when to script, and when to write a function?
It's taken me almost 10 years of Linux usage to ask this question. It was all trial and error and random late-night internet surfing.
But people shouldn't need 10 years for this. If I were just ...
- 13.3k
30
votes
1
answer
8k
views
How to ensure that string interpolated into `sed` substitution escapes all metachars
I have a script that reads a text stream and generates a file of sed commands that is later run with sed -f. The generated sed commands are like:
s/cid:image002\.gif@01CC3D46\.926E77E0/https:\/\/...
- 3,947
115
votes
2
answers
32k
views
How do keyboard input and text output work?
Suppose I press the A key in a text editor and this inserts the character a in the document and displays it on the screen. I know the editor application isn't directly communicating with the hardware (...
477
votes
10
answers
200k
views
How do I delete a file whose name begins with "-" (hyphen a.k.a. dash or minus)?
How do you remove a file whose filename begins with a dash (hyphen or minus) -? I'm ssh'd into a remote OSX server and I have this file in my directory:
tohru:~ $ ls -l
total 8
-rw-r--r-- 1 me ...
- 4,873
67
votes
9
answers
263k
views
How can I use variables in the LHS and RHS of a sed substitution?
I want to do:
cat update_via_sed.sh | sed 's/old_name/new_name/' > new_update_via_sed.sh
in my program.
But I want to use variables, e.g.
old_run='old_name_952'
new_run='old_name_953'
I have ...
- 40.3k
1319
votes
12
answers
3.1m
views
How to correctly add a path to PATH?
I'm wondering where a new path has to be added to the PATH environment variable. I know this can be accomplished by editing .bashrc (for example), but it's not clear how to do this.
This way:
export ...
- 16.7k
769
votes
5
answers
319k
views
What does "--" (double-dash) mean?
I have seen -- used in the compgen command.
For example:
compgen -W "foo bar baz" -- b
What is the meaning of the -- in there?
- 28.4k
47
votes
1
answer
14k
views
Is it possible to use `find -exec sh -c` safely?
I'm trying to use find to echo 0 into some files, but apparently this only works with sh -c:
find /proc/sys/net/ipv6 -name accept_ra -exec sh -c 'echo 0 > {}' \;
But using sh -c with find -exec ...
- 13.6k
149
votes
3
answers
33k
views
Have backticks (i.e. `cmd`) in *sh shells been deprecated?
I've seen this comment many times on Unix & Linux as well as on other sites that use the phrasing "backticks have been deprecated", with respect to shells such as Bash & Zsh.
Is this ...
slm♦
- 358k
77
votes
3
answers
118k
views
How do I run 32-bit programs on a 64-bit Debian/Ubuntu?
I have a 64-bit (amd64 a.k.a. x86_64) Debian or Ubuntu installation. I need to run 32-bit (i386/i686) programs occasionally, or to compile programs for a 32-bit system. How can I do this with a ...
92
votes
6
answers
78k
views
Understanding UNIX permissions and file types
I've never really got how chmod worked up until today. I followed a tutorial that explained a big deal to me.
For example, I've read that you've got three different permission groups:
owner (u)
...
- 991
394
votes
4
answers
690k
views
What characters do I need to escape when using sed in a sh script?
Take the following script:
#!/bin/sh
sed 's/(127\.0\.1\.1)\s/\1/' [some file]
If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
- 4,940
77
votes
2
answers
63k
views
Getting "Not found" message when running a 32-bit binary on a 64-bit system
I have currently a strange problem on debian (wheezy/amd64).
I have created a chroot to install a server (i can't give any more detail about it, sorry). Let's call its path /chr_path/.
To make things ...
- 949
5
votes
1
answer
18k
views
Wi-Fi problems using ASUS USB-N13 adapter (realtek)
I have recently bought an ASUS Wi-Fi adapter (USB-N13), and I seem to be having some trouble with it. Every so often, the Wi-Fi connection just stops working, but the indicator says that I'm still ...
- 1,888
69
votes
1
answer
11k
views
What is the difference between the "...", '...', $'...', and $"..." quotes in the shell?
Sometimes I see shell scripts use all of these different ways of quoting some text: "...", '...', $'...', and $"...". Why are there so many different kinds of quote being used?
Do ...
- 73.4k
931
votes
10
answers
1.8m
views
How can I replace a string in a file(s)?
Replacing strings in files based on certain search criteria is a very common task. How can I
replace string foo with bar in all files in the current directory?
do the same recursively for sub ...
- 228k
83
votes
2
answers
20k
views
When is dd suitable for copying data? (or, when are read() and write() partial)
Short version: In what circumstances is dd safe to use for copying data, safe meaning that there is no risk of corruption due to a partial read or write?
Long version — preamble: dd is often used to ...
204
votes
6
answers
180k
views
How can I execute `date` inside of a cron tab job?
I want to create a log file for a cron script that has the current hour in the log file name. This is the command I tried to use:
0 * * * * echo hello >> ~/cron-logs/hourly/test`date "+%d"`.log
...
- 43.5k
717
votes
9
answers
201k
views
What do the numbers in a man page mean?
So, for example, when I type man ls I see LS(1). But if I type man apachectl I see APACHECTL(8) and if I type man cd I end up with cd(n).
I'm wondering what the significance of the numbers in the ...
- 7,297
119
votes
4
answers
48k
views
Why does parameter expansion with spaces without quotes work inside double brackets "[[" but not inside single brackets "["?
I'm confused with using single or double brackets. Look at this code:
dir="/home/mazimi/VirtualBox VMs"
if [[ -d ${dir} ]]; then
echo "yep"
fi
It works perfectly although the string contains a ...
- 2,978
105
votes
4
answers
26k
views
What is the difference between sourcing ('.' or 'source') and executing a file in bash?
What's the difference between executing a script like this:
./test.sh
and executing a script like this:
. test.sh?
I tried a simple, two-line script to see if I could find if there was a ...
- 1,183