All Questions

257
votes
4answers
193k views

Why does my shell script choke on whitespace or other special characters?

Or, an introductory guide to robust filename handling and other string passing in shell scripts. I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some ...
307
votes
5answers
79k views

Why is Kali Linux so hard to set up? Why won't people help me?

I've installed Kali Linux, or I'm trying to install it. Why is it so hard? Why doesn't it recognize my hardware? Why do I need to set up so many things manually? Why can't I install the applications I ...
198
votes
3answers
15k views

Security implications of forgetting to quote a variable in bash/POSIX shells

If you've been following unix.stackexchange.com for a while, you should hopefully know by now that leaving a variable unquoted in list context (as in echo $var) in Bourne/POSIX shells (zsh being the ...
181
votes
4answers
33k views

Why is using a shell loop to process text considered bad practice?

Is using a while loop to process text generally considered bad practice in POSIX shells? As Stéphane Chazelas pointed out, some of the reasons for not using shell loop are conceptual, reliability, ...
194
votes
8answers
30k views

Why *not* parse `ls` (and what do to instead)?

I consistently see answers quoting this link stating definitively "Don't parse ls!" This bothers me for a couple of reasons: It seems the information in that link has been accepted wholesale with ...
526
votes
4answers
212k views

Why is printf better than echo?

I have heard that printf is better than echo. I can recall only one instance from my experience where I had to use printf because echo didn't work for feeding some text into some program on RHEL 5.8 ...
219
votes
3answers
85k views

What are the shell's control and redirection operators?

I often see tutorials online that connect various commands with different symbols. For example: command1 | command2 command1 & command2 command1 || command2 command1 && command2 ...
162
votes
6answers
16k views

Why is looping over find's output bad practice?

This question is inspired by Why is using a shell loop to process text considered bad practice ? I see these constructs for file in `find . -type f -name ...`; do smth with ${file}; done and for ...
1154
votes
9answers
301k views

What is the exact difference between a 'terminal', a 'shell', a 'tty' and a 'console'?

I think these terms almost refer to the same thing, when used loosely: terminal shell tty console What exactly does each of these terms refer to?
300
votes
4answers
36k views

Why not use “which”? What to use then?

When looking for the path to an executable or checking what would happen if you enter a command name in a Unix shell, there's a plethora of different utilities (which, type, command, whence, where, ...
106
votes
1answer
36k views

When is double-quoting necessary?

The old advice used to be to double-quote any expression involving a $VARIABLE, at least if one wanted it to be interpreted by the shell as one single item, otherwise, any spaces in the content of $...
102
votes
4answers
28k views

Is there a “.bashrc” equivalent file read by all shells?

Is ~/.bashrc the only place to specify user specific environment variables, aliases, modifications to PATH variable, etc? I ask because it seems that ~/.bashrc seems to be bash-only, but other ...
37
votes
1answer
17k views

Understanding the -exec option of `find`

I find myself constantly looking up the syntax of find . -name "FILENAME" -exec rm {} \; mainly because I don't see how exactly the -exec part works. What is the meaning of the braces, the ...
109
votes
4answers
8k views

Why are there so many different ways to measure disk usage?

When I sum up the sizes of my files, I get one figure. If I run du, I get another figure. If I run du on all the files on my partition, it doesn't match what df claims is used. Why are there so many ...
66
votes
1answer
8k views

Why does my regular expression work in X but not in Y?

I wrote a regular expression which works well in a certain program (grep, sed, awk, perl, python, ruby, ksh, bash, zsh, find, emacs, vi, vim, gedit, …). But when I use it in a different program (or on ...
75
votes
4answers
132k views

Open a window on a remote X display (why “Cannot open display”)?

Once upon a time, DISPLAY=:0.0 totem /path/to/movie.avi after ssh 'ing into my desktop from my laptop would cause totem to play movie.avi on my desktop. Now it gives the error: No protocol ...
156
votes
9answers
104k views

How can I disown a running process and associate it to a new screen shell?

I have a running program on a SSH shell. I want to pause it and be able to unpause its execution when I come back. One way I thought of doing that was to transfer its ownership to a screen shell, ...
368
votes
13answers
185k views

Turn off buffering in pipe

I have a script which calls two commands: long_running_command | print_progress The long_running_command prints a progress but I'm unhappy with it. I'm using print_progress to make it more nice (...
105
votes
7answers
15k views

Redirecting stdout to a file you don't have write permission on

When you attempt to modify a file without having write permissions on it, you get an error: > touch /tmp/foo && sudo chown root /tmp/foo > echo test > /tmp/foo zsh: permission denied:...
273
votes
4answers
150k views

Difference between Login Shell and Non-Login Shell?

I understand the basic difference between an interactive shell and a non-interactive shell. But what exactly differentiates a login shell from a non-login shell? Can you give examples for uses of a ...
148
votes
12answers
297k views

How to run a specific program as root without a password prompt?

I need to run something as sudo without a password, so I used visudo and added this to my sudoers file: MYUSERNAME ALL = NOPASSWD: /path/to/my/program Then I tried it out: $ sudo /path/to/my/...
67
votes
2answers
46k views

Getting “Not found” message when running a 32-bit binary on a 64-bit system

I have currently a strange problem on debian (wheezy/amd64). I have created a chroot to install a server (i can't give any more detail about it, sorry). Let's call its path /chr_path/. To make things ...
357
votes
7answers
242k views

Execute vs Read bit. How do directory permissions in Linux work?

In my CMS, I noticed that directories need the executable bit (+x) set for the user to open them. Why is the execute permission required to read a directory, and how do directory permissions in Linux ...
12
votes
3answers
4k views

Brackets in if condition: why am I getting syntax errors without whitespace?

I am using the below script to move two days back when script runs at starting two days of the year and also check first and second days of every month and move two days back. if [$month="01"] &&...
173
votes
8answers
130k views

Allow setuid on shell scripts

The setuid permission bit tells Linux to run a program with the effective user id of the owner instead of the executor: > cat setuid-test.c #include <stdio.h> #include <unistd.h> int ...
323
votes
10answers
129k views

How do I delete a file whose name begins with “-” (hyphen a.k.a. dash or minus)?

How do you remove a file whose filename begins with a dash (hyphen or minus) -? I'm ssh'd into a remote OSX server and I have this file in my directory: tohru:~ $ ls -l total 8 -rw-r--r-- 1 me ...
78
votes
2answers
17k views

How do keyboard input and text output work?

Suppose I press the A key in a text editor and this inserts the character a in the document and displays it on the screen. I know the editor application isn't directly communicating with the hardware (...
271
votes
16answers
319k views

How to do integer & float calculations, in bash or other languages/frameworks?

Using echo "20+5" literally produces the text "20+5". What command can I use to get the numeric sum, 25 in this case? Also, what's the easiest way to do it just using bash for floating point? For ...
70
votes
2answers
92k views

How do I run 32-bit programs on a 64-bit Debian/Ubuntu?

I have a 64-bit (amd64 a.k.a. x86_64) Debian or Ubuntu installation. I need to run 32-bit (i386/i686) programs occasionally, or to compile programs for a 32-bit system. How can I do this with a ...
4
votes
1answer
8k views

Wi-Fi problems using ASUS USB-N13 adapter (realtek)

I have recently bought an ASUS Wi-Fi adapter (USB-N13), and I seem to be having some trouble with it. Every so often, the Wi-Fi connection just stops working, but the indicator says that I'm still ...
695
votes
7answers
1.4m views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
832
votes
11answers
1.9m views

How to correctly add a path to PATH?

I'm wondering where a new path has to be added to the PATH environment variable. I know this can be accomplished by editing .bashrc (for example), but it's not clear how to do this. This way: export ...
20
votes
4answers
20k views

Why is my variable local in one 'while read' loop, but not in another seemingly similar loop?

Why do I get different values for $x from the snippets below? #!/bin/bash x=1 echo fred > junk ; while read var ; do x=55 ; done < junk echo x=$x # x=55 .. I'd expect this result x=1 cat ...
18
votes
1answer
4k views

How to ensure that string interpolated into `sed` substitution escapes all metachars

I have a script that reads a text stream and generates a file of sed commands that is later run with sed -f. The generated sed commands are like: s/cid:image002\.gif@01CC3D46\.926E77E0/https:\/\/...
77
votes
9answers
56k views

Recovering accidentally deleted files

I accidentally deleted a file from my laptop. I'm using Fedora. Is it possible to recover the file?
66
votes
5answers
42k views

Is there a way to determine the optimal value for the bs parameter to dd?

On occasion I've seen comments online along the lines of "make sure you set 'bs=' because the default value will take too long," and my own extremely-unscientific experiences of, "well that seemed to ...
16
votes
4answers
14k views

I added a user to a group, but group permissions on files still have no effect

I changed permissions of a file (chmod g+w testfile) and running ls -l testfile gives: -rwxrwxr-x 1 user1 user1 0 2011-01-24 20:36 testfile I then added a user to that group ("/etc/group" has user1:...
345
votes
14answers
55k views

In Bash, when to alias, when to script, and when to write a function?

It's taken me almost 10 years of Linux usage to ask this question. It was all trial and error and random late-night internet surfing. But people shouldn't need 10 years for this. If I were just ...
98
votes
5answers
78k views

How can I execute `date` inside of a cron tab job?

I want to create a log file for a cron script that has the current hour in the log file name. This is the command I tried to use: 0 * * * * echo hello >> ~/cron-logs/hourly/test`date "+%d"`.log ...
432
votes
8answers
110k views

What do the numbers in a man page mean?

So, for example, when I type man ls I see LS(1). But if I type man apachectl I see APACHECTL(8) and if I type man cd I end up with cd(n). I'm wondering what the significance of the numbers in the ...
200
votes
8answers
151k views

Keep SSH Sessions running after disconnection

I sometimes have long running processes that I want to kick off before going home, so I create a SSH session to the server to start the process, but then I want to close my laptop and go home and ...
139
votes
14answers
136k views

Batch renaming files

I have a directory full of images: image0001.png image0002.png image0003.png ... And I would like a one-liner to rename them to (say). 0001.png 0002.png 0003.png ... How do I do this?
129
votes
4answers
41k views

$VAR vs ${VAR} and to quote or not to quote

I can write VAR=$VAR1 VAR=${VAR1} VAR="$VAR1" VAR="${VAR1}" the end result to me all seems about the same. Why should I write one or the other? are any of these not portable/POSIX?
68
votes
2answers
238k views

How can I assign the output of a command to a shell variable?

I want to assign the result of an expression to a variable and concatenate it with a string, then echo it. Here's what I've got: #!/bin/bash cd ~/Desktop; thefile= ls -t -U | grep -m 1 "Screen Shot"; ...
59
votes
2answers
13k views

When is dd suitable for copying data? (or, when are read() and write() partial)

Short version: In what circumstances is dd safe to use for copying data, safe meaning that there is no risk of corruption due to a partial read or write? Long version — preamble: dd is often used to ...
80
votes
4answers
32k views

Why does parameter expansion with spaces without quotes work inside double brackets “[[” but not inside single brackets “[”?

I'm confused with using single or double brackets. Look at this code: dir="/home/mazimi/VirtualBox VMs" if [[ -d ${dir} ]]; then echo "yep" fi It works perfectly although the string contains a ...
60
votes
11answers
221k views

How can I use variables in the LHS and RHS of a sed substitution?

I want to do: cat update_via_sed.sh | sed 's/old_name/new_name/' > new_update_via_sed.sh in my program. But I want to use variables, e.g. old_run='old_name_952' new_run='old_name_953' I have ...
19
votes
2answers
12k views

How can I install more recent versions of software than what Debian provides?

Suppose I want a more recent version of software than is available for my current version of an operating system, what can I do? Cases to consider: There are semiofficial/official sources of ...
292
votes
1answer
212k views

What is a bind mount?

What is a “bind mount”? How do I make one? What is it good for? I've been told to use a bind mount for something, but I don't understand what it is or how to use it.
49
votes
2answers
70k views

How can I make environment variables “exported” in a shell script stick around?

I have multiple Amazon EC2 accounts and want to quickly be able to switch variables, such as $EC2_HOME, using a script. I have have a shell script set up like this: #!/bin/sh export EC2_HOME=/home/...

15 30 50 per page