Strings are typically delimited by quotes, which raises the problem of dealing with strings that include quotes.

learn more… | top users | synonyms

155
votes
5answers
341k views

How do I grep for multiple patterns?

I want to find all lines in several files that match one of two patterns. I tried to find the patterns I'm looking for by typing grep (foo|bar) *.txt but the shell interprets the | as a pipe and ...
124
votes
8answers
44k views

How do I delete a file whose name begins with “-” (hyphen a.k.a. dash or minus)?

How do you remove a file whose filename begins with a dash (hyphen or minus) -? I'm ssh'd into a remote OSX server and I have this file in my directory: tohru:~ $ ls -l total 8 -rw-r--r-- 1 me ...
95
votes
3answers
141k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
92
votes
3answers
54k views

Why does my shell script choke on whitespace or other special characters?

Or, an introductory guide to robust filename handling and other string passing in shell scripts. I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some ...
83
votes
2answers
3k views

Security implications of forgetting to quote a variable in bash/POSIX shells

If you've been following unix.stackexchange.com for a while, you should hopefully know by now that leaving a variable unquoted in list context (as in echo $var) in Bourne/POSIX shells (zsh being the ...
61
votes
3answers
11k views

$VAR vs ${VAR} and to quote or not to quote

I can write VAR=$VAR1 VAR=${VAR1} VAR="$VAR1" VAR="${VAR1}" the end result to me all seems about the same. Why should I write one or the other? are any of these not portable/POSIX?
57
votes
5answers
72k views

What does ` (backquote/backtick) mean in bash?

I came across the following command sudo chown `id -u` /somedir And I wonder what is the meaning of the ` symbol. I noticed for instance that while the command above works well the one below does ...
49
votes
4answers
3k views

Why is 'ls' suddenly wrapping items with spaces in single quotes?

I just noticed that on one of my machines (running Debian Sid) whenever I type ls any file name with spaces has single quotes surrounding it. I immediately checked my aliases, only to find them ...
48
votes
3answers
23k views

Quoting within $(command substitution) in Bash

In my Bash environment I use variables containing spaces, and I use these variables within command substitution. Unfortunately I cannot find the answer on SE. What is the correct way to quote my ...
44
votes
3answers
16k views

using single or double bracket - bash

I'm confused with using single or double bracket. look at code: dir="/home/mazimi/VirtualBox VMs" if [[ -d ${dir} ]]; then echo "yep" fi It works perfectly although the string contains space. ...
38
votes
3answers
23k views

What does it mean to have a $“dollarsign-prefixed string” in a script?

I just saw this in an init script: echo $"Stopping Apache" What is that dollar-sign for? My research so far: I found this in the bash manual: extquote If set, $'string' and ...
38
votes
3answers
20k views

How can I execute `date` inside of a cron tab job?

I want to create a log file for a cron script that has the current hour in the log file name. This is the command I tried to use: 0 * * * * echo hello >> ~/cron-logs/hourly/test`date "+%d"`.log ...
35
votes
1answer
9k views

When is double-quoting necessary?

The old advice used to be to double-quote any expression involving a $VARIABLE, at least if one wanted it to be interpreted by the shell as one single item, otherwise, any spaces in the content of ...
31
votes
5answers
24k views

Can't use exclamation mark (!) in bash?

I'm trying to use the curl command to access a http url with a exclamation mark (!) in its path. e.g: curl -v "http://example.org/!287s87asdjh2/somepath/someresource" the console replies with bash: ...
30
votes
7answers
35k views

How to add newlines into variables in bash script

When I do str="Hello World\n===========\n" I get the \n printed out too. How can I have newlines then?
28
votes
5answers
33k views

Parenthesis in bash arithmetic: 3 * (2 + 1)

expr does not seem to like parenthesis (used in mathematics to explicit operator priority): expr 3 * (2 + 1) bash: syntax error near unexpected token `(' How to express operator priority in bash?
28
votes
3answers
66k views

Why I can't escape spaces on a bash script? [duplicate]

I'm trying to escape the space char for a path in Bash, but neither using a backslash or quotes works. .sh script: ROOT="/home/hogar/Documents/files/" FILE=${ROOT}"bdd.encrypted" ...
26
votes
2answers
8k views

How can I use column to delimit on tabs and not spaces?

I'd like to use Unix column to format some text. I have fields delimited by tabs, but within each field there are also spaces. column delimits on white space (tabs and spaces). How can I make column ...
25
votes
3answers
10k views

How do you process an entire command as sudo that involves redirecting or piping?

I am trying to follow what I assume is best practises of using sudo instead of root account. I am running a simple concat file operation such as: sudo echo 'clock_hctosys="YES"' >> ...
24
votes
4answers
10k views

Quoting in ssh $host $FOO and ssh $host “sudo su user -c $FOO” type constructs

I often end up issuing complex commands over ssh; these commands involve piping to awk or perl one-lines, and as a result contain single quotes and $'s. I have neither been able to figure out a hard ...
22
votes
7answers
3k views

What is the difference between $* and $@?

Consider the following code: foo () { echo $* } bar () { echo $@ } foo 1 2 3 4 bar 1 2 3 4 It outputs: 1 2 3 4 1 2 3 4 I am using Ksh88, but I am interested in other common ...
21
votes
2answers
850 views

Why escape trivial characters in shell script?

I just opened a legacy shell script (written in old ksh88 on Solaris) and found the following repeated all throughout the code: [ -f $myfile ] && \rm -f $myfile The escaping backslash ...
21
votes
3answers
2k views

gnu find and masking the {} for some shells - which?

The man page for gnu find states: -exec command ; [...] The string `{}' is replaced by the current file name being processed everywhere it occurs in the arguments to the command, ...
20
votes
4answers
3k views

How can I delete a file named “>”?

I was running a Python script that malfunctioned and used sudo to create a file named >. How can I get rid of this file? Of course, when I try sudo rm >, I get the error bash: syntax error ...
20
votes
2answers
16k views

How are parentheses interpreted at the command line?

While reading up on how to set up grub, I came across an article claiming that I need to use one of the following two syntaxes, echo \(hd0,0\) >> /boot/grub/grub.conf or echo '(hd0,0)' ...
20
votes
4answers
32k views

How to escape quotes in shell?

I'm having trouble with escaping characters in bash. I'd like to escape single and double quotes while running a command under a different user. For the purposes of this question let's say I want to ...
19
votes
6answers
13k views

How can you combine all lines that end with a backslash character?

Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash? For example, given the file: foo bar \ bash \ baz dude \ happy ...
19
votes
1answer
973 views

Why doesn't 'find' show this file?

Using find with grep, one can locate files that match a pattern: # find | grep error ./solr-modifiedSolr4/SolrPhpClient/phpdocs/errors.html ./error_log ./includes/classes/error_log However, using ...
19
votes
4answers
35k views

Pass shell variable as a /pattern/ to awk

Having the following in one of my shell functions: function _process () { awk -v l="$line" ' BEGIN {p=0} /'"$1"'/ {p=1} END{ if(p) print l >> "outfile.txt" } ' } , so when called as ...
18
votes
2answers
6k views

Delete all files starting with a question mark

I have a folder in which I have around 4k files. Some of these files start with a a ? or ! character. I need to delete them but can't find an expression that would do so: rm -f ./?* just deletes ...
17
votes
4answers
8k views

How to print “$” in here-document

How can I echo "$" in a here-doc in bash? For example, I want to have a file with the content on remote server like $ABC=home_dir. $ ssh hostname sudo -s <<EOF echo "$ABC=home_dir" > file ...
16
votes
1answer
1k views

Why does this ffmpeg command work in bash and not zsh?

Today I got home from work (run bash on an Ubuntu box) and tried to run some code on my local arch box with my beloved zsh and the commands were failing? The command is below with the personal info ...
16
votes
6answers
579 views

Should variables be quoted when executed?

The general rule in shell scripting is that variables should always be quoted unless there is a compelling reason not to. For more details than you probably want to know, have a look at this great ...
15
votes
2answers
2k views

Are quotes needed for local variable assignment?

Can I safely omit quotes on the right side of a local assignment? function foo { local myvar=${bar} stuff() } I'm mainly interested in bash, but any info on corner cases in other shells are ...
14
votes
6answers
2k views

Why there is such a difference in execution time of echo and cat?

Answering this question caused me to ask another question: I thought the following scripts do the same thing and the second one should be much faster, because the first one uses cat that needs to open ...
13
votes
2answers
3k views

Why can't I do ls *.txt in a subshell?

~$ echo $(ls) arguments.txt cecho.sh Desktop Documents Downloads example.txt Music Pictures Public ~$ echo $(ls "*.txt") ls: cannot access *.txt: No such file or directory Why does the ...
13
votes
2answers
977 views

How can variables use “~” for cd'ing?

How can I use a variable - $BASE in my cd. I tried the following but I get an error $ cd ~/z/repo_1_ruby_193/ 23:23:57 durrantm Castle2012 /home/durrantm/z/repo_1_ruby_193 $ BASE="~/z" 23:24:03 ...
13
votes
5answers
3k views

Why do I need to quote variable for if, but not for echo?

I've read that you need double quotes for expanding variables, e.g. if [ -n "$test" ]; then echo '$test ok'; else echo '$test null'; fi will work as expected, while if [ -n $test ]; then echo ...
12
votes
4answers
11k views

Bash script error with strings with paths that have spaces and wildcards

I am having trouble getting the basics of Bash scripting down. Here's what I have so far: #!/bin/bash FILES="/home/john/my directory/*.txt" for f in "${FILES}" do echo "${f}" done All I want to ...
12
votes
1answer
808 views

How to execute an arbitrary simple command over ssh without knowing the login shell of the remote user?

ssh has an annoying feature in that when you run: ssh user@host cmd and "here's" "one arg" Instead of running that cmd with its arguments on host, it concatenates that cmd and arguments with spaces ...
12
votes
1answer
4k views

Wildcards inside quotes

This will be an easy one, but in my memories, when shell scripting, using double quotes would allow expanding globbing and variables. But in the following code: #!/bin/sh echo *.sh echo "*.sh" echo ...
12
votes
0answers
359 views

Should I use $* or $@? [duplicate]

Which one of $* or $@ is better to use (in array syntax,command line parameters, etc.) to avoid any bugs/problems in code? Or it does not make any difference?
11
votes
4answers
9k views

How to assign space-containing values to variables in bash using eval

I want to dynamically assign values to variables using eval. The following dummy example works: var_name="fruit" var_value="orange" eval $(echo $var_name=$var_value) echo $fruit orange However, ...
11
votes
4answers
38k views

Why is echo ignoring my quote characters?

The echo command isn't including the complete text that I give it. For example, if I do: $ echo ' echo PARAM=` grep $ARG /var/tmp/setfile | awk '{print $2}' ` ' It outputs: echo PARAM=` ...
10
votes
6answers
52k views

Using sed to find and replace complex string (preferrably with regex)

I have a file with the following contents: <username><![CDATA[name]]></username> <password><![CDATA[password]]></password> ...
10
votes
2answers
410 views

What is the difference between “*.pl” and *.pl in grep? Why does quoting change the result?

What is the difference between: grep "string" . -r --include *.pl and grep "string" . -r --include "*.pl" The latter includes files in subdirectoried while the former not. Why?
10
votes
4answers
587 views

Regular expression using \\ vs using \

Why does grep e\\.g\\. <<< "this is an e.g. wow" and grep e\.g\. <<< "this is an e.g. wow" do the same thing? If I add a third slash, it also has the same result. BUT, once I ...
10
votes
2answers
5k views

How to properly escape exclamation points in bash?

Today, I was caught redhanded while attempting to code golf a password generator for Twitter. import string as s,random;print ''.join(random.sample(s.letters+s.digits+s.punctuation,9)) 90 chars. ...
10
votes
3answers
1k views

POSIX compliant way to work with a list of filenames possibly with whitespace

I have seen Bash scripting guides suggesting the use of array for working with filenames containing whitespace. DashAsBinSh however suggests that arrays are not portable so I am looking for a POSIX ...
9
votes
2answers
7k views

What is the significance of single and double quotes in environment variables?

I defined some environment variables in my .profile like this: MY_HOME="/home/my_user" but the variable does not seem to evaluate unless I strip off the quotes and re-source the file. I believe ...