Strings are typically delimited by quotes, which raises the problem of dealing with strings that include quotes.

learn more… | top users | synonyms

76
votes
3answers
41k views

Why does my shell script choke on whitespace or other special characters?

Or, an introductory guide to robust filename handling and other string passing in shell scripts. I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some ...
70
votes
1answer
2k views

Security implications of forgetting to quote a variable in bash/POSIX shells

If you've been following unix.stackexchange.com for a while, you should hopefully know by now that leaving a variable unquoted in list context (as in echo $var) in Bourne/POSIX shells (zsh being the ...
103
votes
9answers
36k views

How do I delete a file whose name begins with “-” (hyphen a.k.a. dash or minus)?

How do you remove a file whose filename begins with a dash (hyphen or minus) -? I'm ssh'd into a remote OSX server and I have this file in my directory: tohru:~ $ ls -l total 8 -rw-r--r-- 1 me ...
55
votes
2answers
9k views

$VAR vs ${VAR} and to quote or not to quote

I can write VAR=$VAR1 VAR=${VAR1} VAR="$VAR1" VAR="${VAR1}" the end result to me all seems about the same. Why should I write one or the other? are any of these not portable/POSIX?
22
votes
7answers
2k views

What is the difference between $* and $@?

Consider the following code: foo () { echo $* } bar () { echo $@ } foo 1 2 3 4 bar 1 2 3 4 It outputs: 1 2 3 4 1 2 3 4 I am using Ksh88, but I am interested in other common ...
36
votes
3answers
15k views

How can I execute `date` inside of a cron tab job?

I want to create a log file for a cron script that has the current hour in the log file name. This is the command I tried to use: 0 * * * * echo hello >> ~/cron-logs/hourly/test`date "+%d"`.log ...
40
votes
3answers
13k views

using single or double bracket - bash

I'm confused with using single or double bracket. look at code: dir="/home/mazimi/VirtualBox VMs" if [[ -d ${dir} ]]; then echo "yep" fi It works perfectly although the string contains space. ...
6
votes
1answer
692 views

How to ensure that string interpolated into `sed` substitution escapes all metachars

I have a script that reads a text stream and generates a file of sed commands that is later run with sed -f. The generated sed commands are like: ...
31
votes
1answer
7k views

When is double-quoting necessary?

The old advice used to be to double-quote any expression involving a $VARIABLE, at least if one wanted it to be interpreted by the shell as one single item, otherwise, any spaces in the content of ...
20
votes
3answers
2k views

gnu find and masking the {} for some shells - which?

The man page for gnu find states: -exec command ; [...] The string `{}' is replaced by the current file name being processed everywhere it occurs in the arguments to the command, ...
4
votes
1answer
524 views

Difference between ' and " on command line (bash)?

I used to use '' and "" interchangeably on the command line, but I recently noticed that '$HOME/some/dir' is not expanded, while "$HOME/some/dir" is. I searched around a little bit and found that "" ...
23
votes
4answers
10k views

Quoting in ssh $host $FOO and ssh $host “sudo su user -c $FOO” type constructs

I often end up issuing complex commands over ssh; these commands involve piping to awk or perl one-lines, and as a result contain single quotes and $'s. I have neither been able to figure out a hard ...
15
votes
4answers
29k views

Pass shell variable as a /pattern/ to awk

Having the following in one of my shell functions: function _process () { awk -v l="$line" ' BEGIN {p=0} /'"$1"'/ {p=1} END{ if(p) print l >> "outfile.txt" } ' } , so when called as ...
75
votes
3answers
120k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
15
votes
4answers
24k views

How to escape quotes in shell?

I'm having trouble with escaping characters in bash. I'd like to escape single and double quotes while running a command under a different user. For the purposes of this question let's say I want to ...
13
votes
5answers
3k views

Why do I need to quote variable for if, but not for echo?

I've read that you need double quotes for expanding variables, e.g. if [ -n "$test" ]; then echo '$test ok'; else echo '$test null'; fi will work as expected, while if [ -n $test ]; then echo ...
5
votes
1answer
4k views

Wrapping a command that includes single and double quotes for another command

I recently learned about watch, but am having trouble making it work with relatively sophisticated commands. For example, I would like to ask watch to run the following command on zsh every three ...
2
votes
2answers
2k views

Use of quotes in GNU grep regular expressions

I could see that certain patterns in the GNU Grep can be enclosed within brackets and certain others need not be. For example, matching the beginning of a word works only if it is enclosed within ...
26
votes
5answers
24k views

Parenthesis in bash arithmetic: 3 * (2 + 1)

expr does not seem to like parenthesis (used in mathematics to explicit operator priority): expr 3 * (2 + 1) bash: syntax error near unexpected token `(' How to express operator priority in bash?
19
votes
1answer
935 views

Why doesn't 'find' show this file?

Using find with grep, one can locate files that match a pattern: # find | grep error ./solr-modifiedSolr4/SolrPhpClient/phpdocs/errors.html ./error_log ./includes/classes/error_log However, using ...
43
votes
5answers
61k views

What does ` (backquote/backtick) mean in bash?

I came across the following command sudo chown `id -u` /somedir And I wonder what is the meaning of the ` symbol. I noticed for instance that while the command above works well the one below does ...
7
votes
2answers
758 views

Expansion of a shell variable and effect of glob and split on it

This post actually contains two separate questions but I think grouping them together will give some more context. I have gone through this question on quotes around variables but I do not quite ...
9
votes
2answers
7k views

What is the significance of single and double quotes in environment variables?

I defined some environment variables in my .profile like this: MY_HOME="/home/my_user" but the variable does not seem to evaluate unless I strip off the quotes and re-source the file. I believe ...
5
votes
2answers
9k views

Trouble in script with spaces in filename

I've got a script that scp's a file from remote host back to local. Sometimes the file names contain spaces. scp does not like spaces in its file names. For some reason my attempts at handling the ...
4
votes
2answers
792 views

Why do I need to escape regex characters in sed to be interpreted as regex characters?

It seems e.g. cat sed_data.txt | sed 's/\b[0-9]\{3\}\b/NUMBER/g' that I must escape characters to form a regular expression. In this case I had to escape braces in order to be interpreted as a number ...
2
votes
3answers
8k views

Using a reference to a bash string variable in sed

I'm a relative Linux novice. Suppose that I have a text file a.txt that contains the following text: A B C Now, if I want to change the text on line 2 (which contains B), I can use the following ...
1
vote
2answers
227 views

Quoting / escaping / expansion issue in “command in a variable”

I want to run a command such as this in a bash script: freebcp <authentication and other parameters> -t "," -r "\r\n" When run directly on the command line, the command works as expected. But ...
28
votes
3answers
53k views

Why I can't escape spaces on a bash script? [duplicate]

I'm trying to escape the space char for a path in Bash, but neither using a backslash or quotes works. .sh script: ROOT="/home/hogar/Documents/files/" FILE=${ROOT}"bdd.encrypted" ...
8
votes
3answers
7k views

Why does the exclamation mark `!` sometimes upset bash?

I realize that ! has special significance on the commandline in the context of the commandline history, but aside from that, in a runing script the exclamation mark can sometimes cause a parsing ...
12
votes
1answer
712 views

How to execute an arbitrary simple command over ssh without knowing the login shell of the remote user?

ssh has an annoying feature in that when you run: ssh user@host cmd and "here's" "one arg" Instead of running that cmd with its arguments on host, it concatenates that cmd and arguments with spaces ...
10
votes
4answers
31k views

Why is echo ignoring my quote characters?

The echo command isn't including the complete text that I give it. For example, if I do: $ echo ' echo PARAM=` grep $ARG /var/tmp/setfile | awk '{print $2}' ` ' It outputs: echo PARAM=` ...
10
votes
2answers
5k views

How to properly escape exclamation points in bash?

Today, I was caught redhanded while attempting to code golf a password generator for Twitter. import string as s,random;print ''.join(random.sample(s.letters+s.digits+s.punctuation,9)) 90 chars. ...
12
votes
0answers
358 views

Should I use $* or $@? [duplicate]

Which one of $* or $@ is better to use (in array syntax,command line parameters, etc.) to avoid any bugs/problems in code? Or it does not make any difference?
6
votes
1answer
322 views

Variable in sed

I have a file that contains filenames with absolute path. In the middle of that path, is a variable directory, whenever I hard code the variable part, it works fine, but when I replace it with the ...
124
votes
5answers
275k views

How do I grep for multiple patterns?

I want to find all lines in several files that match one of two patterns. I tried to find the patterns I'm looking for by typing grep (foo|bar) *.txt but the shell interprets the | as a pipe and ...
28
votes
5answers
20k views

Can't use exclamation mark (!) in bash?

I'm trying to use the curl command to access a http url with a exclamation mark (!) in its path. e.g: curl -v "http://example.org/!287s87asdjh2/somepath/someresource" the console replies with bash: ...
12
votes
3answers
7k views

Bash script error with strings with paths that have spaces and wildcards

I am having trouble getting the basics of Bash scripting down. Here's what I have so far: #!/bin/bash FILES="/home/john/my directory/*.txt" for f in "${FILES}" do echo "${f}" done All I want to ...
10
votes
4answers
570 views

Regular expression using \\ vs using \

Why does grep e\\.g\\. <<< "this is an e.g. wow" and grep e\.g\. <<< "this is an e.g. wow" do the same thing? If I add a third slash, it also has the same result. BUT, once I ...
6
votes
3answers
9k views

Why does bash variable expansion retain quotes?

> echo "hi" hi > VAR='echo "hi"' > $VAR "hi" Why is the output of the above commands different? A similar thing occurs with single quotes: > VAR="echo 'hi'" > $VAR > 'hi'
2
votes
3answers
4k views

How to escape a single quote? [duplicate]

This works fine: sed -i 's# @driver.find_element(:xpath, "//a\[contains(@href,##' temp_spec.rb against a source of @driver.find_element(:xpath, "//a[contains(@href,'change_district')]").click ...
11
votes
1answer
3k views

Wildcards inside quotes

This will be an easy one, but in my memories, when shell scripting, using double quotes would allow expanding globbing and variables. But in the following code: #!/bin/sh echo *.sh echo "*.sh" echo ...
6
votes
5answers
3k views

Deleting files with spaces in their names

I am trying to delete all the files with a space in their names. I am using following command. But it is giving me an error Command : ls | egrep '. ' | xargs rm Here if I am using only ls | egrep '. ...
6
votes
2answers
1k views

Quoted vs unquoted string expansion

for i in $(xrandr); do echo "$i" ; done for i in "$(xrandr)"; do echo "$i"; done for i in "$(xrandr)"; do echo $i; done I understand why 1 differs from 2. But why does 3 give a different output ...
3
votes
4answers
576 views

Piping paths with different types of quotes for slash substitution

I would like to use sed to convert a path with backslashes to the same path with forward slashes: E.g. I would like to pipe \\path\to\file\ and obtain /path/to/file None of the following commands ...
20
votes
3answers
8k views

How do you process an entire command as sudo that involves redirecting or piping?

I am trying to follow what I assume is best practises of using sudo instead of root account. I am running a simple concat file operation such as: sudo echo 'clock_hctosys="YES"' >> ...
10
votes
2answers
2k views

Are quotes needed for local variable assignment?

Can I safely omit quotes on the right side of a local assignment? function foo { local myvar=${bar} stuff() } I'm mainly interested in bash, but any info on corner cases in other shells are ...
4
votes
3answers
2k views

New line in bash variables

I'm trying to store multiple lines in a bash variable, but it doesn't seem to work. For example, if I list /bin one file per line and store it in $LS, then I pass $LS as stdin to wc, it always ...
4
votes
4answers
897 views

How can I grep for a string containing regex metacharacters like $ and '?

I want to search all files in the current directory and below for the substring $form['#node']. So, I have tried, initially: grep -R "\$form\[\\\\'" . and grep -R "\$form\[\'" . But it returns no ...
4
votes
1answer
129 views

Quoting in a function results in error

How should I write the command variable to get the expected behavior? #!/bin/bash function f ( ) { echo "$2" } command="f --option=\"One Two Three\" --another_option=\"Four Five Six\"" $command ...
3
votes
3answers
86 views

Why are square brackets preventing shell expansion?

'4800483343' is a directory, and 'file1' & 'file2' are two files in it. Why is the following happening? $ ls 4800483343 file1 file2 $ md5sum 4800483343/* 36468e77d55ee160477dc9772a99be4b ...