Questions about shell scripts, executable files that are interpreted by a shell (bash, zsh, etc.).

learn more… | top users | synonyms (2)

98
votes
3answers
59k views

Why does my shell script choke on whitespace or other special characters?

Or, an introductory guide to robust filename handling and other string passing in shell scripts. I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some ...
87
votes
3answers
3k views

Security implications of forgetting to quote a variable in bash/POSIX shells

If you've been following unix.stackexchange.com for a while, you should hopefully know by now that leaving a variable unquoted in list context (as in echo $var) in Bourne/POSIX shells (zsh being the ...
66
votes
3answers
12k views

$VAR vs ${VAR} and to quote or not to quote

I can write VAR=$VAR1 VAR=${VAR1} VAR="$VAR1" VAR="${VAR1}" the end result to me all seems about the same. Why should I write one or the other? are any of these not portable/POSIX?
4
votes
3answers
1k views

Brackets in if condition: why am I getting syntax errors without whitespace?

I am using the below script to move two days back when script runs at starting two days of the year and also check first and second days of every month and move two days back. if [$month="01"] &&...
244
votes
14answers
32k views

In Bash, when to alias, when to script, and when to write a function?

It's taken me almost 10 years of Linux usage to ask this question. It was all trial and error and random late-night internet surfing. But people shouldn't need 10 years for this. If I were just ...
191
votes
8answers
57k views

Why is it better to use “#!/usr/bin/env NAME” instead of “#!/path/to/NAME” as my shebang?

I notice that some scripts which I have acquired from others have the shebang #!/path/to/NAME while others (using the same tool, NAME) have the shebang #!/usr/bin/env NAME. Both seem to work properly....
37
votes
1answer
9k views

When is double-quoting necessary?

The old advice used to be to double-quote any expression involving a $VARIABLE, at least if one wanted it to be interpreted by the shell as one single item, otherwise, any spaces in the content of $...
74
votes
5answers
52k views

Using “${a:-b}” for variable assignment in scripts

I have been looking at a few scripts other people wrote (specifically Red Hat), and a lot of their variables are assigned using the following notation VARIABLE1="${VARIABLE1:-some_val}" or some expand ...
12
votes
4answers
55k views

Storing output of command in shell variable [duplicate]

I have an operation using cut that I would like to assign result to a variable var4=echo ztemp.xml |cut -f1 -d '.' I get the error: ztemp.xml is not a command The value of var4 never gets ...
8
votes
3answers
12k views

Why is my variable local one 'while read' loop, but not in another seemingly similar loop?

Why do I get different values for $x from the snippets below? #!/bin/bash x=1 echo fred>junk ; while read var ; do x=55 ; done <junk echo x=$x # x=55 .. I'd expect this result x=1 cat ...
143
votes
21answers
137k views

How can I get my external IP address in a shell script?

I need to find my external IP address from a shell script. At the moment I use this function: myip () { lwp-request -o text checkip.dyndns.org | awk '{ print $NF }' } But it depends on perl-...
32
votes
8answers
46k views

Different ways to execute a shell script

There a several ways to execute a script, the ones I know are: /path/to/script # using the path (absolute or relative) . script # using the . (dot) source script # using the `source` command ...
24
votes
3answers
1k views

What is the difference between [[ $a == z* ]] and [ $a == z* ]?

Is there is any difference between these two. [[ $a == z* ]] and [ $a == z* ] Can I have an example where they would have different outputs? Furthermore, how does the working of [[ ]] differs ...
14
votes
1answer
21k views

File descriptors & shell scripting

I am having a very hard time understanding how does one use file descriptors in shell scripts. I know the basics such as exec 5 > /tmp/foo So fd 5 is attached to foo for writing. exec 6 < /...
14
votes
1answer
427 views

Where are bash line continuations after && and || documented?

I have seen this construct in scripts a lot and used it myself, but it bothers me that I can't seem to find it in the documentation. Example: [ -f file1 ] && [ -f file2 ] && echo "...
2
votes
3answers
481 views

command not found and syntax error near unexpected token

I have this script to divide a file over a 100000 or even more in 50000 lines. desc() { echo $...
47
votes
11answers
34k views

Correct locking in shell scripts?

Sometimes you have to make sure that only one instance of a shell script is running at the same time. For example a cron job which is executed via crond that does not provide locking on its own (e.g. ...
42
votes
15answers
14k views

Detect init system using the shell

This may have more to do with detecting operating systems, but I specifically need the init system currently in use on the system. Fedora 15 now uses systemd, Ubuntu uses Upstart, while others use ...
12
votes
4answers
14k views

How do I handle switches in a shell script?

Are there some built-in tools that will recognize -x and --xxxx as switches, and not arguments, or do you have to go through all the input variables, test for dashes, and then parse the arguments ...
19
votes
3answers
5k views

Understand “IFS= read -r line”?

I obviously understand that one can add value to internal field separator variable. For example: $ IFS=blah $ echo "$IFS" blah $ I also understand that read -r line will save data from stdin to ...
26
votes
1answer
6k views

Execute bash scripts on entering a directory

What is the best way to execute a script when entering into a directory? When I move into a new directory I would like bash to execute the projectSettings.bash script much like RVM does.
17
votes
2answers
2k views

running script with “. ” and with “source ”

I was wondering if the following two ways of running a bash script are equivalent? . ./myScript.sh source myScript.sh Are they both running the content of the script instead of running the script, ...
105
votes
3answers
147k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
37
votes
5answers
15k views

How can I detect if the shell is controlled from SSH?

I want to detect from a shell script (more specifically .zshrc) if it is controlled through SSH. I tried the HOST variable but it's always the name of the computer which is running the shell. Can I ...
39
votes
4answers
3k views

Should I care about unnecessary cats?

A lot of command-line utilities can take their input either from a pipe or as a filename argument. For long shell scripts, I find starting the chain off with a cat makes it more readable, especially ...
16
votes
2answers
3k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets ...
14
votes
4answers
1k views

keep duplicates out of $PATH on source [duplicate]

I have the following code that's source-d by my .shellrc PATH="${PATH}:${HOME}/perl5/bin" PATH="${PATH}:${HOME}/.bin" export PATH but if I make changes to other code and then source this file, my ...
10
votes
6answers
14k views

Shell script for moving oldest files?

How do I write a script for moving just the 20 oldest files from one folder to another? Is there a way to grab the oldest files in a folder?
10
votes
2answers
3k views

Piping for loop output prevents local variable modification

I'm trying to write a simple bash function that takes, as its arguments, a number of files and/or directories. It should: Fully qualify the filenames. Sort them. Remove duplicates. Print all that ...
5
votes
5answers
10k views

Stripping directory paths to get file names

What I am doing is converting mp3's with LAME. I already have an old script I wrote that works but I want to add to it this ability--to no longer delete the file but instead save it in a new root ...
15
votes
4answers
10k views

Executing user defined function in a find -exec call

I'm on Solaris 10 and I have tested the following with ksh (88), bash (3.00) and zsh (4.2.1). The following code doesn't yield any result: function foo { echo "Hello World" } find somedir -exec ...
16
votes
5answers
850 views

Stateful bash function

I’d like to implement a function in Bash which increases (and returns) a count with every call. Unfortunately this seems non-trivial since I’m invoking the function inside a subshell and it ...
7
votes
1answer
827 views

Parallel execution of bash script

I have a table URL_Experiment in my database (mySQL database). I have 2 million URL links in this table. For each URL, I am checking if some particular text is present in the URL and updating the ...
41
votes
6answers
23k views

Split pages in pdf

I have a scanned pdf file which has scanned two pages on one virtual page (page in pdf file). The resolution is with good quality. The problem is I have to zoom when reading and drag from left to the ...
25
votes
3answers
93k views

How to rename multiple files in single command or script in Unix? [duplicate]

I have the below list of files aro_tty-mIF-45875564pmo_opt aro_tty-mIF-45875664pmo_opt aro_tty-mIF-45875964pmo_opt aro_tty-mIF-45875514pmo_opt aro_tty-mIF-45875524pmo_opt that I need to rename to ...
79
votes
2answers
7k views

Have backticks (i.e. `cmd`) in *sh shells been deprecated?

I've seen this comment many times on Unix & Linux as well as on other sites that use the phrasing "backticks have been deprecated", with respect to shells such as Bash & Zsh. Is this ...
34
votes
4answers
19k views

In bash, is it possible to use an integer variable in the loop control of a for loop?

I have the following bash script: #!/bin/bash upperlim=10 for i in {0..10} do echo $i done for i in {0..$upperlim} do echo $i done The first for loop (without the variable upperlim in the loop ...
27
votes
7answers
6k views

What purpose does the colon builtin serve?

I've hacked on a lot of shell scripts, and sometimes the simplest things baffle me. Today I ran across a script that made extensive use of the : (colon) bash builtin. The documenation seems simple ...
13
votes
8answers
10k views

Is there a unix command that gives the minimum/maximum of two numbers?

I was looking for a command to limit numbers read in from stdin. I wrote a little script for that purpose (critique is welcome), but I was wondering if there was not a standard command for this, ...
13
votes
4answers
31k views

Text between two tags

I wanna retrieve whatever is between these two tags – <tr> </tr> – from an html doc. Now I don't have any specific html requirements that would warrant for an html parser. I just plain ...
7
votes
2answers
903 views

Expansion of a shell variable and effect of glob and split on it

This post actually contains two separate questions but I think grouping them together will give some more context. I have gone through this question on quotes around variables but I do not quite ...
8
votes
4answers
28k views

compare two columns of different files and print if it matches

I am using solaris 10 and Grep options involving -f doesnt work . I have two pipe separated files: file1: abc|123|BNY|apple| cab|234|cyx|orange| def|kumar|pki|bird| file 2 : abc|123| kumar|pki|...
20
votes
3answers
13k views

How can one run multiple programs in the background with single command?

How can one run multiple programs in the background with single command? I have tried the commands below, but they do not work. nohup ./script1.sh & && nohup ./script2.sh & -bash: ...
5
votes
2answers
3k views

When sh is a symlink to bash or dash, bash limits itself to POSIX-compliance, so it should be 100% compatible with sh?

From Difference between bash and sh: ABck to the question: If you have /bin/sh as a link to bash, then bash will not behave the same when called as /bin/sh as it does when called as /bin/bash. ...
3
votes
2answers
2k views

Shell script works when saved with nano but not when saved with Notepad++

I've been working on a bash script all day and have been testing on the machine I will be using it on. When I copy the script across from Notepad++ into a new file with nano editor inside SSH and ...
8
votes
4answers
30k views

Use a variable reference “inside” another variable

I'm sure it is relatively simple, I just don't know how to do it. #!/usr/bin/ksh set `iostat` myvar=6 I want to something like echo ${$myvar} which i want interpreted as ${$myvar} -> ${6} -> value
4
votes
4answers
20k views

Check for Process if Same is Running

Is there a script or a way in linux that when I try to execute a shell script/process, if the same is running, it will prompt that same is running and will exit otherwise it will continue.
6
votes
4answers
1k views

Shell, Concatenating 2 strings to reference a 3rd variable

I am trying to do the following in my script: #/bin/bash -x var1=www var2=www2 var3=www3 var4=www4 for i in 1 2 3 4; do echo \$var$i done Obviously, this doesn't work as expected and results in ...
0
votes
1answer
6k views

Script failing with “command not found: ^M”

When I try to run the following script in zsh, via the command /bin/zsh ~/.set_color_scheme.sh I get the following error: command not found: ^M The script has u+x permissions and it used to work on ...
37
votes
3answers
9k views

How can I test for POSIX compliance for shell scripts?

Probably the main thing that this website has taught me so far is the importance of writing portable shell scripts. Considering that POSIX is the closest thing to a common standard between all ...