Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

Questions tagged [shell-script]

Questions about shell scripts, executable files that are interpreted by a shell (bash, zsh, etc.).

Filter by
Sorted by
Tagged with
287
votes
5answers
230k views

Why does my shell script choke on whitespace or other special characters?

Or, an introductory guide to robust filename handling and other string passing in shell scripts. I wrote a shell script which works well most of the time. But it chokes on some inputs (e.g. on some ...
208
votes
3answers
19k views

Security implications of forgetting to quote a variable in bash/POSIX shells

If you've been following unix.stackexchange.com for a while, you should hopefully know by now that leaving a variable unquoted in list context (as in echo $var) in Bourne/POSIX shells (zsh being the ...
121
votes
1answer
43k views

When is double-quoting necessary?

The old advice used to be to double-quote any expression involving a $VARIABLE, at least if one wanted it to be interpreted by the shell as one single item, otherwise, any spaces in the content of $...
16
votes
3answers
5k views

Brackets in if condition: why am I getting syntax errors without whitespace?

I am using the below script to move two days back when script runs at starting two days of the year and also check first and second days of every month and move two days back. if [$month="01"] &&...
363
votes
14answers
60k views

In Bash, when to alias, when to script, and when to write a function?

It's taken me almost 10 years of Linux usage to ask this question. It was all trial and error and random late-night internet surfing. But people shouldn't need 10 years for this. If I were just ...
25
votes
4answers
21k views

Why is my variable local in one 'while read' loop, but not in another seemingly similar loop?

Why do I get different values for $x from the snippets below? #!/bin/bash x=1 echo fred > junk ; while read var ; do x=55 ; done < junk echo x=$x # x=55 .. I'd expect this result x=1 cat ...
61
votes
3answers
65k views

Understanding “IFS= read -r line”

I obviously understand that one can add value to internal field separator variable. For example: $ IFS=blah $ echo "$IFS" blah $ I also understand that read -r line will save data from stdin to ...
144
votes
4answers
51k views

$VAR vs ${VAR} and to quote or not to quote

I can write VAR=$VAR1 VAR=${VAR1} VAR="$VAR1" VAR="${VAR1}" the end result to me all seems about the same. Why should I write one or the other? are any of these not portable/POSIX?
251
votes
3answers
407k views

What characters do I need to escape when using sed in a sh script?

Take the following script: #!/bin/sh sed 's/(127\.0\.1\.1)\s/\1/' [some file] If I try to run this in sh (dash here), it'll fail because of the parentheses, which need to be escaped. But I don't ...
54
votes
3answers
80k views

How can I make environment variables “exported” in a shell script stick around?

I have multiple Amazon EC2 accounts and want to quickly be able to switch variables, such as $EC2_HOME, using a script. I have have a shell script set up like this: #!/bin/sh export EC2_HOME=/home/...
120
votes
2answers
20k views

Have backticks (i.e. `cmd`) in *sh shells been deprecated?

I've seen this comment many times on Unix & Linux as well as on other sites that use the phrasing "backticks have been deprecated", with respect to shells such as Bash & Zsh. Is this ...
468
votes
9answers
193k views

Why is it better to use “#!/usr/bin/env NAME” instead of “#!/path/to/NAME” as my shebang?

I notice that some scripts which I have acquired from others have the shebang #!/path/to/NAME while others (using the same tool, NAME) have the shebang #!/usr/bin/env NAME. Both seem to work properly....
16
votes
3answers
6k views

Which shell interpreter runs a script with no shebang?

Suppose the default shell for my account is zsh but I opened the terminal and fired up bash and executed a script named prac002.sh, which shell interpreter would be used to execute the script, zsh or ...
171
votes
12answers
288k views

How do I change the extension of multiple files?

I would like to change a file extension from *.txt to *.text. I tried using the basename command, but I'm having trouble on changing more than one file. Here's my code: files=`ls -1 *.txt` for x in ...
44
votes
8answers
71k views

Different ways to execute a shell script

There a several ways to execute a script, the ones I know are: /path/to/script # using the path (absolute or relative) . script # using the . (dot) source script # using the `source` command ...
29
votes
8answers
80k views

Test if there are files matching a pattern in order to execute a script

I am trying to write an if statement to test whether there are any files matching a certain pattern. If there is a text file in a directory it should run a given script. My code currently: if [ -f /*...
115
votes
3answers
65k views

Are there naming conventions for variables in shell scripts?

Most languages have naming conventions for variables, the most common style I see in shell scripts is MY_VARIABLE=foo. Is this the convention or is it only for global variables? What about variables ...
275
votes
23answers
359k views

How can I get my external IP address in a shell script?

I need to find my external IP address from a shell script. At the moment I use this function: myip () { lwp-request -o text checkip.dyndns.org | awk '{ print $NF }' } But it depends on perl-...
40
votes
2answers
15k views

Execute bash scripts on entering a directory

What is the best way to execute a script when entering into a directory? When I move into a new directory I would like bash to execute the projectSettings.bash script much like RVM does.
49
votes
1answer
4k views

What is the difference between “…”, '…', $'…', and $“…” quotes?

Sometimes I see scripts use all of these different ways of quoting some text: "...", '...', $'...', and $"...". Why are there so many different kinds of quote being used? Do they behave differently ...
439
votes
3answers
319k views

Using “${a:-b}” for variable assignment in scripts

I have been looking at a few scripts other people wrote (specifically Red Hat), and a lot of their variables are assigned using the following notation VARIABLE1="${VARIABLE1:-some_val}" or some expand ...
35
votes
3answers
3k views

What is the difference between [[ $a == z* ]] and [ $a == z* ]?

Is there is any difference between these two. [[ $a == z* ]] and [ $a == z* ] Can I have an example where they would have different outputs? Furthermore, how does the working of [[ ]] differs ...
40
votes
2answers
15k views

How does awk '!a[$0]++' work?

This one-liner removes duplicate lines from text input without pre-sorting. For example: $ cat >f q w e w r $ awk '!a[$0]++' <f q w e r $ The original code I have found on the internets read:...
26
votes
4answers
113k views

Storing output of command in shell variable [duplicate]

I have an operation using cut that I would like to assign result to a variable var4=echo ztemp.xml |cut -f1 -d '.' I get the error: ztemp.xml is not a command The value of var4 never gets ...
92
votes
18answers
41k views

Detect init system using the shell

This may have more to do with detecting operating systems, but I specifically need the init system currently in use on the system. Fedora 15 and Ubuntu now use systemd, Ubuntu used to use Upstart (...
71
votes
8answers
199k views

#!/bin/bash - no such file or directory

I've created a bash script but when I try to execute it, I get #!/bin/bash no such file or directory I need to run the command: bash script.sh for it to work. How can I fix this?
16
votes
4answers
87k views

compare two columns of different files and print if it matches

I am using Solaris 10 and so grep options involving -f don't work. I have two pipe-separated files: file1: abc|123|BNY|apple| cab|234|cyx|orange| def|kumar|pki|bird| file 2: abc|123| kumar|pki| ...
36
votes
2answers
7k views

Are quotes needed for local variable assignment?

Can I safely omit quotes on the right side of a local assignment? function foo { local myvar=${bar} stuff() } I'm mainly interested in bash, but any info on corner cases in other shells are ...
66
votes
13answers
58k views

Correct locking in shell scripts?

Sometimes you have to make sure that only one instance of a shell script is running at the same time. For example a cron job which is executed via crond that does not provide locking on its own (e.g. ...
61
votes
3answers
223k views

How to rename multiple files in single command or script in Unix? [duplicate]

I have the below list of files aro_tty-mIF-45875564pmo_opt aro_tty-mIF-45875664pmo_opt aro_tty-mIF-45875964pmo_opt aro_tty-mIF-45875514pmo_opt aro_tty-mIF-45875524pmo_opt that I need to rename to ...
66
votes
22answers
123k views

How to get my own IP address and save it to a variable in a shell script?

How can I get my own IP address and save it to a variable in a shell script?
34
votes
6answers
56k views

find n most frequent words in a file

I want to find, say, 10 most common word in a text file. Firstly, solution should be optimized for keystrokes (in other words - my time). Secondly, for the performance. Here is what I have so far to ...
255
votes
7answers
941k views

How can I pass a command line argument into a shell script?

I know that shell scripts just run commands as if they were executed in at the command prompt. I'd like to be able to run shell scripts as if they were functions... That is, taking an input value or ...
32
votes
1answer
41k views

File descriptors & shell scripting

I am having a very hard time understanding how does one use file descriptors in shell scripts. I know the basics such as exec 5 > /tmp/foo So fd 5 is attached to foo for writing. exec 6 < /...
41
votes
5answers
45k views

Replace environment variables in a file with their actual values?

Is there an easy way to substitute/evaluate environment variables in a file? Like let's say I have a file config.xml that contains: <property> <name>instanceId</name> <...
28
votes
1answer
6k views

Where are bash line continuations after && and || documented?

I have seen this construct in scripts a lot and used it myself, but it bothers me that I can't seem to find it in the documentation. Example: [ -f file1 ] && [ -f file2 ] && echo "...
113
votes
10answers
133k views

Parallelize a Bash FOR Loop

I have been trying to parallelize the following script, specifically each of the three FOR loop instances, using GNU Parallel but haven't been able to. The 4 commands contained within the FOR loop run ...
94
votes
2answers
81k views

Do parentheses really put the command in a subshell?

From what I've read, putting a command in parentheses should run it in a subshell, similar to running a script. If this is true, how does it see the variable x if x isn't exported? x=1 Running (...
69
votes
6answers
35k views

How can I detect if the shell is controlled from SSH?

I want to detect from a shell script (more specifically .zshrc) if it is controlled through SSH. I tried the HOST variable but it's always the name of the computer which is running the shell. Can I ...
25
votes
2answers
6k views

running script with “. ” and with “source ”

I was wondering if the following two ways of running a bash script are equivalent? . ./myScript.sh source myScript.sh Are they both running the content of the script instead of running the script, ...
36
votes
3answers
25k views

How can one run multiple programs in the background with single command?

How can one run multiple programs in the background with single command? I have tried the commands below, but they do not work. nohup ./script1.sh & && nohup ./script2.sh & -bash: ...
27
votes
4answers
88k views

Use a variable reference “inside” another variable

I'm sure it is relatively simple, I just don't know how to do it. #!/usr/bin/ksh set `iostat` myvar=6 I want to something like echo ${$myvar} which i want interpreted as ${$myvar} -> ${6} -> value
4
votes
6answers
34k views

Check for Process if Same is Running

Is there a script or a way in linux that when I try to execute a shell script/process, if the same is running, it will prompt that same is running and will exit otherwise it will continue.
65
votes
9answers
56k views

Can a script be executable but not readable?

Is it possible to execute a script if there is no permission to read it? In root mode, I made a script and I want the other user to execute this script but not read it. I did chmod to forbid read and ...
16
votes
4answers
35k views

How do I handle switches in a shell script?

Are there some built-in tools that will recognize -x and --xxxx as switches, and not arguments, or do you have to go through all the input variables, test for dashes, and then parse the arguments ...
22
votes
1answer
4k views

How does a fork bomb work?

WARNING DO NOT ATTEMPT TO RUN THIS ON A PRODUCTION MACHINE In reading the Wikipedia page on the topic I generally follow what's going on with the following code: :(){ :|:& };: excerpt of ...
18
votes
2answers
6k views

Expansion of a shell variable and effect of glob and split on it

This post actually contains two separate questions but I think grouping them together will give some more context. I have gone through this question on quotes around variables but I do not quite ...
30
votes
1answer
21k views

SSH causes while loop to stop

I have finally managed to boil down a problem I have been struggling with for a few weeks. I use SSH with "authorized keys" to run commands remotely. All is fine except when I do it in a while loop. ...
8
votes
1answer
25k views

Provide two arguments to one option using getopts

In below code when I give option r then getopts requires one arguments: while getopts ":hr::l:" opt; do case $opt in r ) echo "Run Numbers - argument = $OPTARG " ;; l ) echo "...
14
votes
7answers
18k views

Shell script for moving oldest files?

How do I write a script for moving just the 20 oldest files from one folder to another? Is there a way to grab the oldest files in a folder?