108

I have came across this script:

#! /bin/bash                                                                                                                                                                                           

if (( $# < 3 )); then
  echo "$0 old_string new_string file [file...]"
  exit 0
else
  ostr="$1"; shift
  nstr="$1"; shift  
fi

echo "Replacing \"$ostr\" with \"$nstr\""
for file in $@; do
  if [ -f $file ]; then
    echo "Working with: $file"
    eval "sed 's/"$ostr"/"$nstr"/g' $file" > $file.tmp 
    mv $file.tmp $file
  fi  
done

What is the meaning of the lines where they use shift? I presume the script should be used with at least arguments so...?

129

shift is a bash built-in which kind of removes arguments from the beginning of the argument list. Given that the 3 arguments provided to the script are available in $1, $2, $3, then a call to shift will make $2 the new $1. A shift 2 will shift by two making new $1 the old $3. For more information, see here:

  • 25
    +1 Note it's not rotating them, it's shifting them off the array (of arguments). Shift/unshift and pop/push are common names for this general kind of manipulation (see, e.g., bash's pushd and popd). – goldilocks Dec 16 '14 at 22:10
  • 1
    The definitive reference: gnu.org/software/bash/manual/bashref.html#index-shift – glenn jackman Dec 16 '14 at 22:11
  • @goldilocks very true. Instead of "rotate" I should have better found a way to use another word, something like "move the arguments forward in the argument variables". Anyway I hope the examples in the text and the link to the reference will have made it clear nonetheless. – humanityANDpeace Dec 18 '14 at 7:58
32

As goldilocks’ comment and humanity’s references describe, shift reassigns the positional parameters ($1, $2, etc.) so that $1 takes on the old value of $2, $2 takes on the value of $3, etc.*  The old value of $1 is discarded.  ($0 is not changed.)  Some reasons for doing this include:

  • It lets you access the tenth argument (if there is one) more easily.  $10 doesn’t work – it’s interpreted as $1 concatenated with a 0 (and so might produce something like Hello0).  After a shift, the tenth argument becomes $9.  (However, in most modern shells, you can use ${10}.)
  • As the Bash Guide for Beginners demonstrates, it can be used to loop through the arguments.  IMNSHO, this is clumsy; for is much better for that.
  • As in your example script, it makes it easy to process all of the arguments the same way except for a few.  For example, in your script, $1 and $2 are text strings, while $3 and all other parameters are file names.

So here’s how it plays out.  Suppose your script is called Patryk_script and it is called as

Patryk_script USSR Russia Treaty1 Atlas2 Pravda3

The script sees

$1 = USSR
$2 = Russia
$3 = Treaty1
$4 = Atlas2
$5 = Pravda3

The statement ostr="$1" sets variable ostr to USSR.  The first shift statement changes the positional parameters as follows:

$1 = Russia
$2 = Treaty1
$3 = Atlas2
$4 = Pravda3

The statement nstr="$1" sets variable nstr to Russia.  The second shift statement changes the positional parameters as follows:

$1 = Treaty1
$2 = Atlas2
$3 = Pravda3

And then the for loop changes USSR ($ostr) to Russia ($nstr) in the files Treaty1, Atlas2, and Pravda3.



There are a few problems with the script.

  1. for file in $@; do

    If the script is invoked as

    Patryk_script USSR Russia Treaty1 "World Atlas2" Pravda3

    it sees

    $1 = USSR
    $2 = Russia
    $3 = Treaty1
    $4 = World Atlas2
    $5 = Pravda3

    but, because $@ isn’t quoted, the space in World Atlas2 isn’t quoted, and the for loop thinks it has four files: Treaty1, World, Atlas2, and Pravda3.  This should be either

    for file in "$@"; do

    (to quote any special characters in the arguments) or simply

    for file do

    (which is equivalent to the longer version).

  2. eval "sed 's/"$ostr"/"$nstr"/g' $file"

    There’s no need for this to be an eval, and passing unchecked user input to an eval can be dangerous.  For example, if the script is invoked as

    Patryk_script "'; rm *;'" Russia Treaty1 Atlas2 Pravda3

    it will execute rm *!  This is a big concern if the script can be run with privileges higher than those of the user who invokes it; e.g., if it can be run via sudo or invoked from a web interface.  It’s probably not so important if you just use it as yourself, in your directory.  But it can be changed to

    sed "s/$ostr/$nstr/g" "$file"

    This still has some risks, but they are much less severe.

  3. if [ -f $file ], > $file.tmp and mv $file.tmp $file should be if [ -f "$file" ], > "$file.tmp" and mv "$file.tmp" "$file", respectively, to handle file names that might have spaces (or other funny characters) in them.  (The eval "sed … command also mangles file names that have spaces in them.)


* shift takes an optional argument: a positive integer that specifies how many parameters to shift.  The default is one (1).  For example, shift 4 causes $5 to become $1, $6 to become $2, and so on.  (Note that the example in the Bash Guide for Beginners is wrong.)  And so your script could be modified to say

ostr="$1"
nstr="$2"
shift 2

which might be considered to be more clear.



End Note / Warning:

The Windows Command Prompt (batch file) language also supports a SHIFT command, which does basically the same thing as the shift command in Unix shells, with one striking difference, which I’ll hide to try to prevent people from being confused by it:

  • A command like SHIFT 4 is an error, yielding an “Invalid parameter to SHIFT command” error message.
  • SHIFT /n, where n is an integer between 0 and 8, is valid — but it doesn’t shift n times.  It shifts once, starting with the n th argument.  So SHIFT /4 causes %5 (the fifth argument) to become %4,  %6 to become %5, and so on, leaving arguments 0 through 3 alone.

  • 2
    shift can be applied to while, until, and even for loops for handling arrays in much more subtle ways than can a simple for loop. I often find it useful in for loops like... set -f -- $args; IFS=$split; for arg do set -- $arg; shift "${number_of_fields_to_discard}" && fn_that_wants_split_arg "$arg"; done – mikeserv Dec 18 '14 at 2:37
3

The simplest explanation is this. Consider the command:

/bin/command.sh SET location Cebu
/bin/command.sh SET location Cebu, Philippines 6014 

Without the help of shift you cannot extract the complete value of the location because this value could get arbitrarily long. When you shift two times, the args SET and location are removed such that:

x="$@"
echo "location = $x"

Take a very long stare at the $@ thing. That means, it can save the complete location into variable x despite the spaces and comma that it has. So in summary, we call shift and then later we retrieve the value of what is left from the variable $@.

UPDATE I am adding below a very short snippet showing the concept and usefulness of shift without which, it would be very,very difficult to extract the fields correctly.

#!/bin/sh
#

[ $# -eq 0 ] && return 0

a=$1
shift
b=$@
echo $a
[ -n "$b" ] && echo $b

Explanation: After the shift, the variable b shall contain the rest of the stuff being passed in, no matter of spaces or etc. The [ -n "$b" ] && echo $b is a protection such that we only print b if it has a content.

  • (1) This is not the simplest explanation.  It’s an interesting angle.  (2) But as long as you want to concatenate a bunch of arguments (or all of them), you might want to get into the habit of using "$*" instead of "$@".  (3) I was going to upvote your answer, but I didn’t, because you say “shift two times” but you don’t actually show it in code.  (4) If you want a script to be able to take a multi-word value from the command line, it’s probably better to require the user to put the entire value into quotes. … (Cont’d) – Scott Feb 10 '18 at 2:15
  • (Cont’d) …  You might not care today, but your approach doesn’t allow you to have multiple spaces (Philippines   6014), leading spaces, trailing spaces, or tabs.  (5) BTW, you may also be able to do what you’re doing here in bash with x="${*:3}". – Scott Feb 10 '18 at 2:15
  • I tried to decipher where your '*** doesn't allow multiple spaces ***' comment is coming from because this is not related to the shift command. You maybe referring to $@ versus $* subtle differences. But the fact still remains, that my snippet above can accurately extract the location no matter how many spaces it has either trailing or in the front it doesn't matter. After the shift, location will contain the correct location. – typelogic Mar 11 '18 at 3:11
2

shift treat command line arguments as a FIFO queue, it popleft element every time it's invoked.

array = [a, b, c]
shift equivalent to
array.popleft
[b, c]
$1, $2,$3 can be interpreted as index of the array.
$# is the length of array

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.