11

Suppose I have a file direction with the lines

east
north
south
west
south-west

and using a loop and echo in a shell script I want to generate this output:

Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west

So in other words I want to do something different with the last line in the script.

10 Answers 10

22

bash can't detect the end of a file (without trying to read the next line and failing), but perl can with its eof function:

$ perl -n -e 'print "Last " if eof; print "Direction: $_"' direction 
Direction: east
Direction: north
Direction: south
Direction: west
Last Direction: south-west

note: unlike echo in bash, the print statement in perl doesn't print a newline unless you either 1. explicitly tell it to by including \n in the string you're printing, or 2. are using perl's -l command-line option, or 3. if the string already contains a newline....as is the case with $_ - which is why you often need to chomp() it to get rid of the newline.

BTW, in perl, $_ is the current input line. Or the default iterator (often called "the current thingy" probably because "dollarunderscore" is a bit of a mouthful) in any loop that doesn't specify an actual variable name. Many perl functions and operators use $_ as their default/implicit argument if one isn't provided. See man perlvar and search for $_.

sed can too - the $ address matches the last line of a file:

$ sed -e 's/^/Direction: /; $s/^/Last /' direction 
Direction: east
Direction: north
Direction: south
Direction: west
Last Direction: south-west

The order of the sed rules is important. My first attempt did it the wrong way around (and printed "Direction: Last south-west"). This sed script always adds "Direction: " to the beginning of each line. On the last line ($) it adds "Last " to the beginning of the line already modified by the previous statement.

5
  • 6
    BTW, there's one major but non-obvious difference between the perl and sed versions. If given multiple input files, the perl version will print "Last Direction: " for the last line of each and every input file. The sed version will only print "Last Direction:" for the very last line of the entire input. You can get perl to behave like sed by using eof() instead of just eof....see perldoc -f eof for why. To get sed to behave like perl does, use the -s option - sed -s -e '.....' - this is a GNU extension to sed and may or may not be available in other versions.
    – cas
    Oct 12 at 10:56
  • If your listing all the ways in perl to add a newline when printing, theres also use v5.10; and then use the say function instead of print, which always appends a newline. Oct 12 at 19:36
  • 1
    @user1937198 i wasn't, really. it was about print. That para was explaining why the print function works to do what the OP wants - it doesn't output a newline unless you tell it to, which makes it useful to print a single line in stages. In shell, you'd have to use printf with a format string without a newline (you can't rely on the many incompatible variations of echo that offer different ways of doing that). perl's say is a similar but different function...and btw, you can just use -E instead of -e to enable it, which is less typing than use v5.10; or -Mv5.10.
    – cas
    Oct 12 at 22:45
  • 1
    I would prefer skipping the print and using a separate substitution for the last line. (So that it also works when the last substitution doesn't contain the first):perl -pne 'if(eof) { s/^/Last Direction: /; } else { s/^/Direction: /; }' direction
    – Garo
    Oct 13 at 2:53
  • 1
    @Garo TIMTOWTDI
    – cas
    Oct 13 at 3:29
16

The loop cannot know when it gets to the end unless it actually gets to the end. So either you process your file twice, once to get the number of lines and another to output, or you print out the previous value on each iteration and the last one on exiting the loop:

prev=""
while read direction; do 
    if [ -n "$prev" ]; then 
        echo "Direction: $prev"
    fi
    prev="$direction"
done < direction
echo "Last Direction: $prev" 
3
  • prev would be a less confusing name for that variable, IMO. You're doing this to handle the last iteration specially, but that variable is used all the time, not just on the last iteration. if [ -n "$last" ] is checking for the first iteration, i.e. no previous iteration, not for last or non-last. (Also, another way to do this might be to peel the first read out of the loop, but bash doesn't have C-style do{}while loops that would naturally work with a read at the bottom. And you'd have to separately check the first read for EOF if you can't assume non-empty input.) Oct 13 at 22:12
  • 1
    lol, I was using last in its meaning of previous ("last week"), it didn't occur to me that it would be read as meaning final. It's obvious in retrospect, of course, but I didn't see it. Thanks, @PeterCordes!
    – terdon
    Oct 14 at 14:14
  • It often takes a 2nd person to find a way to misinterpret something you or I wrote, totally normal :P But yeah, since "Last Direction" was already right there in the problem, that's where my brain went on first reading, before seeing that you'd rotated the loop to allow peeling the part of the last iteration we want to do differently. (And thus to see that you were using that meaning of "last") Oct 15 at 10:26
10

A variation of FelixJN's answer, reading the lines into an array, but using bash builtins instead of bash loops:

mapfile -t DIR < direction
printf "Direction: %s\n" "${DIR[@]::${#DIR[@]}-1}"
printf "Last direction: %s\n" "${DIR[-1]}"

mapfile is a builtin added in Bash 4 which reads standard input into an array with one line per entry. The -t option strips the newlines.

printf will apply its format string repeatedly until it has used up all its input arguments.

1
7

You may read in the values into an array and then use the array length:

#!/bin/bash
i=0
while read line ; do
  arr[$i]="$line"
  ((i++))
done <file

for (( i=0 ; i<=${#arr[@]}-2 ; i++ )) ; do
  echo Direction: ${arr[i]}
done
echo Last direction: ${arr[-1]}

Of course this means having all data in RAM.

The simpler arr=( $(cat file) ) would only work if the line entries do not have spaces, thus I used a while read-loop here.

0
5

You can use a mix of paste, tail and head:

$ paste -s -d'\n' <(head -n -1 direction | sed 's/^/Direction: /') <(tail -n1 direction | sed 's/^/Last direction: /')
Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west

  • paste -s -d'\n' uses new line as delimiter with the option -d'\n', and the option -s:

    -s, --serial
    paste one file at a time instead of in parallel

  • head -n -1 retrieves all lines but the last, then is piped to sed to perform the command you need in those lines only.

  • tail -n1 file retrieves the last line of the file, then is piped to sed to perform the command you need in that line only.


Or more succintly just using cat instead of paste, as noted by @Mick Matteo's comment:

$ cat <(head -n -1 direction | sed 's/^/Direction: /') <(tail -n1 direction | sed 's/^/Last direction: /')
2
  • 3
    Why not just cat (concatenate) instead of paste -s -d'\n'? Oct 12 at 19:48
  • @NickMatteo right! added that to the answer. Oct 12 at 21:19
3

Expanding on the answers by FelixJN and Nick Matteo: using portable shell features (noting that the question has no shell-specific tags) and assuming you are processing a controlled, "small-enough" set of lines, you can know when the loop reaches its last element if you place the data into the array of positional parameters:

IFS='
'
set -f
set -- $(cat direction)
for dir
do
  if [ $(( i = i + 1 )) -eq "$#" ]
  then
    printf '%s' 'Last direction: '
  else
    printf '%s' 'Direction: '
  fi
  printf '%s\n' "$dir"
done

Keep in mind that you may want to properly restore IFS after setting it.

Note, however, that this will remove any empty input lines from the output. If you are not fine with this, you can load the array of positional parameters in a loop:

set --
while IFS= read -r line
do
  set -- "$@" "$line"
done <direction

(No need to set IFS to a newline and set -f in this case).


The approach from the answers by cas (sed part) and terdon — handle the last element "outside" the loop — can also be implemented in AWK using a quite common pattern:

awk '
  NR > 1 { print "Direction: " dir }
  { dir = $0 }
  END { if (NR > 0) print "Last direction: " dir }
' direction
1

A different strategy: note that the last line of your file is the first line of your reversed file. So, instead of relying on line indices, you can use a basic echo loop if you reverse the order of lines in your file with tac, which is part of coreutils. Then, reverse the order of your loop output to retrieve the original order.

prefix='Last direction:'
while read line; do
   echo "$prefix $line"
   prefix='Direction:'
done < <(tac direction) | tac
0
col=`awk 'END{print NR}' filename`
awk -v co="$col" '{if (NR<co){print "Direction: "$0}else{print "Last direction: "$0}}' filename

output

Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west
1
  • If you put a little more thought into this, I’m sure you can figure out how to avoid having to read the file twice! Oct 15 at 13:45
0

One way would be to loop over the lines and delay printing the lines by one iteration: we print the previous line stored from a variable (prev in the code below) and then store the current line into this variable afterwards. Once the loop is finished, we have the last line in prev and can print it with its special prefix:

#! /bin/sh
prev=""
first="yes"
# from https://stackoverflow.com/a/1521498/648741
while IFS="" read -r p || [ -n "$p" ]; do
  if [ "$first" = "yes" ]; then
    first="no"
  else
    printf 'Direction: %s\n' "$prev"
  fi
  prev="$p"
done < directions
if [ "$first" = "no" ]; then
    printf 'Last direction: %s\n' "$prev"
fi

The code here takes some extra care to not damage whitespace and to allow for dollar signs within the lines. If this is not needed, a simpler version also works:

#! /bin/sh
prev=""
while read -r p; do
  if [ -n "$prev" ]; then
    echo "Direction: $prev"
  fi
  prev="$p"
done < directions
if [ -n "$prev" ]; then
    echo "Last direction: $prev"
fi
0

Using Raku (formerly known as Perl_6)

raku -e 'my $fh = $*ARGFILES.IO.open; $fh.eof ?? put("Last direction: $_") !! put("Direction: $_") for $fh.lines; $fh.close;' file.txt

OR

raku -e 'my $fh = $*ARGFILES.IO.open; for $fh.lines() { if not $fh.eof { put "Direction: $_" } else { put "Last direction: $_" };}; $fh.close;' file.txt

Sample Input:

east
north
south
west
south-west

Sample Output:

Direction: east
Direction: north
Direction: south
Direction: west
Last direction: south-west

Currently, Raku implements a .eof method which operates on either IO::Handle or IO::CatHandle objects and "...returns True if the read operations have exhausted the contents of the handle... ."

The first code example above uses Raku's ternary operator $condition ?? $true !! $false to test whether $fh.eof is True. Note above for both code examples how .eof operates directly on the $fh filehandle scalar, whereas $_ gets loaded with retrieved (line-wise) data, which is printed using put (or say, if desired).

Note, the rather simplistic code examples above only work on a single file at a time at the bash command line. However it's likely that feeding a list of files to $fh (for example via Raku's dir() method) may be the best way to manipulate multiple files at once.

https://docs.raku.org/routine/eof
https://docs.raku.org/language/operators#index-entry-operator_ternary
https://raku.org

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.