Command to display first few and last few lines of a file

Question

I have a file with many rows, and each row has a timestamp at the starting, like

[Thread-3] (21/09/12 06:17:38:672) logged message from code.....

So, I frequently check 2 things from this log file.

1. First few rows, that has the global conditions and start time is also given.
2. Last few rows, that has the exit status with some other info.

Is there any quick handy single command that could let me display just the first and last few lines of a file?

Answer 1

You can use sed or awk to make it with one command. However you'll loose at speed, cause sed and awk will need to run through the whole file anyway. From a speed point of view it's much better to make a function or every time to combination of tail + head. This does have the downside of not working if the input is a pipe, however you can use proccess substitution, in case your shell supports it (look at example below).

first_last () {
    head -n 10 -- "$1"
    tail -n 10 -- "$1"
}

and just launch it as

first_last "/path/to/file_to_process"

to proceed with process substitution (bash, zsh, ksh like shells only):

first_last <( command )

ps. you can even add a grep to check if your "global conditions" exist.

Answer 2

@rush is right about using head + tail being more efficient for large files, but for small files (< 20 lines), some lines may be output twice.

{ head; tail;} < /path/to/file

would be equally efficient, but wouldn't have the problem above.

Answer 3

The { head; tail; } solution wouldn't work on pipes (or sockets or any other non-seekable files) because head could consume too much data as it reads by blocks and can't seek back on a pipe potentially leaving the cursor inside the file beyond what tail is meant to select.

So, you could use a tool that reads one character at a time like the shell's read (here using a function that takes the number of head lines and tail lines as arguments).

head_tail() {
  n=0
  while [ "$n" -lt "$1" ]; do
    IFS= read -r line || { printf %s "$line"; break; }
    printf '%s\n' "$line"
    n=$(($n + 1))
  done
  tail -n "${2-$1}"
}
seq 100 | head_tail 5 10
seq 20 | head_tail 5

or implement tail in awk for instance as:

head_tail() {
  awk -v h="$1" -v t="${2-$1}" '
    {l[NR%t]=$0}
    NR<=h
    END{
      n=NR-t+1
      if(n <= h) n = h+1
      for (;n<=NR;n++) print l[n%t]
    }'
}

With sed:

head_tail() {
  sed -e "1,${1}b" -e :1 -e "$(($1+${2-$1})),\$!{N;b1" -e '}' -e 'N;D'
}

(though beware that some sed implementations have a low limitation on the size of their pattern space, so would fail for big values of the number of tail lines).

Answer 4

Using bash process substitution, you can do the following:

make_some_output | tee >(tail -n 2) >(head -n 2; cat >/dev/null) >/dev/null

Note that the lines are not guaranteed to be in order, though for files longer than about 8kB, they very likely will be. This 8kB cutoff is the typical size of the read buffer, and is related to the reason | {head; tail;} doesn't work for small files.

The cat >/dev/null is necessary to keep the head pipeline alive. Otherwise tee will quit early, and while you'll get output from tail, it'll be from somewhere in the middle of the input, rather than the end.

Finally, why the >/dev/null instead of, say, moving tail to another |? In the following case:

make_some_output | tee >(head -n 2; cat >/dev/null) | tail -n 2  # doesn't work

head's stdout is fed into the pipe to tail rather than the console, which isn't what we want at all.

Answer 5

Stephane's first solution in a function so that you can use arguments (works in any Bourne-like or POSIX shell):

head_tail() {
    head "$@";
    tail "$@";
}

Now you can do this:

head_tail -n 5 < /path/to/file

This of course assumes that you're looking at only one file and like Stephane's solution works (reliably) only on regular (seekable) files.

Answer 6

Using ed (which will read the entire file into RAM though):

# cf. http://wiki.bash-hackers.org/howto/edit-ed
printf '%s\n' 'H' '1,10p' '$-10,$p' 'q' | ed -s file