1

This question already has an answer here:

I need to pipe the output of one command to two other commands.

cat filename.txt | tail -n 1
cat filename.txt | wc -l

Since the file is huge I want to avoid reading it twice. I checked tee command but it redirects output to file which I don't want. There are many related posts but did not find anything relevant

marked as duplicate by terdon Jul 23 '15 at 10:13

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

2

If I understood correctly, you want to get the last line of the file along with the line count:

$ cat a
aaa
bb
ccc
$
$ awk 'END{print $0; print NR}' a
ccc
3

Since you need them to save in variables:

$ out=$(awk 'END{print $0"|"NR}' a)
$ last_line=$(echo $out | awk -F"|" '{print $1}' )
$ tot_cnt=$(echo $out | awk -F"|" '{print $2}' )
$ echo $last_line
ccc
$ echo $tot_cnt
3
  • yes thats exactly what I want..and i want both in variable – chhaya vishwakarma Jul 23 '15 at 10:07
  • how can I save it in variable? – chhaya vishwakarma Jul 23 '15 at 10:11
  • I'm not that good in shell..I understand NR=number of records but how awk is getting last record can you please explain? – chhaya vishwakarma Jul 23 '15 at 10:36
  • END label instructions are executed after the last line of the file is processed. Hence, inside END when you try to print $0, it ends up printing the last line. – Guru Jul 23 '15 at 10:43
1

Let the cat sleep, you don't need it.

Use :

tail -n 1 filename.txt
wc -l filename.txt
  • 2
    Yes, but this still reads the input file twice which is what the OP is trying to avoid. – terdon Jul 23 '15 at 10:11
  • I can't avoid cat.. – chhaya vishwakarma Jul 23 '15 at 10:12
  • 1
    @heemayl: you are perfectly right, and it is not just syntactic sugar. Measuring the performance shows that getting rid of cat is much faster: tail knows how big the file is and can be optimized to check directly at the file end before returning the last line, but it does not know how big stdin is going to be, so it has to wait until cat is done before returning its answer. A similar thing applies to wc, which performs better against a file than against stdin. – Emeric Jul 23 '15 at 10:13
1
{   
    {   tee /dev/fd/3 | wc -l >&4
    }   3>&1 | tail -n1
}   <filename.txt 4>&1

It doesn't avoid reading it twice - but on multicore systems it will likely be concurrent. The output order is in no way guaranteed, however.

Probably the fastest way to simulate what you're asking for, though, is:

dd bs=64k <filename.txt | tail -n1

dd will report its read/writes to stderr and still copy all of the file to stdout.

Not the answer you're looking for? Browse other questions tagged or ask your own question.