I am looping through a list of files, extracting the final line, and printing out columns 8, 9, and 10. I need to also print to the output the 'event number', which is essentially the total number of records being processes (NR). How do I print the event/record number in the first column, outputting to the output file, such as what I have below?

for i in `ls -d *mcp`; do
tail -1 "$i" | awk  '{ printf "%s %s %s\n", $8, $9, $10}' >> ${Pout}${output}
echo "Finished Looping through each file."

What I want as the output is:

1 45 60 5
2 30 67 3
3 40 12 4

where the '45 column represents $8, 60 represents $9, and 5 represents $10. the 1,2,3, etc. is what I need to output. I essentially need to print the line number.

  • When I use this for loop structure: for i in ls -d *mcp; do tail -1 "$i" | awk '{ printf "%d %s %s %s\n",NR, $8, $9, $10}' >> ${Pout}${output} done I get the following output: 1 -0.242 125.104 35.0 1 -6.308 151.717 28.1 1 13.764 144.429 130.0 1 -56.022 -27.779 109.3 1 -9.461 156.412 4.0 Instead of all ones in the first column I want 1,2,...n. Does that clear things up? – geeb.24 May 13 '15 at 19:30
  • I understand your suggestion. However, that line in my for loop was working for me, which is why I didn't change it. Yes, it may be longer than what you wrote, but I didn't change it because it did what I needed it to do. – geeb.24 May 13 '15 at 19:35
  • Using backtics and ls means you are using two unnecessary processes. See whether my proposals below work for you. – Janis May 13 '15 at 19:43
  • (1) Don’t parse the output of ls.  for i in `ls -d *mcp` isn’t just inefficient; it produces wrong results if filenames contain certain special characters.  (2) Don’t post multi-line commands or output in comments.  Clarifications to the question, to include things that you’ve tried, results that you’ve gotten, and results that you want, belong in the question — edit the question to put them there.  (3) When you do use command substitution, use $(…) instead of `…`.  (4) If you must display a ` in code in a comment, type \`. – G-Man Says 'Reinstate Monica' May 14 '15 at 9:37

Try this:

for i in ./*.mcp; do
    if [ -f "$i" ]; then
        tail -1 "$i"
done | awk '{ print NR, $8, $9, $10 }'
  • This solution works! Is it correct that the if statement checks if the file exists, then it performs the 'tail' command? Thank you. – geeb.24 May 13 '15 at 19:46
  • @user78872: Yes, exactly. On a side note: you can also "add line numbers" to a file with cat(1), like this: cat -n file. – lcd047 May 13 '15 at 19:49
  • Can you identify any situation in which for i in *mcp fails (i.e., any reason why one should use ./ in that context)? – G-Man Says 'Reinstate Monica' May 14 '15 at 9:44
  • @G-Man: Any situation? Say if one of the files in named -foo.mcp. – lcd047 May 14 '15 at 9:49
  • 1
    @G-Man: Well, back in the days when I worked on Apollo workstations (before Apollo was bought by HP), tail, like most other utilities, didn't accept --. But, other than old dogs vs. new tricks, there's an informative answer about (among other things) the difference between -- and ./*. – lcd047 May 14 '15 at 10:12

With GNU awk (version 4.x) try this:

awk 'ENDFILE { printf "%d %s %s %s\n", ++c, $8, $9, $10}' *mcp > "${Pout}${output}"

echo "Finished Looping through each file."

With other awks and shells like bash try:

for f in *mcp
    awk -v c="$((++c))" 'END { printf "%d %s %s %s\n", c, $8, $9, $10}' "$f"
done > "${Pout}${output}"

echo "Finished Looping through each file."

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.