I have folder which contains pipe(|) separated data. I want to have a list of the 8th column in first line of every file in the folder along with the file name, would be good if all this data can be consolidated in a file

3 Answers 3


For passing files to awk we can use shell globstar. -F can be used to define field delimiter, so -F'|' is what we need. And as far as only getting first line, using nextfile is sufficient. Thus, we can do:

awk -F'|' '{print FILENAME,$8;nextfile}' ./*

from within the folder in which files are located.

It should be noted that nextfile might not be supported. For instance, as Stephane noted, Debian and its derivatives ship with older version of mawk, which doesn't have nextfile as option. In the case where you need to have more portable awk code, you can take advantage of FNR variable. For instance:

awk -F'|' 'FNR==1{print FILENAME,$8}'  ./*
  • 1
    Though nextfile originated as a gawk extension, it's now supported in many other awk implementations, including mawk (though not the antiquated version found on Debian and derivatives). With awk implementations that don't support it nextfile would still be a harmless statement (references the nextfile variable), so awk -F'|' 'FNR==8{print FILENAME, $8; nextfile}' ./* would be portable (and more efficient with awks that have nextfile). You need the ./* prefix btw or it won't work for filenames containing = characters. Commented Mar 3, 2017 at 7:37
  • @StéphaneChazelas wait, why FNR==8 ? OP wanted to only process first record of each file Commented Mar 3, 2017 at 7:57
  • Sorry, typo, early morning and all. awk -F'|' 'FNR==1{print FILENAME, $8; nextfile}', the point being to use nextfile and to check on FNR for those awks that don't support nextfile and for which nextfile would be a harmless statement. Commented Mar 3, 2017 at 8:24
  • 1
    For the record (I had to check as I can't trust my memory much these days), nextfile was introduced in gawk 3.0.0 in 1996, renamed from next file added in gawk 2.14 (1992). Commented Mar 3, 2017 at 8:30

Though I still prefer the awk approach (more legible, more shell-like, more efficient), that's one case where using a shell loop would work relatively well:

for file in *; do
  IFS='|' read -r x x x x x x x text x < "$file" || continue
  printf '%s\n' "$file: $text" || exit
find . ! -name . -prune -exec test -f {} \; -exec sh -c '
   printf "%s:%s\n" "$1" "$(head -q -n 1 "$1" | cut -d\| -f8)"
' {} {} \;

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .