Can anyone please expain me step by step below AWK script written.

I have the below code written in my script to format the flat file data. Just wanted to understand so that i can reuse -- I am not a unix guy but task has being assigned to me.kindly help!

awk -vsep=$SEPARATOR 'NR>2{if(NF){if(!s){gsub(" *"sep"[ \t]*",sep);printf "%d%s\n",NR-2,$0}}else s=1}' file_name > new_file

 # where $SEPARATOR = ';'

Thanks in Advance.

  • Somebody did a pooh-pooh on the keyboard, and left the clean-up to the next developer. That's what happened. And the next developer should give away the dog and get a python (or a cup of coffee, or a precious stone...). – l0b0 May 7 '14 at 7:15
  • 1
    It would have been good if you made some sort of an attempt to understand it, and then posted a question saying that this is what you could figure and need help to figure the rest of it. Such questions are rarely helpful to any future visitor to the site. – devnull May 7 '14 at 7:36
  • @l0b0 Really now; let's see the Python program which does the same thing. – Kaz Jun 14 '14 at 0:51

The command line option -vsep=$SEPERATOR sets an awk variable sep (which is used in the search/replace) to whatever you specify. ; in your case.

# NR = Number of current Record, or line number
# Skip the first line
if ( NR > 2 ) {

  # NF = Number of fields in the current record
  # If the line contains something other than a blank line or the 
  # awk field separator characters (whitespace by default)
  if ( NF ) {

    # If we have not seen a blank line (script flag s)
    if ( !s ) {

      # Search the current line repeatedly (gsub) for any number of spaces (" *") 
      # before a ";" then any number of spaces or tabs ([ \t]*) after the `;`
      # and replace it all with just a ";"
      gsub( " *"sep"[ \t]*", sep );

      # Print the line number, 0 based (NR-2) as a signed decimal integer (`%d`)
      # then the complete line ($0) followed by a new line character (\n)
      printf "%d%s\n", NR-2, $0;

  } else { 

    # Set the "seen a blank line" flag
    s = 1


file_name > new_file writes the output into a new file called new_file

By the way, if you structure the script like the following it's a lot easier to read and will be quicker if you have large amounts of data occurring after a blank line.

awk -vsep=$SEPERATOR '{

# Skip the first line
if (NR == 1) { next; }

# Stop processing if we see a blank line
if (NF == 0) { exit; }

# Remove spaces before and spaces/tabs after separator
gsub( " *"sep"[ \t]*", sep );

# Print the line with a record number starting from 0
printf "%d%s\n", NR-2, $0;

}' file_name > new_file
  • "NR = Number of the current Record"; "If the line contains something" – Hauke Laging May 7 '14 at 7:43
  • Thanks. Yeah I was explaining the opposite of the NF if which was a bit misleading/dumb. – Matt May 7 '14 at 7:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.