1

I have numerous 2GB space-delimited files from a source system. Each row in each file contains the same number of strings surrounded by " as text qualifiers.

I need to eliminate the last two strings and their text qualifiers from every row in each file. (like removing the last two columns from a columnar report). With smaller files, I can import into Excel, delimit, delete the columns, save as tab delimited (much more useful than spaces).

Anycase, these files are too large and have too many rows for excel. So sed??

"text1" "text2" "text3" "text4" "text5" "text6"

Every row has the same number of strings. How do I drop "text5" "text6" from every row?

  • awk '{$5=$6=""}1' file... – jasonwryan May 18 '17 at 1:41
  • 1
    @jasonwryan: Or just awk 'NF=4' – Thor May 18 '17 at 5:04
  • @Thor better... – jasonwryan May 18 '17 at 5:07
3

If you know that you always want to delete the last two columns, this idiom can be used:

awk 'NF-=2' file

I noticed that this does not work with nawk, not sure why. The portable way is to force the field splitting with `$1=$1:

awk '{NF-=2} $1=$1' file

Output:

"text1" "text2" "text3" "text4"
2

This sed command will remove the last two space separated, quoted strings from the end of each line from file infile and send the results to outfile:

sed 's/ *"[^"]*" *"[^"]*" *$//' < infile > outfile
1
awk '{$(NF-1)=$NF=""}1'  inp

perl -pale '$_ = "@F[0..@F-3]"' inp

sed -ne '
   s/" "/"\
"/g
   :a
   s/\n/ /
   /\n.*\n.*\n/ba
   P
' inp

Explanation:

  • awk code nulls out the last and second-last fields and prints.
  • In perl fields are stored in @F array and the slice from 0th to third-last are selected and stored in the current line $_. The double quotes are there to effect the array->string xformation and joined together by the $" superglobal whose default value is a space. -p Perl option then carries the $_ value to the stdout.
  • In sed we first turn all the patterns " " ---> "\n" then we enter a loop where we take back the \n till there are two left. At which point of time we use the P uppercase p, command to print the first portion of the pattern space.
0

Printing every field till last-2..AWK provided us number of fields in a row using variable NF

echo "text1" "text2" "text3" "text4" "text5" "text6" | awk  -v ORS=""  '{
for(i=1;i<=NF-2;i++)print $i, " " ; print "\n"}'
  • (1) This will put two spaces after each word, including the last one (i.e., at the end of the line).  (2) If you would use printf instead of print, you wouldn’t need to change ORS. (3) Why bother submitting this when much more elegant awk solutions have already been posted? – G-Man Mar 9 at 6:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.