how to remove last two "-delimited strings from each line in a large file

Question

I have numerous 2GB space-delimited files from a source system. Each row in each file contains the same number of strings surrounded by " as text qualifiers.

I need to eliminate the last two strings and their text qualifiers from every row in each file. (like removing the last two columns from a columnar report). With smaller files, I can import into Excel, delimit, delete the columns, save as tab delimited (much more useful than spaces).

Anycase, these files are too large and have too many rows for excel. So sed??

"text1" "text2" "text3" "text4" "text5" "text6"

Every row has the same number of strings. How do I drop "text5" "text6" from every row?

Answer 1

If you know that you always want to delete the last two columns, this idiom can be used:

awk 'NF-=2' file

I noticed that this does not work with nawk, not sure why. The portable way is to force the field splitting with `$1=$1:

awk '{NF-=2} $1=$1' file

Output:

"text1" "text2" "text3" "text4"

Answer 2

This sed command will remove the last two space separated, quoted strings from the end of each line from file infile and send the results to outfile:

sed 's/ *"[^"]*" *"[^"]*" *$//' < infile > outfile

Answer 3

awk '{$(NF-1)=$NF=""}1'  inp

perl -pale '$_ = "@F[0..@F-3]"' inp

sed -ne '
   s/" "/"\
"/g
   :a
   s/\n/ /
   /\n.*\n.*\n/ba
   P
' inp

Explanation:

• awk code nulls out the last and second-last fields and prints.
• In perl fields are stored in @F array and the slice from 0th to third-last are selected and stored in the current line $_. The double quotes are there to effect the array->string xformation and joined together by the $" superglobal whose default value is a space. -p Perl option then carries the $_ value to the stdout.
• In sed we first turn all the patterns " " ---> "\n" then we enter a loop where we take back the \n till there are two left. At which point of time we use the P uppercase p, command to print the first portion of the pattern space.

Answer 4

Printing every field till last-2..AWK provided us number of fields in a row using variable NF

echo "text1" "text2" "text3" "text4" "text5" "text6" | awk  -v ORS=""  '{
for(i=1;i<=NF-2;i++)print $i, " " ; print "\n"}'