How to remove a column or multiple columns from file using shell command?

Question

Input:

ARCHIVE  B1_NAME  B2_NAME  B3_NAME  ELEMENT  INFO_NAM WERT PROCID
-------- -------- -------- -------- -------- -------- ---- ------
15MinAvg AIRSS    33-GIS   DMDMGIS1 I        MvAvr15m 1123  CP
15MinAvg AIRSS    33-GIS   DMDMGIS1 P        MvAvr15m 2344  CP
15MinAvg AIRSS    33-GIS   DMDMGIS1 Q        MvAvr15m 4545  CP
15MinAvg AIRSS    33-GIS   DMDMGIS2 I        MvAvr15m 6576  CP
15MinAvg AIRSS    33-GIS   DMDMGIS2 P        MvAvr15m 4355  CP
15MinAvg AIRSS    33-GIS   DMDMGIS2 Q        MvAvr15m 6664  CP

Output:

ARCHIVE  B1_NAME  B2_NAME  B3_NAME  ELEMENT WERT
-------- -------- -------- -------- ------- ----
15MinAvg AIRSS    33-GIS   DMDMGIS1 I       1123
15MinAvg AIRSS    33-GIS   DMDMGIS1 P       2344
15MinAvg AIRSS    33-GIS   DMDMGIS1 Q       4545
15MinAvg AIRSS    33-GIS   DMDMGIS2 I       6576
15MinAvg AIRSS    33-GIS   DMDMGIS2 P       4355
15MinAvg AIRSS    33-GIS   DMDMGIS2 Q       6664

I want to delete the two columns INFO_NAM and PROCID from my input file.

Answer 1

If you are content to clear the columns rather than completely removing them:

Don't print 6^th and 8^th column

awk '{$6=$8=""; print $0}' file

Answer 2

This has been answered before elsewhere on Stack Overflow:

• delete a column with awk or sed

• Deleting columns from a file with awk or from command line on linux

  etc..

I believe awk is the best for that.

awk '{print $1,$2,$3,$4,$5,$7}' file

It is possible to use cut as well.

cut -f1,2,3,4,5,7 file

Answer 3

The best way to delete columns is with cut:

cut --complement -d' ' -f6,8 file

Explanation:

• -d specifies the delimiter; in this case, a space
• --complement keeps the columns other than the ones specified in -f
• -f specifies the columns to cut (rather than the columns to keep, since --complement is being used); in this case, columns 6 and 8

From the cut man page:

  -d, --delimiter=DELIM
        use DELIM instead of TAB for field delimiter

  -f, --fields=LIST
         select only these fields;  also print any line that contains
         no delimiter character, unless the -s option is specified

  --complement
         complement the set of selected bytes, characters or fields

Some others have proposed using a construction like awk '{$2=$4=""; print $0}', but this doesn't work in cases where the delimiter is something besides space; it makes the contents of the field the empty string, but does not remove the field altogether. For example, echo "a|b|c|d|e" | awk -v FS='|' -v OFS='|' '{$2=$4=""; print}' yields a||c||e.

Answer 4

Use printf to preserve the format of each field. Each field is x characters long and the minus sign justifies the string left.

awk '{ printf("%-8s %-8s %-8s %-8s %-8s %-4s\n", $1, $2, $3, $4, $5, $7)}' file

Answer 5

If you're open to a Perl solution...

perl -ane 'printf("%-8s %-8s %-8s %-8s %-8s %-4s\n", $F[0], $F[1], $F[2], $F[3], $F[4], $F[6])' file

These command-line options are used:

• -n loop around every line of the input file, do not automatically print every line

• -a autosplit mode – split input lines into the @F array. Defaults to splitting on whitespace

• -e execute the following perl code