6

I know this is a simple question and in many ways has been answered before, however every use I've seen of awk, sed, grep etc have all been selecting and printing the columns you want. In a case where there is an unknown number of columns that you want, and a specific column you don't want, how do you remove the column you don't want?

Do you simply select the columns before and then select the n columns after the column you don't want? Is there not just a simple delete column n function?

I don't have an example as it's more a question of curiosity, but I can draw one up if necessary.

  • 3
    In recent versions of GNU cut at least, there is a --complement option – steeldriver May 27 '16 at 16:19
  • The line size vary on the file? – Luciano Andress Martini May 27 '16 at 18:00
  • line sizes are the same – Giles May 27 '16 at 19:15
  • 1
    cut is very useful but has the extremely annoying limitation that it can only handle single-character field delimiters (this is why I usually reach for awk or perl without even considering cut). cut really needs the ability to handle regexp delimiters. The --complement option is a very nice new feature. – cas May 28 '16 at 4:01
2

While awk doesn't have a function to actually delete a field from a line, you can set particular column(s) to the empty string.

However, there is a common trick which can be used to delete columns (e.g. columns 5 and 7):

awk '{$5=$7="" ; $0=$0;$1=$1}1'

The $0=$0;$1=1 sets $0 to itself and $1 to itself. This has the side-effect of removing any extra field delimiters (i.e. the delimiters that would otherwise exist immediately after the now-empty fields $5 and $7)

NOTE: this will also convert all the field separators in $0 to whatever the Output Field Separator (OFS) is currently set to (e.g. with default FS and OFS, it will convert multiple tabs and spaces between fields to a single space)

With perl, it's much easier. It does have a function (splice()) to delete columns from an array:

perl -lane 'BEGIN{$,=" "}; splice(@F,4,1); splice(@F,5,1); print @F'

$,=" " sets the output field separator to a single space.

Notes:

  1. perl arrays start from 0, not 1. so @F[4] is the fifth column.

  2. this deletes column 5, so the second column (7th) we want to delete is now the 6th column, which is why the second splice deletes @F[5].

To avoid any potential confusion here, delete the columns in reverse order:

perl -lane 'BEGIN{$,=" "}; splice(@F,6,1); splice(@F,4,1); print @F'

or you can use a loop:

perl -lane 'BEGIN{$,=" "}; foreach $c (7,5) {splice(@F,$c-1,1)}; print @F'

BTW, if you only want to delete the first or the last column, you can use shift @F, or pop @F.

Output:

with the following input:

1 2 3 4 5 6 7 8 9 10
10  9   8   7   6   5   4   3   2   1
a b c d e f h i j k

All of the above scripts will produce the following output:

1 2 3 4 6 8 9 10
10 9 8 7 5 3 2 1
a b c d f i j k
3

Okay, so you can use the cut command, however it still isn't the deletion of said column, its selection of the columns you want, which does have an infinite column option. So one solution if you wanted to select the 2nd column and then every column from 4 onwards is:

cut -f 2,4- file.txt

However I am still interested to know if there's an actual deletion command?

  • 3
    With the GNU coreutils implementation, to delete (equivalently) the first and third columns you should be able to use cut -f 1,3 --complement file.txt – steeldriver May 27 '16 at 22:54
  • I tried this out just now, when i needed to delete just a single column and it didn't work, any ideas why? there were no errors, just no change to outputted text file. cut -f 2 --complement ~/file.txt > ~/file1.txt – Giles May 29 '16 at 0:03
2

It depends on what you mean by columns. If by columns you actually mean character columns and not delimited fields, the obscure colrm allows to specify the starting column, or the starting and ending columns for a range of columns to be deleted. There is no way through to delete an arbitrary number of non-contiguous columns in one invocation

  • I was thinking more for delimited fields, however being able to specify where columns start etc and be able to delete that range is pretty handy. Can you not specify the delimiter as the new column and then use this as a deletion tool? – Giles May 27 '16 at 20:23
1

I believe the cut command would work for you perhaps?

Found this in a previous answer: https://stackoverflow.com/questions/13690461/using-cut-command-to-remove-multiple-columns

A bit more on the cut command: http://www.computerhope.com/unix/ucut.htm https://en.wikipedia.org/wiki/Cut_(Unix)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.