1

I have a tab delimited file with 2500 columns and 3000 rows. I would like to extract a specific column based on the header. By the way, this is being done in a while loop so each time a different column name will be selected.

Example file format:

a     b    c    d    e     f     g    h   
xy    1    2    f    21    4     5    6
qu    9    10   z    50    12    7    8

Desired output:

a     b    c    d    g 
xy    1    2    f    5
qu    9    10   z    7

I was thinking to identify the column number, and then store the number as a variable and pass it under -f in cut. Example if we are selecting column "g" which here is the 7th field.

colNum=$(head -1 file.txt | tr '\t' '\n' | cat -n | grep "g" | cut -f 1)
cut -f1,2,3,4,"$colNum" file.txt > new_file.txt

I get the following error:

cut: fields and positions are numbered from 1
Try 'cut --help' for more information.
4
  • I'm not sure whether to suggest it as answer since you restricted the question to cut, but GNU Datamash is a command line tool that does this very easily.
    – cryptarch
    Jan 13, 2023 at 23:00
  • 1
    I'm open to other solutions, this was just the one I thought of, the purpose is just to get the desired output.
    – Sarah
    Jan 13, 2023 at 23:02
  • The error seems to be because cat -n right-justifies the line numbers, so colNum contains whitespace before the 7. However I'd suggest using a tool like awk, datamash, or miller instead. Jan 14, 2023 at 1:37
  • Yes, thanks for this comment, you are correct the white space was also causing an error
    – Sarah
    Jan 17, 2023 at 15:34

2 Answers 2

2

In a comment, it was agreed that an answer doesn't need to involve cut. I would like to suggest that GNU Datamash (https://www.gnu.org/software/datamash/) solves this problem elegantly.

I put the sample input from the question into a file called data.txt. Then an example run processing it with datamash:

$ datamash -HW cut a,b,c,d,g < data.txt
cut(a)  cut(b)  cut(c)  cut(d)  cut(g)
xy      1       2       f       5
qu      9       10      z       7

If it's undesirable to add the name of the operation to the headers, a little post-processing can tidy it up a bit:

$ datamash -HW cut a,b,c,d,g < data.txt | sed 's/cut(\([^[:space:]]*\))/\1/g'
a       b       c       d       g
xy      1       2       f       5
qu      9       10      z       7
1
  • 2
    or similarly with miller, mlr --tsv cut -f a,b,c,d,g file.txt Jan 14, 2023 at 2:42
0

It is possible to extract a specific column from a tab delimited file using the 'cut' command. To do this, you need to identify the column number of the required column and then use the '-f' option with the cut command to specify the column number. For example, if you want to extract the 'g' column from your file, you can use the following command:

colNum=$(head -1 file.txt | tr '\t' '\n' | cat -n | grep "g" | cut -f 1)
cut -f1,2,3,4,"$colNum" file.txt > new_file.txt

The first command identifies the column number of the desired column:

colNum=$(head -1 file.txt | tr '\t' '\n' | cat -n | grep "g" | cut -f 1)

The second command uses the 'cut' command to extract the specified column from the file:

cut -f1,2,3,4,"$colNum" file.txt > new_file.txt.

This should extract the desired column from the file and save it in the new_file.txt. For more information on the 'cut' command, please consult the official documentation here: https://linux.die.net/man/1/cut.

As far as I can see error message you're seeing is indicating that the cut command expects its field argument to be a number, but you're passing in a variable. To fix this, you can remove the quotes around "$colNum" in the cut command, so that the variable is interpreted as a number rather than a string.

Here's the corrected command:

cut -f1,2,3,4,$colNum file.txt > new_file.txt

Alternatively, you can use awk command to extract the specific column based on the header.

awk -v col="g" -F'\t' 'NR==1{for(i=1;i<=NF;i++){if($i==col){c=i;break}}} {print $1,$2,$3,$4,$c}' file.txt > new_file.txt

This command will look for the column header "g" in the first row and saves the column number in variable "c" and prints the first four columns and the column with header "g" in the new_file.txt.

1
  • This works, thank you! I also had to remove white space from the stored variable to make it work though, so I just piped to xargs at the end.
    – Sarah
    Jan 17, 2023 at 15:33

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .