Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have the following file:

id  name  age
1   ed    50
2   joe   70   

I want to print just the id and age columns. Right now I just use awk:

cat file.tsv | awk '{ print $1, $3 }'

However, this requires knowing the column numbers. Is there a way to do it where I can use the name of the column (specified on the first row), instead of the column number?

share|improve this question
cat isn't necessary, BTW. You could use awk '{ print $1, $3 }' file.tsv – Eric Wilson Nov 22 '11 at 16:58
If not column number, then what would you like to depend on? – rozcietrzewiacz Nov 22 '11 at 17:22
@rozcietrzewiacz The name; he wants to say id instead of $1 and age instead of $3 – Michael Mrozek Nov 22 '11 at 17:25
see also discussion on stackoverflow – Hotschke Feb 23 '15 at 10:41
up vote 20 down vote accepted

Maybe something like this:

$ cat t.awk
NR==1 {
    for (i=1; i<=NF; i++) {
        ix[$i] = i
NR>1 {
    print $ix[c1], $ix[c2]
$ awk -f t.awk c1=id c2=name input 
1 ed
2 joe
$ awk -f t.awk c1=age c2=name input 
50 ed
70 joe

If you want to specify the columns to print on the command line, you could do something like this:

$ cat t.awk 
NR==1 {
    for (i=1; i<=NF; i++)
        ix[$i] = i
NR>1 {
    for (i in out)
        printf "%s%s", $ix[out[i]], OFS
    print ""
$ awk -f t.awk -v cols=name,age,id,name,id input 
ed 1 ed 50 1 
joe 2 joe 70 2 

(Note the -v switch to get the variable defined in the BEGIN block.)

share|improve this answer
I've been putting off learning awk...what's the best way to support a variable number of columns? awk -f t.awk col1 col2 ... coln input would be ideal; awk -f t.awk cols=col1,col2,...,coln input would work too – Brett Thomas Dec 1 '11 at 15:03
Updated my answer. Stop putting off learning it if you want to do stuff with it :) – Mat Dec 3 '11 at 11:07
The 2nd example does not output the columns in the expected order, for (i in out) has no inherent ordering. gawk offers PROCINFO["sorted_in"] as a solution, iterating over the index with a for( ; ; ) is probably better. – mr.spuratic Mar 4 at 10:32
@BrettThomas, highly recommend this tutorial. (If you have access to lynda.com, I even more highly recommend "Awk Essential Training," which covers all the same material but more concisely and with practice exercises.) – Wildcard May 14 at 6:05

Just trowing a Perl solution into the lot:

#!/usr/bin/perl -wnla

    @f = ('id', 'age');   # field names to print
    print "@f";           # print field names

if ($. == 1) {            # if line number 1
    @n = @F;              #   get all field names
} else {                  # or else
    @v{@n} = @F;          #   map field names to values
    print "@v{@f}";       #   print values based on names
share|improve this answer

For what it's worth. This can handle any number of columns in the source, and any number of columns to print, in whatever output sequence you choose; just re-arrange the args...

eg. call: script-name id age

  for ((i; i<${#outseq[@]}; i++)) ;do 
    head -n 1 file |
     sed -r 's/ +/\n/g' |
      sed -nr "/^${outseq[$i]}$/="
  done ))
tr ' ' '\t' <<<"${outseq[@]}"
sed -nr '1!{s/ +/\t/gp}' file |
  cut -f $(tr ' ' ','<<<"${colnum[@]}") 


id      age
1       50
2       70
share|improve this answer

If you just want to refer to those fields by their names instead of numbers, you can use read:

while read id name age
  echo "$id $age"
done < file.tsv 


I saw your meaning at last! Here's a bash function that will print out only the columns you specify on the command line (by name).

printColumns () 
read names
while read $names; do
    for col in $*
        eval "printf '%s ' \$$col"

Here's how you can use it with the file you've presented:

$ < file.tsv printColumns id name
1 ed 
2 joe 

(The function reads stdin. < file.tsv printColumns ... is equivalent of printColumns ... < file.tsv and cat file.tsv | printColumns ...)

$ < file.tsv printColumns name age
ed 50 
joe 70 

$ < file.tsv printColumns name age id name name name
ed 50 1 ed ed ed 
joe 70 2 joe joe joe

Note: Pay attention to the names of the columns you request! This version lacks sanity checks, so nasty things can happen if one of the arguments is something like "anything; rm /my/precious/file"

share|improve this answer
This also requires knowing the column numbers. Just because you name them id, name and age, does not change the fact that the order is hard-coded in your read line. – janmoesen Nov 22 '11 at 19:54
@janmoesen Yes, I finally got the point :) – rozcietrzewiacz Nov 23 '11 at 0:43
This is nice, thanks. I'm working with large files (1000 columns, millions of rows) so am using awk for speed. – Brett Thomas Dec 1 '11 at 15:06
@BrettThomas Oh I see. I'm very curious then: could you post some benchmark that gives the time comparison? (Use time { command(s); }). – rozcietrzewiacz Dec 1 '11 at 15:45
@rozceitrewaicz: time cat temp.txt | ./col1 CHR POS > /dev/null 99.144u 38.966s 2:19.27 99.1% 0+0k 0+0io 0pf+0w time awk -f col2 c1=CHR c2=POS temp.txt > /dev/null 0.294u 0.127s 0:00.50 82.0% 0+0k 0+0io 0pf+0w – Brett Thomas Dec 1 '11 at 19:03

Convert it to a csv file and use a csv tool such as csvcut from the csvkit:

$ cat test-cols.dat 
x  y1  y2  y3  ei1  ei2  ei3  es1  es2  es3
1  4   5   4   7    7    2    4    7    7
2  7   3   3   3    8    3    3    3    8
3  2   1   4   4    9    6    4    4    9

Install csvkit:

$ pip install csvkit

Use tr with its squeeze option -s to convert it into a valid csv file and apply csvcut:

$ cat test-cols.dat | tr -s ' ' ',' | csvcut -c y2,es3

If you want to return to the old data format, use tr ',' ' ' | column -t

$ cat test-cols.dat | tr -s ' ' ',' | csvcut -c y2,es3 | tr ',' ' ' | column -t
y2  es3
5   7
3   8
1   9

PS. Unnecessary use of cat. But I like it this way to construct the command.

share|improve this answer
+1. But unnecessary uses of tr, too. TSV files are supported directly, without any need to convert them to CSV. The -t (aka --tabs) option tells cvscut to use tabs as field delimiter. And -d or --delimiter to use any character as delimiter. – cas May 14 at 7:19
With some testing, it seems the -d and -t options are semi-broken. they work to specify the input delimiter, but the output delimiter is hardcoded to always be a comma. IMO that's broken - it should either be the same as the input delimiter or have another option to allow the user to set the output delimiter, like awk's FS and OFS vars. – cas May 14 at 7:25

Usually it is easier just to look at the file header, count the number of the column you need (c) and then use Unix cut:

cut -f c -d, file.csv

But when there are many columns or many files I use the following ugly trick:

cut \
  -f $(head -1 file.csv | sed 's/,/\'$'\n/g' | grep -n 'column name' | cut -f1 -d,) \
  -d, \ 

Tested on OSX, the file.csv is comma-delimted.

share|improve this answer

Here's one quick way for selecting a single column.

Say we want the column named "foo":

f=file.csv; colnum=`head -1 ${f} | sed 's/,/\n/g' | nl | grep 'foo$' | cut -f 1 `; cut -d, -f ${colnum} ${f}

Basically, take the header line, split it into multiple lines with one column name per line, number the lines, select the line with the desired name, and retrieve the associated line number; then use that line number as the column number to the cut command.

share|improve this answer

Looking for a similar solution (I need the column named id, which might have a varying column number), I came across this one:

head -n 1 file.csv | awk -F',' ' {
      for(i=1;i < NF;i++) {
         if($i ~ /id/) { print i }
} '
share|improve this answer

If the file you're reading could never possibly be user-generated, you could abuse the read builtin:

read $(head -n1 "$f") extra <<<`seq 100`
awk "{print \$$id, \$$age}" "$f"

The input file's entire first line is substituted into the argument list, so read is passed all the field names from the header line as variable names. The first of these gets assigned the 1 that seq 100 generates, the second gets the 2, the third gets the 3 and so on. Excess seq output is soaked up by the dummy variable extra. If you know the number of input columns ahead of time, you can change the 100 to match and get rid of extra.

The awk script is a double-quoted string, allowing the shell variables defined by read to be substituted into the script as awk field numbers.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.