3

I have a big file with several columns on each line. I'm familiar with using cut -f -d to select specific columns by their number.

I checked the manual for cut and it doesn't seem that there's a way to regex match columns.

What I want to do specifically is:

  • select the 2nd column of every line
  • and also select all columns that contain the string "hello" (there may be none, if not it could be any column(s) and not the same column(s) for each line)

What's the most convenient terminal tools for this operation?

EDIT:

Simplified example

x ID23 a b c hello1
x ID47 hello2 a b c
x ID49 hello3 a b hello4
x ID53 a b c d

The result I would want is:

ID23 hello1
ID47 hello2
ID49 hello3 hello4

or alternatively:

ID23 hello1
ID47 hello2
ID49 hello3 hello4
ID53

To elaborate the example given:

  • Columns are defined by one space
  • whether or not "only print if the string is present" is not really important, I can just grep for "hello" if necessary
  • we can assume the string "hello" will never be in column 1 or 2.
6

If one space at the end of the line doesn't hurt you much:

$ awk '{for(i=1;i<=NF;i++) if(i==2 || $i~"hello") printf $i" ";print ""}' file

ID23 hello1 
ID47 hello2 
ID49 hello3 hello4 
ID53 

This doesn't assume anything about the position of the "hello" string.

| improve this answer | |
  • You can post process to remove the additional space character : awk '{ ...}' | sed 's, $,,' – Emmanuel Jun 30 '16 at 15:46
4

I don't think this will be possible using simple tools like cut. Or, at least, not easily. Here's a Perl solution:

$ perl -lane '$k=join " ",grep{/hello/}@F; print "$F[1] $k" if $k' file 
ID23 hello1
ID47 hello2
ID49 hello3 hello4

Which you could simplify by using grep first:

$ grep hello file | perl -lane 'print "$F[1] ", join(" ", grep{/hello/}@F)'
ID23 hello1
ID47 hello2
ID49 hello3 hello4

Explanation

The -n option tells perl to read its input line by line and apply the script given by -e. The -l flag adds a newline (\n) to each print call. The -a option makes perl split its input lines on whitespace into the array @F.

So, the script itself is looking for all elements of @F (all columns) matching hello and saving them as the space-separated string $k ($k=join " ",grep{/hello/}@F;). Then, the 2nd field ($F[1]) and $k are printed if $k is defined, if at least one hello was found.

The second version is the same except that we don't need $k and print directly since we know that at least one hello will always be present.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.