I have a file that looks like this :

Marketing  Ranjit Singh   FULLEagles       Dean Johnson   
Marketing  Ken Whillans   FULLEagles       Karen Thompson 
Sales      Peter RobertsonPARTGolden TigersRich Gardener  
President  Sandeep Jain   CONTWimps        Ken Whillans   
Operations John Thompson  PARTHawks        Cher           
Operations Cher           CONTVegans       Karen Patel    
Sales      John Jacobs    FULLHawks        Davinder Singh 
Finance    Dean Johnson   FULLVegans       Sandeep Jain   
EngineeringKaren Thompson PARTVegans       John Thompson  
IT         Rich Gardener  FULLGolden TigersPeter Robertson
IT         Karen Patel    FULLWimps        Ranjit Singh   

I want to use a grep command to search for "John" in the 2nd column w.o searching the last column, but for every "John" in the 2nd column, I want the output of the last column.

The end result should look like this :

       John Thompson                   Cher           
       John Jacobs                     Davinder Singh 
       Dean Johnson                    Sandeep Jain   
  • 2
    It doesn't make sense to me to use grep for that instead of awk. And I doubt that grep alone is capable of that. May 23, 2014 at 23:57
  • i was told that i cannot use awk, sed, or perl. so any solution that excludes them would help.
    – BigDave
    May 24, 2014 at 0:04

4 Answers 4

#! /bin/bash
while read line; do
  if [[ ${line:11:15} =~ John ]]; then
    echo "      ${line:11:15}                   ${line:43}"
done <file

You can perform the selection in grep by matching the right number of characters from the beginning of the line:

grep -E '^.{11,22}John'

John has to start and end within the range of columns 11–26.

Replacing some columns by whitespace is not within grep's capabilities. With GNU grep, you can use -o to output only the matching part, but you won't be able to combine the two columns with additional whitespace in between.

  • If it has to end no later than column 25, it can only have 21 characters preceding it, not 26.
    – rici
    May 24, 2014 at 1:00
  • 1
    @rici Er, yes, thank you. It's 1–26, actually, so 1,22. May 24, 2014 at 1:06
tab=$(printf '\t')
cut -c12-26 file | paste - file | grep "^[^$tab]*John" | cut -c1-16,60-

Like Gilles said you cannot do it with grep alone. grep stands for g/re/p (a ed/ex/vi command), that is, print the lines that match the re.


Let's we combine some hints of the solutions just showed.

From Stephane+ Gilles:

grep -E '^.{11,22}John' source.txt  | cut -c12-26,44-
# or, if you want only "John " and not Johnson, add a space after John.
grep -E '^.{11,22}John ' source.txt  | cut -c12-26,44-

Here you need to use /bin/grep and also /bin/cut.

From Gilles + Hauke:

grep -E '^.{11,22}John' source.txt | while read line; do echo "${line:11:14} ${line:43}"; done

Here you need to use /bin/grep and also echo. In modern shell you find echo like a built-in command so it requires less.

The Hauke's solution is the less expensive in term of installed programs: it requires only echo (built-in so bash), but not even /bin/grep, as wished in the question.

Update Let's we play a little.
In the everyday life we run snippet of code, created on the fly and used maybe nevermore; not always it results convenient to spend our human time to optimize it or to try different variations... but semel in anno licet insanire: on the other hand and with an high probability this was a case-study question, maybe even an homework not completely clear. In this optic, and IMHO, all the different approaches and solutions are useful.

We all agree, it seems, that for a big number of lines it is better (faster) to use a compiled program then a sequence of shell commands. But what is big? I think that numbers (and plots) can fix ideas better than only words: so let's we see.

Here the recipe: A set of files with a growing number of lines, N, was prepared. Each line was randomly extracted from the 11 original posted here.
The values of N used are
11 22 55 110 220 550 1100 2200 5500 11000 110000 1100000 11000000.

The snippet tested are:

  • The Cut+Paste+Grep+Cut tab=$(printf '\t') ; cut -c12-26 file | paste - file | grep "^[^$tab]*John" | cut -c1-16,60- - Green
  • The Only Grep - Blue grep -E '^.{11,22}John' source.txt. Simply the grep with no formatted output as asked.
  • The Grep+Cut - Red [grep -E '^.{11,22}John' source.txt | cut -c12-26,44-].
  • The Grep+While - Violet [see above in this answer, the grep before and the while loop].
  • The While loop - Yellow a full bash solution that do not require grep

For each size of the files and snippet, a number of repetitions, NRep, was made starting from 400 (for the shorter files) and decreasing with N down to 100, 10, and 1 for the last one.
It was recorded the value of Time per line, Tpl, aka the real time as measured by the built-in function time and averaged on NRep and N. Since both Tpl and N span over powers of 10 it was plotted the common logarithm (to base 10, or the powers of 10). The lines reported are Bezier curves that touch each point.

Full trend of *Time per Line* vs *Number of lines*

For large values of N the Time per Line become almost constant. It is the so-called asymptotic behaviour. As expected, the lesser the compiled program used number, the faster the result.

Crossover Region

On the contrary, with small files, the result is the opposite. The efficiencies of different codes cross each other in a region that (for our example) is between 40 and 140 lines. Even if it is true that for small files is small even the overall human time used, the same consideration is not anymore valid when it is needed to work with a big number of little files: the pure bash code (yellow) that asymptotically is 8.12 times slower than the green one, and even 157 times slower that the red one (for the 11M lines file it uses 334.56 s. instead of 41.21s of the green and 2.16 of the red) it is instead respectively 2.16 and 1.58 times faster for the 11 lines one, when it uses 1.20s for 400 repetitions instead of 1.89s of the red or 2.59s of the green.

Conclusion: The more you know, the better you challenge, and check always when you can! :-)

Ps> Similar consideration can be done on the user time and sys time, but with slightly different crossing region.

bash 4.3.11(1)-release
paste (GNU coreutils) 8.21
cut (GNU coreutils) 8.21
grep (GNU grep) 2.16
kernel 3.13.0-24-generic x86_64

  • I would't bet on bash to win against grep plus reduced bash code... Probably less expensive with respect to RAM but probably not faster. May 24, 2014 at 19:11
  • @HaukeLaging You're right:I had to specify what I mean for expensive. :-) I guess this was an homework. So, in this case, less expensive would mean only with less requests to the system, alias less program installed... for what it concerns speed and memory usage we can check if we remain curios. :-)
    – Hastur
    May 24, 2014 at 19:33
  • Guys, a shell loop especially in bash is always going to be slower than using the right tool for the task (and is bad coding practice) except for very small input. I downvote anything that uses a while read loop solely for text processing for no good reason. May 25, 2014 at 19:52
  • @StephaneChazelas It depends... if you need to deal often with a big number of little files, for example. And what is big or little? More variant you (me, everybody) know, better is. Always IMHO. ;-)
    – Hastur
    May 26, 2014 at 23:08
  • @HaukeLaging did you know that the only bash was the faster snippet till 55-56 lines :-) I have no will at all to test memory usage too :)
    – Hastur
    May 26, 2014 at 23:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .