0

I have a 33 GB file. In this file the first column is the country code (eg: AT, BE, CA, DE, DK, GB, IE, IT, etc). I'm using awk command like the below:

awk -F"|" '$1~/^AT/ {print $0}'

This command works fine. But it is taking nearly 1.5 hours as awk reads the entire file.

Is there a way to break the awk command when if it has finished searching for a pattern.

Example: When it finished searching for AT it should break searching the rest of the file.

The file I am using is sorted.

|improve this question
  • You mean you want to exit after the 1st AT match or after the last AT match ? – don_crissti Apr 18 '17 at 16:57
  • yeah.. please see the below example: file: AT|123|1234 AT|12341|231 AT|1231|fwe AT|svdg|dgs BE|asdfa|afsfa BE|ava|gjrfj CE|gws|shad awk -F"|" '$1~/^AT/ {print $0}' file This awk command will search the entire file. But what i want is to break the awk command when it has reached "BE" – royal23enfield Apr 18 '17 at 16:59
  • gnu.org/s/gawk/manual/html_node/Exit-Statement.html – don_crissti Apr 18 '17 at 17:19
1

If you know the next code in the file,

awk '/^BE/ { exit }; /^AT/' file

Notice also how the -F option isn't really useful in this isolated scenario, and how the default action { print $0 } never needs to be explicitly given if that's the specific action you want.

If you need to do this repeatedly, a single script which extracts the sections you want to separate files in a single pass through the original file would obviously be more efficient. If you want to extract all of them (or a dominant part), see also csplit (maybe delete the files you don't need after having it extract them all?)

|improve this answer
1

Since the file is sorted, you could use string comparison to to exit when the first field sorts after the one you are looking for.

awk -F"|" '$1 ~ /^AT/ {print $0} $1 > "AT" {exit}'

Of course that will not make it any faster to find the first occurrence of a code late in the alphabet, so it might be a good idea to consider something more sophisticated with proper indexing.

|improve this answer
0

Sure, just add exit:

awk -F"|" '$1~/^AT/ {print $0; exit}'

That will cause it to exit as soon as it finds the first line whose 1st field starts with AT. If you want it to stop as soon as it finds the first line whose 1st field doesn't start with AT, you can use:

awk -F"|" '{if($1~/^AT/){print $0}else{exit}}'

And if you can have lines that don't start with AT before the first one that does and you want to stop after the lines starting with AT have been printed, use:

awk -F"|" '{if($1~/^AT/){print $0; a=1}else if(a==1){exit}}'
|improve this answer
  • i tried with this. it worked but when i give multiple conditions it is not working. multiple conditions like $1~/^AT|^AU/&&$55=="TRUE" – royal23enfield Apr 19 '17 at 10:57
  • @royal23enfield well, that's why you need to explain what you are trying to do in your question. We can't guess. I tried guessing here, I gave three different guesses and it turns out you needed something else. So, if you want a useful answer, edit your question, show an example of your input file and the output you would want to get from that example. That said, your syntax is completely wrong for awk. – terdon♦ Apr 19 '17 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.