Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
ORDER EVENT .........[] [] ... so many other tags... [Account<25106>=ACCT1] [Destination...] .. so many other tags.

I am currently trying to get the account like this. I tried using match in awk, but it seems slower. Can you suggest anything else other than the one below which is even faster?

j = index($0, "<25106>=");
account=substr($0, j + accountTagLength);
account=substr(account,1,index(account, "]") - 1);

Account is not 2nd field and the field position my vary..

Timings:

bash-3.2$ time head -1000000 temp.log | awk -F'<25106>=' '{print $2}' | sed -e 's/].*//' > /dev/null

real    0m2.410s
user    0m2.782s
sys     0m0.319s
bash-3.2$ time head -1000000 temp.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.690s
user    0m1.737s
sys     0m0.448s
bash-3.2$ time head -1000000 temp.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.588s
user    0m1.733s
sys     0m0.179s
bash-3.2$ time head -1000000 temp.log | awk -F'<25106>=' '{print $2}' | sed -e 's/].*//' > /dev/null                               
real    0m2.384s
user    0m2.762s
sys     0m0.272s
bash-3.2$ time head -1000000 temp.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.703s
user    0m1.709s
sys     0m0.484s

bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

real    0m3.449s
user    0m3.661s
sys     0m0.290s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

real    0m3.410s
user    0m3.551s
sys     0m0.236s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

real    0m3.361s
user    0m3.487s
sys     0m0.286s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.626s
user    0m1.831s
sys     0m0.263s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'

real    0m2.721s
user    0m2.808s
sys     0m0.265s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'

real    0m2.787s
user    0m2.863s
sys     0m0.516s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'

real    0m2.724s
user    0m2.882s
sys     0m0.278s
bash-3.2$ time head -1000000 dumper/cam_verbose.20120220.000.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m1.576s
user    0m1.748s
sys     0m0.235s

bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | grep -oE '<25106>=([A-Za-z0-9]*)+' | cut -d= -f2 > /dev/null                                     
real    0m2.098s
user    0m2.131s
sys     0m0.033s
bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); print substr(account,1,index(account, "]") - 1);} }' > /dev/null

real    0m0.253s
user    0m0.275s
sys     0m0.040s
bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | grep -oE '<25106>=([A-Za-z0-9]*)+' | cut -d= -f2 > /dev/null                                     
real    0m2.070s
user    0m2.105s
sys     0m0.034s
bash-3.2$ time head -100000 ORDER_EVENTS_CHAS_20120224.log | grep -oE '<25106>=([A-Za-z0-9]*)+' > /dev/null

real    0m2.065s
user    0m2.090s
sys     0m0.037s
    bash-3.2$ time head -1000000 ORDER_EVENTS_CHAS_20120228.log | awk -F'<25106>=' '{ substr($2,0,index($2,"]")-1);}'

real    0m3.426s
user    0m3.637s
sys     0m0.412s
bash-3.2$ time head -1000000 ORDER_EVENTS_CHAS_20120228.log | awk -F'<25106>=' '{ substr($2,0,index($2,"]")-1);}'

real    0m3.463s
user    0m3.603s
sys     0m0.408s
bash-3.2$ time head -1000000 ORDER_EVENTS_CHAS_20120228.log | awk '{j = index($0, "25106>="); if (j > 0) { account=substr($0, j + 7); substr(account,1,index(account, "]") - 1);} }'

real    0m2.247s
user    0m2.307s
sys     0m0.649s
share|improve this question
    
just found out even though I am not looking for a grep solution, regular expressions with grep are so slow. –  user14039 Feb 28 '12 at 19:33
    
Of course the literal string comparison is going to be faster than any regex comparison. And what you propose above is the straightforward implementation of that; so I wouldn't expect there to be anythin faster. –  dubiousjim Apr 19 '12 at 19:12

3 Answers 3

up vote 1 down vote accepted

Try this:

awk -F'<25106>=' '{print substr($2,0,index($2,"]")-1);}'

Not using regex, just strict string operations.

share|improve this answer
    
even that is slower than the index and substring. I updated the stats, if you go to the end. –  user14039 Feb 29 '12 at 15:52

If you have GNU awk (gawk) you can use the match() function with capturing parentheses:

gawk 'match($0, /<25106>=([^]]+)/, ary) {account = ary[1]}'

Alternately, you can use a complex field separator:

awk -F '<25106>=' '{split($2, ary, /\]/); account = ary[1]}'
share|improve this answer
    
anything with regex like match is slow. I have tried. I have updated the timings. –  user14039 Feb 28 '12 at 16:01

If you will only print this number, you can try this:

echo "ORDER EVENT ......... [Account<25106>=ACCT1]" | awk -F'<25106>=' '{print $2}' | sed -e 's/].*//'

EDIT: sed -only solution:

echo "ORDER EVENT ......... [Account<25106>=ACCT1]" | sed -e 's/.*25106>=//' -e 's/].*//'

EDIT2:

awk '{if (split($0, a, "25106>=") > 1) {print substr(a[2], 0, index(a[2], "]")-1)} }'
share|improve this answer
    
sorry, Jan, I was vague in my explanation of question. It is not 2nd field and the field position my vary. I updated my question. –  user14039 Feb 28 '12 at 15:34
    
@srikanthradix Updated. –  Jan Marek Feb 28 '12 at 15:41
    
updated with timings using time command. still index + substring seems to be faster. –  user14039 Feb 28 '12 at 15:56
    
@srikanthradix will be solution with only sed more faster? –  Jan Marek Feb 28 '12 at 16:08
    
Actually, I am trying to find out whether there is anything faster only with awk. –  user14039 Feb 28 '12 at 16:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.