Extracting trace information using gawk

Question

I am using gawk to extract trace information from a mrt file in order to use it further for analysis. I have successfully been able to extract trace information from a pcap file format but cannot figure it out for the mrt format. Let me explain what I am trying to extract first by showing you the exampleof pcap format.

My pcap input file is:

No.     Time        Source                Destination           Protocol Length User Datagram Protocol Info
  1 0.000000    2001:4958:10:2::2     2001:4958:10:2::3     BGP      143                           UPDATE Message
Frame 1: 143 bytes on wire (1144 bits), 143 bytes captured (1144 bits)
Ethernet II, Src: JuniperN_36:98:52 (5c:5e:ab:36:98:52), Dst: JuniperN_3e:bf:49 (78:19:f7:3e:bf:49)
Internet Protocol Version 6, Src: 2001:4958:10:2::2 (2001:4958:10:2::2), Dst: 2001:4958:10:2::3 (2001:4958:10:2::3)
Transmission Control Protocol, Src Port: bgp (179), Dst Port: 56797 (56797), Seq: 1, Ack: 1, Len: 37
Border Gateway Protocol
No.     Time        Source                Destination           Protocol Length User Datagram Protocol Info
  2 0.326625    2001:4958:10:2::2     2001:4958:10:2::3     BGP      184                           UPDATE Message
Frame 2: 184 bytes on wire (1472 bits), 184 bytes captured (1472 bits)
Ethernet II, Src: JuniperN_36:98:52 (5c:5e:ab:36:98:52), Dst: JuniperN_3e:bf:49 (78:19:f7:3e:bf:49)
Internet Protocol Version 6, Src: 2001:4958:10:2::2 (2001:4958:10:2::2), Dst: 2001:4958:10:2::3 (2001:4958:10:2::3)
Transmission Control Protocol, Src Port: bgp (179), Dst Port: 56797 (56797), Seq: 38, Ack: 1, Len: 78
Border Gateway Protocol
No.     Time        Source                Destination           Protocol Length User Datagram Protocol Info
  3 1.178114    2001:4958:10:2::2     2001:4958:10:2::3     TCP      106                           bgp > 56797 [ACK] Seq=116 Ack=20 Win=16384 Len=0 TSval=3269200636 TSecr=371929488
Frame 3: 106 bytes on wire (848 bits), 106 bytes captured (848 bits)
Ethernet II, Src: JuniperN_36:98:52 (5c:5e:ab:36:98:52), Dst: JuniperN_3e:bf:49 (78:19:f7:3e:bf:49)
Internet Protocol Version 6, Src: 2001:4958:10:2::2 (2001:4958:10:2::2), Dst: 2001:4958:10:2::3 (2001:4958:10:2::3)
Transmission Control Protocol, Src Port: bgp (179), Dst Port: 56797 (56797), Seq: 116, Ack: 20, Len: 0
No.     Time        Source                Destination           Protocol Length User Datagram Protocol Info
  4 2.410144    64.251.87.209         64.251.87.210         BGP      228                           UPDATE Message, UPDATE Message
Frame 4: 228 bytes on wire (1824 bits), 228 bytes captured (1824 bits)
Ethernet II, Src: Cisco_e7:a1:c0 (00:1b:0d:e7:a1:c0), Dst: JuniperN_3e:ba:bd (78:19:f7:3e:ba:bd)
Internet Protocol Version 4, Src: 64.251.87.209 (64.251.87.209), Dst: 64.251.87.210 (64.251.87.210)
Transmission Control Protocol, Src Port: bgp (179), Dst Port: 65502 (65502), Seq: 1, Ack: 1, Len: 154
Border Gateway Protocol
Border Gateway Protocol
No.     Time        Source                Destination           Protocol Length User Datagram Protocol Info
  5 3.467853    206.47.102.206        206.47.102.201        BGP      105                           KEEPALIVE Message
Frame 5: 105 bytes on wire (840 bits), 105 bytes captured (840 bits)
Ethernet II, Src: JuniperN_36:98:52 (5c:5e:ab:36:98:52), Dst: JuniperN_3e:bf:49 (78:19:f7:3e:bf:49)
Internet Protocol Version 4, Src: 206.47.102.206 (206.47.102.206), Dst: 206.47.102.201 (206.47.102.201)
Transmission Control Protocol, Src Port: bgp (179), Dst Port: 55700 (55700), Seq: 1, Ack: 1, Len: 19
Border Gateway Protocol

I want to extract the following fields from the file:

Time
Source
Destination
Protocol
Length User Datagram

I created a simple script:

{

if($1 ~ /[0-9]/)
    {
        print $2,$3,$4,$5,$6
    }
}

And ran it on the input trace to receive (gawk -f script.txt input-pcap.txt >> pcap-out.txt) as shown below:

0.000000 2001:4958:10:2::2 2001:4958:10:2::3 BGP 143
0.326625 2001:4958:10:2::2 2001:4958:10:2::3 BGP 184
1.178114 2001:4958:10:2::2 2001:4958:10:2::3 TCP 106
2.410144 64.251.87.209 64.251.87.210 BGP 228
3.467853 206.47.102.206 206.47.102.201 BGP 105

Now, I want to do the same for mrt input format. The input file looks like:

TIME: 11/01/07 00:11:09
TYPE: TABLE_DUMP/INET
VIEW: 0
SEQUENCE: 0
PREFIX: 0.0.0.0/0
FROM:96.4.0.55 AS11686
ORIGINATED: 10/24/07 06:26:23
ORIGIN: IGP
ASPATH: 11686 3561
NEXT_HOP: 96.4.0.55
STATUS: 0x1

TIME: 11/01/07 00:11:09TYPE: TABLE_DUMP/INETVIEW: 0SEQUENCE: 1PREFIX: 0.0.0.0/0
FROM:213.140.32.148 AS12956
ORIGINATED: 10/24/07 06:26:16
ORIGIN: IGP
ASPATH: 12956
NEXT_HOP: 213.140.32.148
STATUS:0x1

TIME: 11/01/07 00:11:09
TYPE: TABLE_DUMP/INET
VIEW: 0
SEQUENCE: 2
PREFIX: 3.0.0.0/8
FROM:207.45.223.244 AS6453
ORIGINATED: 10/31/07 07:37:39
ORIGIN: IGP
ASPATH: 6453 701 703 80
NEXT_HOP: 207.45.223.244
STATUS: 0x1

TIME: 11/01/07 00:11:09
TYPE: TABLE_DUMP/INET
VIEW: 0
SEQUENCE: 3
PREFIX: 3.0.0.0/8
FROM:195.219.96.239 AS6453
ORIGINATED: 10/31/07 07:49:07
ORIGIN: IGP
ASPATH: 6453 701 703 80
NEXT_HOP: 195.219.96.239
STATUS: 0x1

TIME: 11/01/07 00:11:09
TYPE: TABLE_DUMP/INET
VIEW: 0
SEQUENCE: 4
PREFIX: 3.0.0.0/8
FROM:129.250.0.11 AS2914
ORIGINATED: 10/31/07 06:09:07
ORIGIN: IGP
ASPATH: 2914 701 703 80
NEXT_HOP: 129.250.0.11
MULTI_EXIT_DISC: 6
COMMUNITY: 2914:420 2914:2000 2914:3000 65504:701
STATUS: 0x1

I wish to extract the following information in the same style as done in pcap format so that it can be read by the analysis software:

Time (difference from first packet time field in seconds)
From (6th field, only the IP address is needed)
Next_hop (10th field)
Protocol/Origin (8th field)

So that the output looks like:

0.0000 96.4.0.55 96.4.0.55 IGP
0.0000 213.140.32.148 213.140.32.148 IGP
0.0000 207.45.223.244 207.45.223.244 IGP
0.0000 195.219.96.239 195.219.96.239 IGP

Would you think, may be grep is enough? : grep -o 'TIME:[^T]*\|FROM:[^A]*\|NEXT_HOP:[^SM]*\|ORIGIN:[^A]*' — Costas, Nov 29, 2014 at 23:06
Please edit your question and fix format of your input file. For example as it is written right now FROM is not on 6th field, but on 1st field in next line. This is huge difference for awk and other tools like sed or grep as well. Alternatively correct your question if your input file looks exactly like that. — jimmij, Nov 29, 2014 at 23:13
The software that would be used to analyze the trace has been trained to use gawk. I could try grep but when I tried your response, I get the following output which would not work: TIME: 11/01/07 00:11:09 ORIGIN: IGP NEXT_HOP: 96.4.0.55 TIME: 11/01/07 00:11:09 FROM:213.140.32.148 ORIGIN: IGP NEXT_HOP: 213.140.32.148 TIME: 11/01/07 00:11:09 FROM:207.45.223.244 ORIGIN: IGP NEXT_HOP: 207.45.223.244 TIME: 11/01/07 00:11:09 FROM:195.219.96.239 ORIGIN: IGP NEXT_HOP: 195.219.96.239 TIME: 11/01/07 00:11:09 FROM:129.250.0.11 ORIGIN: IGP NEXT_HOP: 129.250.0.11 — smartx, Nov 30, 2014 at 2:13

Peter.O · Accepted Answer · 2015-06-29 15:51:26Z

A couple of things to consider re the following gawk script:

Seeing that your input date does not show the century, you need to consider that format in realation to what date defaults to.
The provided sample output shows a decimal time, but your description says seconds. The data is in seconds resolution, so this script outputs integer seconds

gawk 'function d2s( date ){ c="date -d\""date"\" +%s"; c|getline b; close(c); return gensub("\n","","",b) }
      BEGIN{ RS="\n *\n" }
      { 
        nf=split($0,f,"\n"); delete o
        for(fi=1;fi<=nf;fi++){
            split(f[fi],s," ")
            if( s[1]=="TIME:"     ) o[1]=s[2]" "s[3]
            if( s[1]=="FROM:"     ) o[2]=s[2]      
            if( s[1]=="NEXT_HOP:" ) o[3]=s[2]      
            if( s[1]=="ORIGIN:"   ) o[4]=s[2]      
        }
        if( o[1] ){
            if( fps ) tds=d2s(o[1])-fps
            else { tds=0; fps=d2s(o[1]) }
            printf( "%s %s %s %s\n", tds, o[2], o[3], o[4] ) 
        } 
      }' "$f"

output - with input times altered from those posted (they were all the same):

0 96.4.0.55 96.4.0.55 IGP
1 213.140.32.148 213.140.32.148 IGP
3 207.45.223.244 207.45.223.244 IGP
7 195.219.96.239 195.219.96.239 IGP
15 129.250.0.11 129.250.0.11 IGP

Stack Exchange Network

current community

your communities

more stack exchange communities

Extracting trace information using gawk

1 Answer 1

Your Answer

Not the answer you're looking for? Browse other questions tagged linux shell awk gawk or ask your own question.

Hot Network Questions

Extracting trace information using gawk

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Not the answer you're looking for? Browse other questions tagged linux shell awk gawk or ask your own question.

Related

Hot Network Questions