Drawing a state machine from logs

Question

I have logs in the following format:

03/22/2011 14:45:06;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 35707.arien.ics.muni.cz state from EXITING-RETURNSTD to EXITING-STAGEOUT (5-51)
03/22/2011 14:45:06;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 35709.arien.ics.muni.cz state from RUNNING-PRERUN to RUNNING-RUNNING (4-42)
03/22/2011 14:45:07;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 35708.arien.ics.muni.cz state from RUNNING-RUNNING to EXITING-EXITING (5-50)
03/22/2011 14:45:07;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 35708.arien.ics.muni.cz state from EXITING-EXITING to EXITING-RETURNSTD (5-70)
03/22/2011 14:45:07;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 35708.arien.ics.muni.cz state from EXITING-RETURNSTD to EXITING-STAGEOUT (5-51)
03/22/2011 14:45:08;0001;PBS_Server;Svr;PBS_Server;svr_setjobstate: setting job 35709.arien.ics.muni.cz state from RUNNING-RUNNING to EXITING-EXITING (5-50)

Now, I would like to graphically reconstruct the state machine, but I'm kind of hesitating on how to approach this problem. Cutting out the transitions shouldn't be a problem, but I'm not sure how to reconstruct a graphical representation from them.

Answer 1

I'm not positive I know what you mean, but are you looking for something like this?

I used Graphviz, which takes text input files describing transitions, and figures out the graph automatically. Here's the exact command:

$ sed 's/-/_/g' input | gawk '
    BEGIN {print "digraph g {"}
    END {print "}"}
    match($0, /from ([^ ]*) to ([^ ]*) \((.*)\)$/, groups) {
        print groups[1] " -> " groups[2] " [label = \"" groups[3] "\"];"
    }' > output.dot
$ dot -Tpng output.dot > output.png

Explanation

• sed 's/-/_/g' input -- Dot doesn't like hyphens in the node name, so I converted them to underscores
• gawk -- Standard awk doesn't have the match function that gawk has; you can do the string manipulation any way you like though (e.g. perl is another good choice)
  • BEGIN {print "digraph g {"} -- Dot specifications start with this line (the name of the graph, "g", doesn't really matter)
  • END {print "}"} -- Ends the digraph g started in the BEGIN block
  • match($0, /from ([^ ]*) to ([^ ]*) \((.*)\)$/, groups) -- A regular expression that matches your log file format; it stores the results in the groups variable
  • print groups[1] " -> " groups[2] " [label = \"" groups[3] "\"];" -- Outputs a dot-compatible line (for example, A -> B [label = "C"]; will show two nodes, A and B, with a transition between them labeled C)
• dot -Tpng output.dot > output.png -- Tell graphviz to convert the dot file to a PNG

Resulting dot file

digraph g {
EXITING_RETURNSTD -> EXITING_STAGEOUT [label = "5_51"];
RUNNING_PRERUN -> RUNNING_RUNNING [label = "4_42"];
RUNNING_RUNNING -> EXITING_EXITING [label = "5_50"];
EXITING_EXITING -> EXITING_RETURNSTD [label = "5_70"];
EXITING_RETURNSTD -> EXITING_STAGEOUT [label = "5_51"];
RUNNING_RUNNING -> EXITING_EXITING [label = "5_50"];
}

The PNG you get when running that file through dot is above

Answer 2

I'm working on a tool to do exactly what you want -- generate finite state machines from logs. The tool is called Synoptic, and you can find out more about it here: http://code.google.com/p/synoptic/