Stadard unix command line tools (grep, cut, sed etc.) all work one line at a time. And nearly always, this is great.
However I am trying to parse some postgresql slow query logs. Each entry has some stuff at the start (datetime, duration), and then the SQL query. The SQL query might have newlines, so each "entry" in the log file can take more than 1 line (since the newlines in the query are put into the log file directly, and not escaped). I'd like to somehow "merge" these lines together, so that 1 line = 1 log file entry. Now, sometimes one entry is wholly on one line, and sometimes one entry is spread across up to 10 lines.
Is there a unix tool that can somehow "lineify" this file? I'd like to give it a (PCRE) regex and it will split the line/stdin based on that. All actual newlines between occurrences of this regex should be replaced with
"\n" or something I can specify.
There is probably one liners to do this with perl, but I'd like to see if anyone has already made this programme before I make it myself.
UPDATE: I could provide sample data, but I'd like to know the generic problem. SQL Servers can potentially make multi-line log files. I want a generic solution to turn any file into a newline separated unix-y style file.