1

I have a text file with information separated in blocks. I would like to separate these blocks into records so that I can handle them separately.

This is my text file (file.txt):

Alarm Stats:
com.android.calendar
     38ms running, 0 wakeups
     1 alarms: act=com.android.calendar.APPWIDGET_SCHEDULED_UPDATE dat=content://com.android.calendar typ=vnd.
com.android.providers.calendar
     2ms running, 2 wakeups
     2 alarms: act=com.android.providers.calendar.intent.CalendarProvider2 flg=0x14
android
     35563ms running, 11 wakeups
     1 alarms: act=android.intent.action.DATE_CHANGED flg=0x20000014
     1210 alarms: act=android.intent.action.TIME_TICK flg=0x40000014
     120 alarms: act=com.android.server.ThrottleManager.action.POLL flg=0x14
     1 alarms: act=android.net.wifi.DHCP_RENEW flg=0x14
     10 alarms: act=android.content.syncmanager.SYNC_ALARM flg=0x14
com.apollo.apollonetworkcheck
     1026ms running, 88 wakeups
     88 alarms: flg=0x14
com.android.phone
     4ms running, 0 wakeups
     2 alarms: act=com.android.phone.UPDATE_CALLER_INFO_CACHE flg=0x14

Using "apollo" as my search string, the output should be:

com.apollo.apollonetworkcheck
     1026ms running, 88 wakeups
     88 alarms: flg=0x14

So far I have this for my command-line, but I don't know where to put the search string "apollo". Also, I don't know how to specify "non-blank" for the record separator.

$ awk 'BEGIN { RS = "^RTC" } { print $1 }' file.txt

Archemar, wurtel, steeldriver, terdon, your answers are great and adds lots of substances and resource resolution to the question. I up-voted each answer for the great value. The one with "gawk" of course was more precise to the question and objective to use the unix/linux command.

  • Are you sure the string "apollo" will only appear once, or only appear in those un-indented "headers"? I'm asking because that input file looks like the (potentially unpredictable) output of another program, which makes parsing code very vulnerable to code rot. – Anko Sep 12 '14 at 10:14
  • @Anko I'm not sure how many times the search string (in this case "apollo") would appear in the record, or where it will appear. I just want to match records that has the search string anywhere in it. – L. D. James Sep 12 '14 at 12:50
2

Your idea to use a regex record separator is elegant, but remember that awk will consume the corresponding text, which in your case will be the first non-whitespace character of the following record.

If your system has the GNU version of awk, you can access the most recently matched RS via the RT variable, however you would still need to save the result in order to stitch it back onto the start of the following record - perhaps something like

gawk 'BEGIN{RS="\n[^[:blank:]]"}; {lastRT=RT}; /apollo/ {$0=substr(lastRT,2)""$0; print}' file.txt

The substr(lastRT,2) just removes the newline from the match so that only the non-whitespace character is prepended.

3

I always throw perl at everything :-)

perl -ne 'if (/^\s/) { $x.=$_ }else{print $x if $x=~/apollo/; $x=$_} END {print $x if $x=~/apollo/}' file.txt

Edit: explanation of the one-liner:

  • -n implies a loop over the input, not printing by default at the end of each loop (-p is the same but does print the line by default).
  • -e specifies an expression, or code fragment. This is executed inside the implicit loop.
  • I use a $x variable to buffer the input text until a complete "record" is found.
  • /^\s/ matches whitespace at the beginning of a line. If that's found, the input line is added to the $x buffer. If not, then the "record" is complete and inspected for the search string "apollo". If that's found, the record is printed. The buffer is cleared after it's handled.
  • The END {} part is executed after the loop is completed to handle the case of the last record in the input.
  • As with most uncommented one-liner Perl, my first thought was "Did your cat just walk all over your keyboard?". Some clarification on what this is doing and why would be useful. – Anko Sep 12 '14 at 10:11
  • Thanks for the workaround. I'll experiement with perl if I can't get awk to work. I tested your cli on the text file. It doesn't give any output at present. I'm still working with it. – L. D. James Sep 12 '14 at 10:12
  • The one-liner gathers input lines into $x until a line that doesn't start with whitespace is encountered. Then it prints $x (the gathered data) if it contains apollo; $x is then set to the current (new) line of text. The input file needs to be added at the end, I forgot that. One problem I just considered: it won't match the last "paragraph" as it doesn't find a next line with whitespace at the beginning... I'll try to fix it. Otherwise it works here with the sample data. – wurtel Sep 12 '14 at 10:16
  • @Anko I hope to have the text file broken into records. The start of each record is specified by non blank characters in the first column of the text file. The example text file has 6 records. I hope to handle one record at a time or any specific record. In this example I would like to isolate the record that has the string "apollo" in it. I'll also run a routine to check each record that has "DATE_CHANGED". When I can process the record(s) that has "apollo" in it, I'll use the same formula to process other strings... no cat in the house. – L. D. James Sep 12 '14 at 10:18
  • @wurtel - can you please add an explanation to your answer? You can click the "edit" link and do so directly on your A. Putting it in the comments causes it to get lost in the noise. – slm Sep 12 '14 at 10:39
1

with a awk and a parameter

(file filter.awk)

BEGIN { p=0 ; } # no printing
 { if ( (substr($0,1,1) != " ") && (substr($0,1,1) != "\t" ) ) p=0 ; # if no blank stop printing
   if ( index($0,name) > 1 ) p=1 ; # pattern found ?
  if (p) print ;
 }
  • $0 is the entire line
  • substr($0,1,1) is first caracter of line

and the result

awk -v name=apollo -f filter.awk a.txt

    com.apollo.apollonetworkcheck
     1026ms running, 88 wakeups
     88 alarms: flg=0x14
  • Nice, +1! Note that this will fail if the file is indented by tabs and not spaces though. – terdon Sep 12 '14 at 12:29
  • Variables are empty/zero by default and non-indented selects one way or the other, so a little simpler: awk '/^[^ \t]/{p=/apollo/} p' or if you need to make it variable {p=$0~pattern} or {p=index($0,literal)} – dave_thompson_085 May 21 '16 at 0:43
1

I also throw perl at everything but will sometimes add sed for flavor:

$ sed  's/^\w/\n&/' file | perl -000ne 'print if /apollo/'
com.apollo.apollonetworkcheck
    1026ms running, 88 wakeups
    88 alarms: flg=0x14

The sed will add an extra newline between each record. It simply looks for lines that begin with an alphanumeric character (^\w) and the replace that character with a newline and followed by the character itself (\n&, the & means "whatever you just matched"). The result is a file where records look like paragraphs, they have a blank line before them.

We can now use perl's "paragraph mode", activated by -000. Combined with -n (read input record by record), this allows us to read the entire record at once. So, all we need to do is print the current "line" (record) if it matches the desired pattern. In this case apollo.

I'm not sure how portable the \w notation is. If your sed can't deal with it, use this instead:

sed  's/^[^ \t]/\n&/' file | perl -000ne 'print if /apollo/'

You can also use the same approach with awk:

$ sed  's/^[^ \t]/\n&/' file | awk -v RS="\n\n" '/apollo/'
com.apollo.apollonetworkcheck
     1026ms running, 88 wakeups
     88 alarms: flg=0x14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.