0

I have below input file,

Policy Name:       KE15-LOCALHOST-APP-RADIX-DAILY

  Policy Type:         Standard
  Active:              yes
  Include:  /appussd
            /home/ussd2ke
            /var/log
            /etc
            /usr

  Schedule:              Montlhy_Full
    Type:                Full Backup
    PFI Recovery:        0
    Maximum MPX:         16
    Retention Level:     5 (3 months)
    Daily Windows:
          Sunday     00:00:00  -->  Sunday     07:00:00
          Monday     00:00:00  -->  Monday     07:00:00
          Tuesday    00:00:00  -->  Tuesday    07:00:00
          Wednesday  00:00:00  -->  Wednesday  07:00:00
          Thursday   00:00:00  -->  Thursday   07:00:00
          Friday     00:00:00  -->  Friday     07:00:00
          Saturday   00:00:00  -->  Saturday   07:00:00

  Schedule:              Weekly_Full
    Type:                Full Backup
    PFI Recovery:        0
    Maximum MPX:         16
    Retention Level:     3 (1 month)
    Daily Windows:
          Wednesday  00:00:00  -->  Wednesday  10:00:00

  Schedule:              Daily_Inc
    Type:                Differential Incremental Backup
    PFI Recovery:        0
    Maximum MPX:         16
    Retention Level:     2 (3 weeks)
    Daily Windows:
          Sunday     01:00:00  -->  Sunday     16:00:00
          Monday     01:00:00  -->  Monday     16:00:00
          Tuesday    01:00:00  -->  Tuesday    16:00:00
          Wednesday  01:00:00  -->  Wednesday  16:00:00
          Thursday   01:00:00  -->  Thursday   16:00:00
          Friday     01:00:00  -->  Friday     16:00:00
          Saturday   01:00:00  -->  Saturday   16:00:00

Now I need different pairs of Type: (Below Schedule), Retention Level, and Daily Window separated by a comma and by ; for multiple entries.

Here is the command I have tried, the issue is with Daily Window, I am able to fetch the data in between and chuck out Daily Window Line, Now i need to chuck out Weekdays name and just want unique time slot

awk '
  BEGIN { SEP = "" }
  $1 == "Type:" { $1 = ""; T = T SEP $0 }
  $1 == "Retention" && $2 == "Level:" {
    sub(/^.*\(/," ")
    sub(/\).*/,"")
    L = L SEP $0
    if (SEP == "") {
      SEP = ";"
    }
  }
  /Daily Windows:/,/^$/ {
  sub(/^.*Daily.*/,"")
  sub(/^[^A-Z][a-z]+y$/,"")
  S = S SEP $0}
  END {
  sub(/^ */,"",T)
  print T "," L "," S
}'

Below is the output:

Full Backup; Full Backup; Differential Incremental Backup, 3 months; 1 month; 3 weeks,;;          Sunday     00:00:00  -->  Sunday     07:00:00;          Monday     00:00:00  -->  Monday     07:00:00;          Tuesday    00:00:00  -->  Tuesday    07:00:00;          Wednesday  00:00:00  -->  Wednesday  07:00:00;          Thursday   00:00:00  -->  Thursday   07:00:00;          Friday     00:00:00  -->  Friday     07:00:00;          Saturday   00:00:00  -->  Saturday   07:00:00;;;          Wednesday  00:00:00  -->  Wednesday  10:00:00;;;          Sunday     01:00:00  -->  Sunday     16:00:00;          Monday     01:00:00  -->  Monday     16:00:00;          Tuesday    01:00:00  -->  Tuesday    16:00:00;          Wednesday  01:00:00  -->  Wednesday  16:00:00;          Thursday   01:00:00  -->  Thursday   16:00:00;          Friday     01:00:00  -->  Friday     16:00:00;          Saturday   01:00:00  -->  Saturday   16:00:00

However, the desired output is below:

Full Backup; Full Backup; Differential Incremental Backup, 3 months; 1 month; 3 weeks, 00:00:00  -->  07:00:00; 00:00:00  -->  10:00:00; 01:00:00  -->  16:00:00
  • Is the 'odd' day always Wednesday? Is the time interval always the same within each schedule? Is the use of awk a requirement? – bu5hman Apr 8 at 17:30
1

Looks if we use an optional colon : following by at least two spaces as the FS (FS = ":? *"), most of the main fields used in this task can be isolated out without hassling issue from extra spaces:

$ cat t20.awk
BEGIN { FS=":?   *"; OFS = ", "; SEP = "; "; }

# if $2 is "Type", append $3 to T
$2 == "Type" { T = (T ? T SEP : "") $3;}

# if $2 is "Retention Level", append sub-string in parenthesis to L
$2 == "Retention Level" && match($0, /\(.*?\)/) {
    L = (L ? L SEP : "") substr($0, RSTART+1, RLENGTH-2)
}

# in Daily window block, skip all line without " --> "
# use an associative array "a" to make sure unique time range
/Daily Windows:/,/^\s*$/ {
    if (!/ --> /) next
    key = $3 " --> " $6
    if (!a[key]++) S = (S ? S SEP : "") key
}

END { print T, L, S }

Note:

  1. in S = (S ? S SEP : "") key, the ternary (S ? S SEP : "") is to avoid the leading SEP when concatenating strings, similar to those in concatenating T, L.

  2. in substr($0, RSTART+1, RLENGTH-2), using RSTART+1 to remove the leading (, and RLENGTH-2 to remove the two parenthesis

Run the code:

$ awk -f t20.awk file.txt
#Full Backup; Full Backup; Differential Incremental Backup, 3 months; 1 month; 3 weeks, 00:00:00 --> 07:00:00; 00:00:00 --> 10:00:00; 01:00:00 --> 16:00:00

Update:

Base on your description in the comments, I did the following adjustments to code on the Daily Windows part:

  • Added a flag dw_on to identify the start and end of Daily Windows block. All lines with dw_on == 1 and matching the pattern / --> / should be checked for S. this flag will be reset to 0 whenever the next EMPTY line /^\s*$/ is detected
  • Added a variable cnt_DW to count number of Daily Windows entries in each Schedule. this will be reset at the beginning of each Daily Windows block

The uniqueness is maintained by a hash(associative array) a, which will be reset at the beginning of each Daily Windows block. The key of this hash is key = $3 " --> " $6 which is the window you want to retrieve. the syntax: if (!a[key]++) S = (S ? S SEP : "") key is the same as

  if (!a[key]) { 
      a[key] = a[key] + 1
      S = (S ? S SEP : "") key 
  }

so only if a key is not seen before (a[key]=""), can a key be appended to S, the 2nd times when the same key is processed, it already have a[key]==1 and will skip the above code block. This is one of the common way in awk to check uniqueness.

$ cat t20.1.awk
BEGIN { FS=":?   *"; OFS = ", "; SEP = "; "; }

# if $2 is "Type", append $3 to T
$2 == "Type" { T = (T ? T SEP : "") $3;}

# if $2 is "Retention Level", append sub-string in parenthesis to L
$2 == "Retention Level" && match($0, /\(.*?\)/) {
    L = (L ? L SEP : "") substr($0, RSTART+1, RLENGTH-2)
}

/Daily Windows:/ {
    # turn on the dw_on flag and reset cnt_DW (number of DW entries in a section)
    dw_on = 1; cnt_DW=0;
    # reset the hash 'a' for uniqueness check
    # if you need the uniqueness across all Schedules, then comment it out
    delete a; 
    next;
}

# if dw_on flag is true, i.e. "dw_on == 1"
dw_on {
    # match " --> ", then increase cnt_DW, check the unique window
    # and then append qualified entry to "S"
    if (/ --> /) {
        cnt_DW++
        key = $3 " --> " $6
        if (!a[key]++) S = (S ? S SEP : "") key
    # else if EMPTY line, reset dw_on flag, if cnt_DW is 0, append "No Window" to S
    } else if (/^\s*$/) {
        dw_on = 0;
        if (!cnt_DW) S = (S ? S SEP : "") "No Window"
    }
}

END { 
    # last Schedule section does not have a EMPTY line, so we will need
    # to check up cnt_DW in the last Schedule section in "END" block
    if(dw_on && !cnt_DW) S = (S ? S SEP : "") "No Window";

    # print the result.
    print T, L, S 
}

I made the following small modifications on your original data to test the above code:

  1. deleted the lone Daily Windows entry under the 2nd Schedule section
  2. replaced the line Friday 00:00:00 --> Friday 07:00:00 in the first Schedule to Friday 01:00:00 --> Friday 16:00:00 which is the same as in the 3rd Schedule section.

So now, in the 1st Schedule, there are 2 unique windows, in the 2nd Schedule, there is no window, in the 3rd Schedule, there is 1 unique window which is the same as one in the 1st Schedule.

Run the updated code with the above data, you will get:

awk -f t20.1.awk file.txt 
#Full Backup; Full Backup; Differential Incremental Backup, 3 months; 1 month; 3 weeks, 00:00:00 --> 07:00:00; 01:00:00 --> 16:00:00; No Window; 01:00:00 --> 16:00:00

Notice that there are two 01:00:00 --> 16:00:00 because they are in different Schedules. If you want to remove the last 01:00:00 --> 16:00:00, comment out the line delete a as shown in the code, you will get the following result:

#Full Backup; Full Backup; Differential Incremental Backup, 3 months; 1 month; 3 weeks, 00:00:00 --> 07:00:00; 01:00:00 --> 16:00:00; No Window
  • Thanks for the response jxc, could you please help me understand the solution? Also the Daily Window section, How did you achieve unique entry? Also in case there is no window in the input it should echo "No Window" – Sid Apr 9 at 12:00
  • @Sid Need some clarifications: there are three Schedule sections in your sample, when you say "No Window", do you mean No Window in all sections or any of the section, so that the result can be 00:00:00 --> 07:00:00; No Window; 01:00:00 --> 16:00:00;. same question to the uniqueness check, is this based on each Schedule or all Schedules? – jxc Apr 9 at 12:47
  • Thanks for the response, First of all, there will always be triplets of Schedule Type, Retention Level, Daily Window, Now Schedule Type & Retention level can never be blank, so that's fine in code, However Daily Windows: section can be blank, means nothing below it and another schedule type may start. For uniqueness, I meant there would be multiple entry under Daily Window, i just need unique entry (how did you achieve it?) – Sid Apr 10 at 9:00
  • @Sid, updated per your comments, let me know if there is anything you need clarification. – jxc Apr 10 at 14:08
  • @Sid, would you mind give me a feedback if the code works for your task? – jxc Apr 12 at 23:04
0

You can do all with below awk:

awk -F'(: *|[)(])' '
    /^ *Type/     { type=type==""?$2 : type ";" $2 }
    /^ *Retention/{ Retention=Retention==""?$3 : Retention ";" $3}
    /^ *Wednesday/{ gsub(/ +Wednesday/,"",$0); day=day==""?$0 : day ";" $0}
END{ print type, Retention, day }' OFS=, infile

You may need to adjust condition parts / ... / to match as more exact as your field's values.

Output is as you desired.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.