How to get data count between two strings with multiple occurences in the same file

Question

I have the data in my file as

START-OF-FIELDS
ID
NAME
DEPT
END-OF-FIELDS

START-OF-DATA
1|joy|cs
2|sam|ec
END-OF-DATA

START-OF-FIELDS
ID 
NAME
DOB
DEPT
ADDRESS
END-OF-FIELDS

START-OF-DATA
5|joe|13/2/2001/|ee|street1
3|gwen|4/08/1999|cs|street2
END-OF-DATA

EXPECTED OUTPUT:-

ID
NAME
DEPT

Count for data set 1 is: 3

ID 
NAME
DOB
DEPT
ADDRESS

Count for data set 2 is: 5 and so on for dynamic no. of data sets.

I want count of data between both START-OF-FIELDS & END-OF-FIELDS & for START-OF-DATA & END-OF-DATA. can any body give me exact code to do this. I have used

sed -n '/^START-OF-FIELDS/,/END-OF-FIELDS/{s/^START-OF-FIELDS//;/^END-OF-FIELDS/d;p;}' Sourcefile.txt > START_OF_FIELDS.TXT

wc -l START_OF_FIELDS.TXT

But I am not getting desired result.

Answer 1

Given the new better problem description, this is actually easier

#!/bin/bash
awk 'BEGIN {
            SETNR=0; 
            MODE="Non-Fields"}
(MODE=="Fields") && (! /START-OF-/ ) && (! /END-OF/ ) {
            print; 
            COUNT++
           }
/START-OF-FIELDS/ {
            COUNT=0; 
            SETNR++; 
            MODE="Fields"
           }
/END-OF/ {
            if (MODE=="Fields") {
                 printf ("Count for data set %d is: %d\n", SETNR, COUNT)
                };
            MODE="Non-Fields"
         }' $*

Answer 2

A solution in awk:

$ awk '/END-OF/{flag=0;printf "Count for data set %d is: %d\n",++i,count;count=0;printf "\n"}
    {if(flag){
         count++;
         print;
     }}
    /START-OF/{flag=1}' file
ID
NAME
DEPT
Count for data set 1 is: 3

1|joy|cs
2|sam|ec
Count for data set 2 is: 2

ID 
NAME
DOB
DEPT
ADDRESS
Count for data set 3 is: 5

5|joe|13/2/2001/|ee|street1
3|gwen|4/08/1999|cs|street2
Count for data set 4 is: 2

Answer 3

Your problem needs to be better defined, but looking at your sample input I am making a guess as to what you want to do. Use a small AWK script like this:

#!/bin/bash

awk '! /START-OF-/ {
  if (MODE=="DEFINE") { HEADER=HEADER $0 "|" }
  else if ((MODE=="DATA") && (NF>0)) { print }
}
 /START-OF-FIELDS/ { MODE="DEFINE"; HEADER="" }
 /START-OF-DATA/ {
      print HEADER
      MODE="DATA"
      HEADER=""
     }' | sed 's/|$//'

This produces:

ID|NAME|DEPT
1|joy|cs
2|sam|ec
ID |NAME|DOB|DEPT|ADDRESS
5|joe|13/2/2001/|ee|street1
3|gwen|4/08/1999|cs|street2

Note: I am lazy, so I add a vertical-bar after every field in the header. Then I use the SED command to remove the trailing V-bars afterwards.

Answer 4

sed -n '/^START-OF-DATA$/,/^$/{/^START-OF-DATA$/d;/^$/d;p;}' filename

That is:

• Collect the lines between START-OF-DATA and the next empty line - /^START-OF-DATA$/,/^$/. With those lines:
  • Delete the delimiter lines - /^START-OF-DATA$/d and /^$/d
  • Print the rest - p