1

I'm trying to pre-process an invoice file and separate each invoice into separate files. The invoices may be multi-page. The header of EACH PAGE looks like this:







                                                                 121084
                                                                   A134
                                                               09.17.19
                                                                      1

There are 6 blank lines at the top of each page, followed by the invoice #, customer #, date and page # (then the rest of the invoice).

At each " 1" (70 spaces followed by "1") on line 10 of each page (Line 10 of each page is the invoice's page #), I need to insert some text on line 1 (which will be used as a delimiter to split the file). Invoices can have multiple pages but the " 1" (70 spaces followed by "1") indicates that this is a new invoice.

When " 1" (70 spaces followed by "1") is encountered, I want to insert some text on the blank line that is 9 lines above it (which is the first line of this invoice). Do this for every every occurrence in the file. Then I can break the file apart into a separate file for each invoice.

I know I can use sed to insert data immediately before the pattern but how can I insert data 9 lines above it?

I can usually get it done with sed & awk but this one is stumping me.

  • Thanks for adding the line numbers, but I'm afraid that makes things considerably less clear, not more :). Can you please edit your question and use the formatting tools to format your file as code? Just paste the actual file into your question, then select the pasted lines and click on the {} button. That way, we can be sure your example accurately represents your real data and we can then use that example to test out solutions. – terdon Sep 19 '19 at 14:27
  • Also, if the invoices can have variable lengths, how can we know where they're supposed to end? We can't just use the line number since that will change. – terdon Sep 19 '19 at 14:30
  • Do you have to do this for every set of 10 lines, or is it one per file, or ? Also, you say "at each 1 on line 10 (which is the page number)" -- do we assume that every page is numbered 1 or that the value changes? – Jeff Schaller Sep 19 '19 at 14:55
  • Let me put this another way; could you just process lines 1, 11, 21, etc to add the delimiter there; or even just blindly split the files on every tenth line? Why not? (Please edit your question to clarify; thank you!) – Jeff Schaller Sep 19 '19 at 15:19
  • Thanks for the info & questions. I've edited the post, hopefully clarifying it. – John Zupancic Sep 19 '19 at 16:29
2

Assuming that what you mean to say is:

I have a long file with several invoices. Each invoice starts with the text 70 spaces and a 1. I need to insert new content 9 lines before that one for each invoice

Then what you need to do is accumulate 10 lines. When the present line matches the invoice start insert at the first line some new text.

In practice, this accomplishes that:

sed -e '1{N;N;N;N;N;N;N;N}' -e 'N;l;/\n \{70\}1$/{iNew content here' -e 'P};P;D' file

Or, in long form (comments doesn't work on some sed implementations):

sed '1{                              # (only) on the first line
         N;N;N;N;N;N;N;N             # accumulate 9 lines (first one plus 8 more).
      }
     N                               # On every line,also accumulate that line 
     /\n \{70\}1$/{                  # If buffer ends ($) in 70 spaces and a "1"
i\
New content added here               # insert the content of this line at
                                     # the start of the buffer (10 lines above).
                       P             # and then print it.
                  }
     P;D                             # close the N cycle above by
                                     # printing and deleting one line
    ' file                           # On the selected file.

An awk solution could take advantage of setting the RS to the line that marks an invoice start. In that case, setting the FS to a newline character breaks each line into a field and the $(NF-9) will refer to a line 9 lines above :

awk -v ln=9 '
              ( NF < ln ){ print; next };           # not enought fields?
                                                    # include
                         { $(NF-ln) = "New Text to include" newln $(NF-ln);
                           print                    # and print
                         };
              BEGIN{ breakln = sprintf("%71s",1);   # 70 spaces and a 1
                     newln   = sprintf("\n");       # a newline
                     RS      = breakln newln;       # set the Record Separator
                     FS      = "\n";                # set the Field separator
                     OFS     = FS;                  # print what got removed
                     ORS     = RS                   # print what got removed
                   }
            ' file

Or, alternatively (shell/awk solution):

breakln="$(printf '%71s' 1)";
newl=$'\n';
awk '  (NF<ln){print;next};
              { $(NF-ln)="New Text to include\n"$(NF-ln); print}' \
    ln=10 \
    RS="$breakln$newl" \
    FS="$newl" \
    OFS="$newl" \
    ORS="$breakln$mewl" \
    file
  • Question from someone less sed-savvy: I interpreted the request to be to change line N-9 to some new text; will the i command insert a new line there, or change the current one? The OP's wording is a little ambiguous, but by "I need to insert some text on line 1" I interpreted it to mean "make line 1 look like this" versus "insert a new line 1" – Jeff Schaller Sep 20 '19 at 0:13
  • The sed command i is used to insert text at the beginning of the pattern buffer (as a is used to append at the end of the buffer). To change the buffer you can use (not surprise) the c command. But is almost always easier to use a s/// command. In this case, a s/^[^\n]*\n/New Text/ (in GNU sed) could replace the first line, if needed. @JeffSchaller – Isaac Sep 20 '19 at 1:28
1

With awk + tac:

tac file | awk -v delim="--split page here--" '{
  if (nextnr=="" && $1~/^[0-9]+$/ && $0=="                                                                      "$1) {
     nextnr=NR+9  # pagenr found, remember next position
  }
  else if (NR==nextnr) {
     $0=delim     # overwrite line with delimiter
     nextnr=""    # reset
  }  
  print
}' | tac

First reverse the file with tac, so we can search top-down for the page number and insert the delimiter.

The search begins in the if-clause when these conditions are met:

  • nextnr is (initially) not set (nextnr==""). This is the variable holding the the line number of the next separator.
  • the first field is a number ($1~/^[0-9]+$/)
  • the line contains 70 spaces and the number

If all three conditions are true, nextnr is set to the current line number (number of records) NR + 9.

If the current line is the line of the delimiter (NR==nextnr), overwrite the line with the delimiter and reset nextnr.

The last line of the script prints the current line (original or overwritten by the delimiter).

In the last step the output is reversed again with tac.

0

Here's a scriptable-editor solution. The idea is to find out how many invoices there are in the file, based solely on the number of "(70 spaces) 1" lines there are in the file. The script then loops that many times and outputs commands for ed. The loop outputs enough commands to change every "9th line before the one with '(70 spaces) 1'" to instead have a dashed snipping line. Replace the -----8<----- text with anything you like besides a bare leading period (which ed uses to distinguish the end of the replacement string. If we are in the midst of looping through the invoices (i < count), skip ahead 10 lines after making the change so that we don't rediscover the page we just snipped. If we've finished the loop (i == count), print out ed's "write and quit" commands. All of that printf/echo output goes to the pipe, which ed reads as input. The -s option is for "silent" mode -- ed will not report the number of bytes read or written.

#!/usr/bin/bash
count=$(grep -c '^                                                                      1' input)
for((i=1; i<=count; i++))
do
  printf '%s\n' '/^                                                                      1/-9c'
  printf '%s\n' '-----8<-----' '.'
  [[ $i < $count ]] && printf '%s\n' '+10'
  [[ $i == $count ]] && echo wq
done | ed -s input

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.