0

I have a huge text file where data is separated pagewise. For example, the text is in below format and I want to read lines pagewise and store each page in separate files.

Page 1
 line 1
 line 2
 line 3
 line 4
 line 5
Page 2
 line 1
 line 2
 line 3
 line 4
 line 5
Page 3
 line 1
 line 2
 line 3
 line 4
 line 5
1
  • 3
    Please Edit your question and explain what you have tried so far and what your problem is within the scope of Linux/Unix.
    – X Tian
    Commented May 16, 2019 at 10:36

4 Answers 4

4

Solution:

awk '/^Page [0-9]+$/ {N = $2}; {print > "page"N}' pages

Test run:

==> page1 <==
 line 1
 line 2
 line 3
 line 4
 line 5

==> page2 <==
 line 1
 line 2
 line 3
 line 4
 line 5

==> page3 <==
 line 1
 line 2
 line 3
 line 4
 line 5

If there are thousands of pages though, with some awk implementations, you may run into the limit of number of open files. You can address that by closing the file for the previous page when finding the new page header. You may also want to 0-pad the numbers in the file names so that ls shows them in ascending order:

awk '
  /^Page [0-9]+$/ {close(file); file = sprintf("page%04d", $2)}
  {print > file}' pages

That assumes the input contains only one occurrence of Page n for each page.

1

Alternate solution, ableit not as elegant as the awk one because it requires:

  • knowing the number of lines per page
  • this number has to remain constant across all pages

    split -l 6 -a 2 -d input page_
    

Here, 6 is the number of lines per page: 1 title line + 5 data lines

1

Using csplit:

$ csplit -s -f page- file '/^Page/' '{1}'
$ ls
file    page-00 page-01 page-02
$ cat page-00
Page 2
 line 1
 line 2
 line 3
 line 4
 line 5
$ cat page-01
Page 2
 line 1
 line 2
 line 3
 line 4
 line 5
$ cat page-02
Page 3
 line 1
 line 2
 line 3
 line 4
 line 5

The csplit utility splits the given file according to its last arguments. Here, /^Page/ and {1} instructs csplit to split the file on the given regular expression, each time it's matching. With GNU csplit, you would want to use {*} instead of {1} (the example above was written for OpenBSD csplit).

-1

command:

awk '$0 ~/Page 1/{f=1}$0 ~/Page 2/{f=0}f' o.txt > firstfile
 awk '$0 ~/Page 2/{f=1}$0 ~/Page 3/{f=0}f' o.txt > secondfile
awk '/Page 3/,/,/{print $0}' o.txt >thirdfile

output

een@praveen:~$ awk '$0 ~/Page 1/{f=1}$0 ~/Page 2/{f=0}f' o.txt > firstfile
Page 1
 line 1
 line 2
 line 3
 line 4
 line 5
praveen@praveen:~$ awk '$0 ~/Page 2/{f=1}$0 ~/Page 3/{f=0}f' o.txt > secondfile
Page 2
 line 1
 line 2
 line 3
 line 4
 line 5



awk '/Page 3/,/,/{print $0}' o.txt >thirdfile
Page 3
 line 1
 line 2
 line 3
 line 4
 line 5
praveen@praveen:

~$

1
  • 2
    The word "huge" in the question implies that there may well be more than three pages.
    – Kusalananda
    Commented May 20, 2019 at 20:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .