28

I have a text file which I want to split into 64 unequal parts, according to the 64 hexagrams of the Yi Jing. Since the passage for each hexagram begins with some digit(s), a period, and two newlines, the regex should be pretty easy to write.

But how do I actually split the text file into 64 new files according to this regex? It seems like more of a task for perl. But maybe there's a more obvious way that I'm just totally missing.

5 Answers 5

38

This would be csplit except that the regex has to be a single line. That also makes sed difficult; I'd go with Perl or Python.

You could see if

csplit foo.txt '/^[0-9][0-9]*\.$/' '{64}'

is good enough for your purposes. (csplit requires a POSIX BRE, so it can't use \d or +, among others.)

5
  • Thanks, @geekosaur. It worked perfectly, though I had to change it to {63}.
    – ixtmixilix
    Jun 26, 2011 at 19:47
  • 1
    So, '\.' won't work too?
    – Vanuan
    Feb 3, 2016 at 10:18
  • 1
    What does the {64} mean here?
    – alberto56
    Oct 23, 2022 at 14:48
  • @alberto56 The man page for csplit says that {64} means that the first pattern, /^[0-9][0-9]*\.$/, is to be repeated 64 times.
    – Student
    Dec 19, 2022 at 23:51
  • For anyone whose use case is slightly different and you don't know the number of fragments, use '{*}' instead Apr 25 at 3:56
7

I think the best way is awk and gawk.

awk

awk -F "([.] )|( / )" '/^[0-9]{1,3}[.]/{x="F"$1"("$2").txt";}{print >x;}' I_Ching_Wilhelm_Translation.txt

-F will specify fields seperator for each line. It is a regex, here we use multiple seperators: ". " and " / ". Thus a line like 1. Ch'ien / The Creative will be split into 3 fields: 1 Ch'ien and The Creative. Later we can refer to these fields with $n. $0 is the entire line.

We then tell the awk to match the lines with pattern ^[0-9]{1,3}[.] If there is match we then assign value to x. The value x will be used as file name for print operation. In this example we use "F"$1"("$2").txt" so the line 1. Ch'ien / The Creative gives a filename F1(Ch'ien).txt

gawk

In gawk, we can also access captured group. So we can simplify the command to:

gawk 'match($0, /^([0-9]{1,3})[.] (.*) \/ (.*)$/, ary){x="F"ary[1]"("ary[2]")";}{print >x;}' I_Ching_Wilhelm_Translation.txt

here we use match the capture the groups and put them into variable list ary. $0 is the entire line. ary[0] is everything matched. ary[1...n] is each group.

perl

We can also do it with perl:

perl -ne 'if(/^([0-9]{1,3})[.] (.*) \/ (.*)$/) {close F; open F, ">", sprintf("F$1($2).txt");} print F' I_Ching_Wilhelm_Translation.txt

Results:

> ls F*
F10(Lü).txt         F22(Pi).txt       F34(Ta Chuang).txt  F46(Shêng).txt     F58(Tui).txt
F11(T'ai).txt       F23(Po).txt       F35(Chin).txt       F47(K'un).txt      F59(Huan).txt
F12(P'i).txt        F24(Fu).txt       F36(Ming I).txt     F48(Ching).txt     F5(Hsü).txt
F13(T'ung Jên).txt  F25(Wu Wang).txt  F37(Chia Jên).txt   F49(Ko).txt        F60(Chieh).txt
F14(Ta Yu).txt      F26(Ta Ch'u).txt  F38(K'uei).txt      F4(Mêng).txt       F61(Chung Fu).txt
F15(Ch'ien).txt     F27(I).txt        F39(Chien).txt      F50(Ting).txt      F62(Hsiao Kuo).txt
F16(Yü).txt         F28(Ta Kuo).txt   F3(Chun).txt        F51(Chên).txt      F63(Chi Chi).txt
F17(Sui).txt        F29(K'an).txt     F40(Hsieh).txt      F52(Kên).txt       F64(Wei Chi).txt
F18(Ku).txt         F2(K'un).txt      F41(Sun).txt        F53(Chien).txt     F6(Sung).txt
F19(Lin).txt        F30(Li).txt       F42(I).txt          F54(Kuei Mei).txt  F7(Shih).txt
F1(Ch'ien).txt      F31(Hsien).txt    F43(Kuai).txt       F55(Fêng).txt      F8(Pi).txt
F20(Kuan).txt       F32(Hêng).txt     F44(Kou).txt        F56(Lü).txt        F9(Hsiao Ch'u).txt
F21(Shih Ho).txt    F33(TUN).txt      F45(Ts'ui).txt      F57(Sun).txt

how to get the example file:

curl http://www2.unipr.it/~deyoung/I_Ching_Wilhelm_Translation.html|html2text -o I_Ching_Wilhelm_Translation.plain
sed 's|^[[:blank:]]*||g' I_Ching_Wilhelm_Translation.plain > I_Ching_Wilhelm_Translation.txt
3

With GNU coreutils, you can use csplit to break a file into regexp-delimited pieces, as shown by geekosaur.

Here's a portable awk script to break a file into pieces. It works by

  • calling getline to deal with the multiline (2-line) separator;
  • setting a variable outfile to the name of the file to print to, when a section header is encountered.
BEGIN {outfile="header.txt"}
{
    while (/^[0-9]+\.$/) {
        prev = $0; getline;
        if ($0 == "") outfile = prev "txt";
        print prev >outfile
    }
    print >outfile
}
2
  • This works in principle, but the section-header of the actual web page data is not as represented by the regex (likewise with geekosaur's answer). The leading nunber. is followed by text which contain a slash /. I'm pretty sure the two newlines ixtmixilix mentioned are the 2 blank lines which precede the numeric identifier and would more specifically identify the header, but as the data on the web page only matches /^[0-9]+\. in the section headers, there is no need to cater for them (in this particular case). thanks; especially for the intro to getline.. PS. can while be if?
    – Peter.O
    Jun 27, 2011 at 13:58
  • @fred geekosaur and I went by the description in the question, not by the data on the website. The layout will depend on the HTML rendering engine used to convert to text; the part where this is rendered from a web page is actually irrelevant to the question. ||| while is there in case the input contains 1.\n2.\n\n (where \n are newlines): the 2. must be recognized in the header line. It's not going to occur here, but I support it in my code to make it more general (and match the specification in the question more strictly). Jun 27, 2011 at 14:20
0

On MacOS, split has a -p parameter that takes a regexp. Each time split encounters a line matching the regexp, a new output file is opened, starting with that line.

The MacOS manpage I have is dated Aug 21st 2005.

0

IF your reason for splitting is to process each block with a different command, GNU Parallel may be the tool of choice:

cat I_Ching_Wilhelm_Translation.txt |
  parallel -N1 --pipe --regexp --recend '\n' --recstart '[0-9]{1,3}[.] [^\n]+ /' 'cat > {#}'

Here you can replace cat > {#} with the command to run.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .