1

I have a file ~/bigfile.txt that consists of thousands of blocks of text of the form

BLOCK NUMBER : <block>
SIZE : <size1> <size2>
EXTRA : <extraNumber>
<block of text>

For this example say

$ cat ~/bigfile.txt
BLOCK NUMBER : 1
SIZE : 7 6
EXTRA : 0
john paul
george ringo

BLOCK NUMBER : 2
SIZE : 7 3
EXTRA : -10
i am
the
walrus

BLOCK NUMBER : 3
SIZE : 4 3
EXTRA : -1024
hello world

I am trying to write a script that separates each block into a separate file named <block>-block.txt, nested into subdirectories of ~/data/ indexed by <size1> and <size2>. For instance, running the script should result in

$ tree ~/data/
~/data/
|- 4-size1
   |- 3-size2
      |- 3-block.txt
|- 7-size1
   |- 3-size2
      |- 2-block.txt
   |- 6-size2
      |- 1-block.txt

Currently I have a script that simply dumps each block to a separate file in ~/data/ but I can't figure out how to alter it.

I can post my current script if that would help but I suspect that it's quite inefficient and not suited to tackle this sort of organizational task.

I would appreciate any pointers on how to accomplish this task with perl.

1

I would write:

perl -MFile::Path=make_path -00 -ne '
    ($block) = /BLOCK NUMBER : (\d+)/; 
    @sizes = /SIZE : (\d+) (\d+)/; 
    $dir = sprintf "data/%d-size1/%d-size2", @sizes;
    make_path $dir;
    if (open $fh, ">", "$dir/$block-block.txt") {
        print $fh $_;
        close $fh;
    }
' bigfile.txt 
  • Is it possible to adjust this code to allow for variable spacing between the : and block number as well as the : and the size numbers? As the numbers get bigger in my file these spaces are different. – Brian Fitzpatrick Dec 2 '15 at 6:59
  • Absolutely. Just change the space to \s* – glenn jackman Dec 2 '15 at 11:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.