How can I manipulate the content of a file, by duplicating and changing some parts?

Question

I want to manipulate a text-file. It contains many blocks of the form

CLASS
   ...some stuff ...
END

I want to duplicate each of these blocks and add and remove a line of it's content. Can I script that?

Answer 1

Well, you can script anything.

However, if I were doing this, I would use vim macros. (If you are not familiar with vim already, you should learn it, but otherwise ignore this)

A few basic pointers:

q -- start recording a macro

qx -- stop recording a macro and name it 'x'

@x -- run the macro named 'x'

A macro will immediately re-run whatever commands you typed during the recording session.

So if you wanted a macro to duplicate a block like that type:

q          start macro
/CLASS     jump to next "CLASS"
V          start selecting full lines
/END       jump to next "END
Y          yank selected lines (yank = copy)
j          scroll down one line
p          put (paste)
qd         stop recording and name the macro d

There are plenty of good resources and examples out there for vim usages and vim macros.

For what it's worth, sed is a sort of a command line version of vim, so you might be able to adapt what I said to sed, but I'm less familiar with it, so I can't vouch for it.

~~ edit ~~

Another thing that makes this more useful is vim's ability to open up multiple files. Multiple files opened are referred to as buffers. You activate this by having multiple filenames after vim (IE vim *.txt to open all txt files)

To run a macro on multiple buffers, first open up all the files at the same time. Then, use :bufdo normal @x to run the macro x on each buffer.

Note, you must have :w as the last step of the macro to avoid getting stuck when it tries to switch to the next buffer if the current one isn't saved.

:bn will take manually to the next buffer.

Also, see https://stackoverflow.com/questions/1291962/replay-a-vim-macro-until-end-of-buffer for how to run a macro for each line. Mix and match :)

Answer 2

The natural tools for this are awk and Perl (assuming you want to script: for a once-off, the natural tool is an interactive editor). Here's an awk script that duplicates all CLASS…END blocks (no balancing supported: each CLASS matches the next END), except that foo lines are omitted from the second copy.

awk '
  /^CLASS$/ { store = 1; }  # start storing
  store && ! /^foo$/ { hold = hold ORS $0; }  # if storing, maybe save line
  /^END$/ {
      $0 = $0 hold;  # append hold copy to current line
      store = 0; hold = "";  # end of block
  }
  1 { print; }  # print original line, with hold prepended if at end of block
'

Here's a sed solution; don't take it too seriously. Don't expect it to behave if the CLASS/END lines are not in strict alternation.

sed -e '/^CLASS$/,/^END$/!b' \
    -e '/^CLASS$/{' -e 'h' -e 'b' -e '}' \
    -e '/^foo$/!H' \
    -e '/^END$/G'

Answer 3

The exact tool to use will depend on the specifics of the job. If the job is only going to happen once, you might consider using a macro in vim or emacs. If you truly want to script it, I would look into sed:

Sed is a stream editor. A stream editor is used to perform basic text transformations on an input stream (a file or input from a pipeline).

Again, the exact way to do this is will depend on your situation, but to add a line after a line starting with PATTERN you can do this:

sed -e 's/\(^PATTERN.*\)/\1\nTextToAdd/' file.old > file.new

To add a line before a line starting with PATTERN you could do something like this:

sed -e 's/\(^PATTERN.*\)/TextToAdd\n\1/' file.old > file.new

To remove a line starting with PATTERN

sed -e 's/\(^PATTERN.*\)//' file.old > file.new

The trick will be to find the correct PATTERN that will catch only the lines you want.

You can combine these by doing soemthing like this:

sed -e 's/\(^PATTERN1.*\)/\1\nTextToAdd/' -e 's/\(^PATTERN2.*\)/TextToAdd\n\1/' file.old > file.new

Answer 4

And Python solution from me:

file = open('a.txt', 'r')
arr = []
tmp = []
remember = False

for line in file:
    if "CLASS" in line:
        remember = True     # start storing lines
        tmp.append(line)    # we're storing lines in list
        continue            # go to next line
    if "END" in line:
        tmp.append(line)    # store line
        remember = False    # stop storing lines
        arr.append(tmp)     # add line to list
        tmp = []            # clear temp list
        continue            # go to next line
    if remember:
        tmp.append(line)    # add to list line between CLASS and END
file.close()

file = open('a.txt', 'a')
for tmp in arr:
    if '   ...some stuff ...1\n' in tmp:    # we can skip 
        continue                            # whole block if we find this
    for i in tmp:
        if "333" in i:  # we can skip only one line 
            continue
        file.write(i)
file.close()

I think that it is easiest to understand and modify than awk version (if you don't know awk)... Btw awk is next language on my list of languages that I want to learn... ;)