6

Is there a way to edit a matched pattern and then replace another pattern with the edited pattern?

Input:

a11.t
some text here
a06.t
some text here

Output:

a11.t 11
some text here
a06.t 06
some text here

The above example shows the first two digits (matched by first pattern) extracted and placed at the end of the line (second pattern).

In a programming language, I would load the file into a data structure, edit, replace, and write to a new file. But is there a one-line equivalent?

Trial:

sed 's/\(a[0-9][0-9].*\)/& \1/I' stack.fa | sed -e 's#a##g2' -e 's#\.\w##g2'

Trial output:

a11.t 11
some text here
a06.t 06
some text here

Obviously the trial works, but is there a more robust way? Further, is there another text processing language this could done in more easily?

1
  • Do we need to consider cases where you have more than one set of digits on the same line? Can you have things like foo 10 bar 20 baz and, if yes, what should happen to it? Should it become foo 10 bar 20 baz 10 20 or just foo 10 bar 20 baz 10?
    – terdon
    Aug 6, 2020 at 15:07

5 Answers 5

13

Although it has fallen out of fashion, few languages can match perl for text processing. For instance:

  1. Assume only one set of numbers, copy to the end of the line:

     $ perl -pe 's/.*?a(\d+).*/$& $1/' file
     a11.t 11
     some text here
     a06.t 06
     some text here
    
  2. Multiple sets of numbers, add both to the end

     $ cat file
     a11.t
     some text here
     a06.t
     some text here
     a11.t a54.g
    
     $ perl -pe '@nums=(/a(\d+)/g); s/$/ @nums/' file
     a11.t 11
     some text here 
     a06.t 06
     some text here 
     a11.t a54.g 11 54
    
2
  • 1
    While Perl may be passe, you would be hard-pressed to find a *nix installation that doesn't automatically have it installed. Python might be the only one to compete at that distinction.
    – UncleCarl
    Aug 7, 2020 at 20:25
  • @UncleCarl you're preaching to the choir :)
    – terdon
    Aug 7, 2020 at 23:04
10

sed here is the perfect tool for the task. However note that you almost never need to pipe several sed invocations together as a sed script can be made of several commands.

If you wanted to extract the first sequence of 2 decimal digits and append following a space to end of the line if found, you'd do:

sed 's/\([[:digit:]]\{2\}\).*$/& \1/' < your-file

If you wanted to do that only if it's found in second position on the line and following a a:

sed 's/^a\([[:digit:]]\{2\}\).*$/& \1/' < your-file

And if you don't want to do it if that sequence of 2 digits is followed by more digits:

sed 's/^a\([[:digit:]]\{2\}\)\([^[:digit:]].*\)\{0,1\}$/& \1/' < your-file

In terms of robustness it all boils down to answering the question: what should be matched? and what should not be?. That's why it's important to specify your requirements clearly, and also understand what the input may look like (like can there be digits in the lines where you don't want to find a match?, can there be non-ASCII characters in the input?, is the input encoded in the locale's charset? etc.).

Above, depending on the sed implementation, the input will be decoded into text based on the locale's charmap (see output of locale charmap), or interpreted as if each byte corresponded to a character and bytes 0 to 127 interpreted as per the ASCII charmap (assuming you're not on a EBCDIC based system).

For sed implementations in the first category, it may not work properly if the file is not encoded in the right charset. For those in the second category, it could fail if there are characters in the input whose encoding contains the encoding of decimal digits.

4
  • +1 but why include a redirection for the input file ?
    – Cbhihe
    Aug 7, 2020 at 8:20
  • @Cbhihe, why not? There are plenty of good reasons to prefer redirection over passing a filename argument to the command for it to open the corresponding file themselves. You'll find a number Q&As covering that here. Aug 7, 2020 at 8:29
  • 1
    @Cbhihe one example: try echo "foo" > file1; chmod oug-r file1; echo "bar" > file2; sed 's/o/O/' < file1 > file2; cat file2; sed 's/o/O/' file1 > file2; cat file2 and note that the approach with input redirection doesn't zap file2 when file1 can't be opened.
    – Ed Morton
    Aug 7, 2020 at 14:55
  • 1
    @EdMorton: I'd never realized that before. (Does it ever stop, o_O ¿@#!? ) Thank you.
    – Cbhihe
    Aug 7, 2020 at 15:26
3

The simplest way is via the following:

$ perl -lne '$,=$"; print $_, /a(\d+)/' file
# or this 
$ perl -lpe 's/a(\d+).*\K/ $1/' file
$ awk '
    match($1, /^a[[:digit:]]+/) &&
    gsub(/$/, FS substr($1, RSTART+1, RLENGTH-1)) ||
  1' file

Note:it is safe to use substr within gsub's replacement portion since we already made sure it is pure digital.

0

command

sed 's/\([a-z]\{1,\}\)\([0-9]\{2\}\)\(\..*\)/& \2/g' file

output

a11.t 11
some text here
a06.t 06
some text here

Python

#!/usr/bin/python
import re
m=re.compile(r'[0-9]')
k=open('g','r')
for i in k:
    if re.search(m,i):
        print i.strip(),i.split(".")[0][1:]
    else:
        print i.strip()

output

a11.t 11
some text here
a06.t 06
some text here
0
0

perl or sed are the way to go, but just for the sake of completion, a re-write of @PraveeenKumarBS' python snippet's first version, using module re's concept of matched "group":

#!/usr/bin/python3
import re
pattern = re.compile(r'(\d{2})')
with open('data', 'r') as file:
    for line in file:
        match = re.search(pattern, line)
        if match:
            print(line.rstrip('\n'), match.group(1))
        else:
            print(line.rstrip('\n'))

If as OP seems to imply the pattern to look for always includes a first letter, just make the pattern: pattern = re.compile(r'[a-zA-Z](\d{2})').

The corner(?) case of more than one match per line (as remarked by @terdon) is also easily dealt with using a new match expression match = re.finditer(pattern, line) and a modified print instruction.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .