Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

Suppose there is some text from a file:

(bookmarks
("Chapter 1 Introduction 1" "#1"
("1.1 Problem Statement and Basic Definitions 23" "#2")
("Exercises 31" "#30")
("Notes and References 42" "#34"))
)

I want to add 11 to each number followed by a " in each line if there is one, ie

(bookmarks
("Chapter 1 Introduction 12" "#12"
("1.1 Problem Statement and Basic Definitions 34" "#13")
("Exercises 42" "#41")
("Notes and References 53" "#45"))
)

Here is my solution by using GNU AWK and regex:

awk -F'#' 'NF>1{gsub(/"(\d+)\""/, "\1+11\"")}'

i.e., I want to replace (\d+)\" with \1+10\", where \1 is the group representing (\d+). But it doesn't work. How can I make it work?

If gawk is not the best solution, what else can be used?

share|improve this question

migrated from stackoverflow.com Nov 22 '11 at 8:29

This question came from our site for professional and enthusiast programmers.

    
Sorry about duplication. But I first asked on stackoverflow, and got no satisfactory answer, so I flagged for migration. But it didn't happen for a while, so I didn't expect it to happen and then asked on Unix.SE. –  Tim Nov 22 '11 at 13:36

3 Answers 3

up vote 10 down vote accepted

try this (gawk is needed)

awk '{a=gensub(/.*#([0-9]+)(\").*/,"\\1","g",$0);if(a~/[0-9]+/) {gsub(/[0-9]+\"/,a+11"\"",$0);}print $0}' YourFile

test with your example:

kent$  echo '(bookmarks
("Chapter 1 Introduction 1" "#1"
("1.1 Problem Statement and Basic Definitions 2" "#2")
("Exercises 30" "#30")
("Notes and References 34" "#34"))
)
'|awk '{a=gensub(/.*#([0-9]+)(\").*/,"\\1","g",$0);if(a~/[0-9]+/) {gsub(/[0-9]+\"/,a+11"\"",$0);}print $0}'   
(bookmarks
("Chapter 1 Introduction 12" "#12"
("1.1 Problem Statement and Basic Definitions 13" "#13")
("Exercises 41" "#41")
("Notes and References 45" "#45"))
)

note that this command won't work if the two numbers e.g. 1" and "#1" are different. or there are more numbers in same line with this pattern. e.g. 23" ...32"..."#123" in one line.

UPDATE

since @Tim (OP) said the number followed by " in same line could be different, I did some changes on my previous solution, and make it work for your new example.

btw, from the example I feel that it could be a table of content structure, I don't see how come the two numbers could be different. first would be the printed page number, and 2nd with # would be the page index. am I right?

anyway, you know your reqirement best. now the new solution, still with gawk (I break the command into lines to make it easier to read):

awk 'BEGIN{FS=OFS="\" \"#"}{if(NF<2){print;next;}
        a=gensub(/.* ([0-9]+)$/,"\\1","g",$1);
        b=gensub(/([0-9]+)\"/,"\\1","g",$2); 
        gsub(/[0-9]+$/,a+11,$1);
        gsub(/^[0-9]+/,b+11,$2);
        print $1,$2
}' yourFile

test with your new example:

kent$  echo '(bookmarks
("Chapter 1 Introduction 1" "#1"
("1.1 Problem Statement and Basic Definitions 23" "#2")
("Exercises 31" "#30")
("Notes and References 42" "#34"))
)
'|awk 'BEGIN{FS=OFS="\" \"#"}{if(NF<2){print;next;}
        a=gensub(/.* ([0-9]+)$/,"\\1","g",$1);
        b=gensub(/([0-9]+)\"/,"\\1","g",$2); 
        gsub(/[0-9]+$/,a+11,$1);
        gsub(/^[0-9]+/,b+11,$2);
        print $1,$2
}'                        
(bookmarks
("Chapter 1 Introduction 12" "#12"
("1.1 Problem Statement and Basic Definitions 34" "#13")
("Exercises 42" "#41")
("Notes and References 53" "#45"))
)

EIDT2 based on @Tim 's comment

(1) does FS=OFS="\" \"#" mean the separator of field in both input and output is double quote, space, double quote and #? why specify double quote twice?

you are right for the separator in both input and output part. it defined separator as:

" "#

there are two double quotes, because it is easier to catch the two numbers you want(based on your example input).

(2) in /.* ([0-9]+)$/, does $ mean the end of the string?

exactly!

(3) in the third argument of gensub(), what is the difference between "g" and "G"? there is no difference between G and g. check this out:

gensub(regexp, replacement, how [, target]) #
    Search the target string target for matches of the regular expression regexp. If how is a string beginning with ‘g’ or ‘G’ (short for “global”), then replace all matches of regexp with replacement.

this is from http://www.gnu.org/s/gawk/manual/html_node/String-Functions.html. you can read to get detailed usage of gensub.

share|improve this answer
    
Thanks! I wonder how to make it work if the two numbers e.g. 1" and "#1" are different? –  Tim Nov 20 '11 at 0:03
    
this answer works for your current reqirement/example. if the requirement is changed, maybe you could edit the question, and give a better example. and from your code awk -F'#', it seems that you only want to do the change on the part after the '#' ? –  Kent Nov 20 '11 at 0:10
    
Thanks for your suggestion. I just modified my example so that the two numbers are not the same. –  Tim Nov 20 '11 at 0:12
    
@Tim see my updated answer, for your new example. –  Kent Nov 20 '11 at 0:28
    
Thanks! Some questions: (1) does FS=OFS="\" \"#" mean the separator of field in both input and output is double quote, space, double quote and #? why specify double quote twice? (2) in /.* ([0-9]+)$/, does $ mean the end of the string? (3) in the third argument of gensub(), what is the difference between "g" and "G"? –  Tim Nov 20 '11 at 20:56

awk can do it, but it isn't direct, even using backreferencing.
GNU awk has (partial) backreferecing, in the form of gensub.

Instances of 123" are temporarily wrapped in \x01 and \x02 to mark them as unmodified (for sub(). co

Or you could just step through the loop changing candidates as you go, in which case, the backreferencing and "brackets" aren't needed; but keeping track of the character index is needed.

awk '{$0=gensub(/([0-9]+)\"/, "\x01\\1\"\x02", "g", $0 )
      while ( match($0, /\x01[0-9]+\"\x02/) ) {
        temp=substr( $0, RSTART, RLENGTH )
        numb=substr( temp, 2, RLENGTH-3 ) + 11
        sub( /\x01[0-9]+\"\x02/, numb "\"" ) 
      } print }'

Here is another way, using gensub and array split and \x01 as a field delimiter (for split).. \x02 marks an array element as a candidate for the arithmetic addition.

awk 'BEGIN{ ORS="" } {
     $0=gensub(/([0-9]+)\"/, "\x01\x02\\1\x01\"", "g", $0 )
     split( $0, a, "\x01" )
     for (i=0; i<length(a); i++) { 
       if( substr(a[i],1,1)=="\x02" ) { a[i]=substr(a[i],2) + 11 }
       print a[i]
     } print "\n" }'
share|improve this answer
    
Thanks! In your first code, (1) what does "\x01\\1\"\x02" mean? I still don't understand \x01 and \x02. (2) how different is the return $0 by gensub and the $0 as the last argument to gensub? –  Tim Nov 20 '11 at 21:04
    
@Tim. The hex values \x01 and \x02 are used as substitution markers. These values are highly unlikely to be in any normal text file, so they are equally "highly" safe to use (ie. not encounter a clash with pre-existing ones).. They are just temporary labels.. Re $0=gensub(... $0).. see this link String-Manipulation Functions, but in summary: It (gensub) returns the modified string as the result of the function and the original target string is not changed.... The $0= simply modifies the original target.. –  Peter.O Nov 23 '11 at 11:37

Unlike just about every tool that provides regexp substitutions, awk does not allow backreferences such as \1 in replacement text. [GNU awk] gives access to matched groups if you use the match function, but not with ~ or sub or gsub.

Note also that even if \1 was supported, your snippet would append the string +11, not perform a numerical computation. Also, your regexp isn't quite right, you're matching things like "42"" and not "#42".

Here's an awk solution (warning, untested). It only performs a single replacement per line.

awk '
  match($0, /"#[0-9]+"/) {
    n = substr($0, RSTART+2, RLENGTH-3) + 11;
    $0 = substr($0, 1, RSTART+1) n substr($0, RSTART+RLENGTH-1)
  }
  1 {print}'

It would be simpler in Perl.

perl -pe 's/(?<="#)[0-9]+(?=")/$1+11/e'
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.