I have a file which has many random lines like

aaa bbb
ccc ddd
eee mark: 98 fff
ggg ggg jjjj iii
jjj kkkk

I want to use awk AND only gensub to match the number "98" above. So far I have this code below, I think it does not work cause I need to make gensub treat "\n" as any other character.

cat file.txt | awk 'printf(gensub(/^.*mark: ([0-9]+).*$/,"\\1","g"))}'

I need the output of the code above to be only "98". How do I do that?


even when I use the s or m modifier it does not work as it should cause as far as I know the "s" modifier should make regex treat . as any character including \n.


You seem to think that awk treats its input as a multiline string. It doesn't. When you run an awk script on a file, the script is applied to each line of the file separately. So, your gensub was run once per line. You can actually do what you want with awk but it really isn't the best tool for the job.

As far as I can tell, you have a large file and only want to print a number that comes after mark: and whitespace. If so, all of these approaches are simpler than fooling around with gensub:

  1. Use grep with Perl Compatible Regular Expressions (-P)

    $ grep -oP 'mark:\s*\K\d+' file 

    The -o makes grep only print the matching portion of the line. The \K is a PCRE construct which means "ignore anything matched before this point".

  2. sed

    $ sed -n 's/.*mark:\s*\([0-9]\+\).*/\1/p' file

    The -n suppresses normal output. The p at the end makes sed print only if the substitution was successful. The regex itself captures a string of numbers following mark: and 0 or more whitespace characters and replaces the whole line with what was captured.

  3. Perl

    $ perl -ne 'print if s/.*mark:\s*(\d+).*/$1/' file

    The -n tells perl to read an input file line by line and apply the script given by -e. The script will print any lines where the substitution was successful.

If you really, really want to use gensub, you could do something like:

$ awk '/mark:/{print gensub(/.*mark:\s*([0-9]+).*/,"\\1","g")}' file

Personally, I would do it this way in awk:

$ awk '/mark:/{gsub(/[^0-9]/,"");print}' file

Since you seemed to be trying to get awk to receive multiline input, this is how you can do that (assuming there are no NULL characters in your file):

$ awk '{print(gensub(/^.*mark: ([0-9]+).*$/,"\\1","g"))}' RS='\0' file

The RS='\0' sets the input record separator (that's what defines a "line" for awk) to \0. Since there are no such characters in your file, this results in awk reading the whole thing at once.


The smallest change to get it working will be:

cat file | awk '/mark:/{printf( "%s\n",gensub(/^.*mark: ([0-9]+).*$/,"\\1","g"))}'

The /mark:/ is to select a line that contain "mark:".
But, then, why is a printf needed? This will also work:

cat file | awk '/mark:/{print(gensub(/^.*mark: ([0-9]+).*$/,"\\1","g"))}'

But that would be a "useless use of cat", as awk could directly read from a file:

awk '/mark:/{print(gensub(/^.*mark: ([0-9]+).*$/,"\\1","g"))}' file


On user request: How to use the regex on file and string.

Well, with the rules you set: awk with only gensub is not possible.
Also, the idea of matching with .*mark: ([0-9]+).* to replace all of that with the match inside the parenthesis will mean that it is needed to match the whole file to extract a part. That is one reason why grep was created.

Just use:

grep -oP "mark: \K([0-9]+)" file


echo "$string" | grep -oP "mark: \K([0-9]+)"

And you will get the result.

  • Really good, really good! But why is it so impossible to run a regex on a multiline string without those nice "workarounds" you gave me? Is it at least possible to remove the "\n\r" or "\n" from the string before using gensub? Something like "remove_linebreak(cat file.txt) | awk 'printf(gensub(/^.*mark: ([0-9]+).*$/,"\\1","g"))}'" ? Your answer is good but I really need to be able to run this on a file but also on a string so I would appreciate a lot if your answer could be using the pipe. – Samul Nov 1 '15 at 20:33
  • The regex is running on multiline, "one line at a time". So you need to select both row and column. Select the line, the use the regex. – user79743 Nov 1 '15 at 21:32
  • @Samul because that's not a multiline string. Awk deals with single lines unless you tell it otherwise, so your script is run on each line separately. – terdon Nov 1 '15 at 21:49
  • @terdon so how do I tell "awk" to deal with multiple lines as if they where single lines? – Samul Nov 1 '15 at 22:04
  • @Samul see my answer. It is possible as I show in the end but there is usually a simpler approach. – terdon Nov 1 '15 at 22:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.