6

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output?

I was using something like the following:

 awk '/301|domain.com|200/'
 pcregrep -M '301|domain\.com|200'

but the order doesn't matter. I'm not sure how to state it so it does. What I thought would work apparently isn't catching the line endings.

pcregrep -M '301.*domain\.com.*200'

Background:

I'm building a small mod_rewrite server and I need a way to monitor what domains are getting redirected to what destinations.

As a result I'm assembling a small Nagios check script that will handle this for me.

What I have so far is the following:

curl qa-mod-rewrite.domain.com -i -I -L

HTTP/1.1 301 Moved Permanently
Via: 1.1 GREGORY
Connection: close
Proxy-Connection: close
Date: Thu, 14 Nov 2013 16:35:19 GMT
Location: http://qa.domain.com/
Content-Type: text/html; charset=iso-8859-1
Server: Apache/2.2.3 (CentOS)

HTTP/1.1 200 OK
Via: 1.1 GREGORY
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 56772
Expires: -1
Date: Thu, 14 Nov 2013 16:35:03 GMT
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
Cache-Control: no-cache, no-store
Pragma: no-cache
X-AspNet-Version: 2.0.50727
Set-Cookie: cfcausa.qa#sc_wede=1; path=/
Set-Cookie: ASP.NET_SessionId=i4z1c4ahqoiw13552z413hbs; path=/; HttpOnly
X-Powered-By: ASP.NET

5 Answers 5

6

There's scope for false positive there as 301.*domain\.com.*200 would match for instance on:

HTTP/1.1 404 Not found
Content-Length: 3010
X-Pingback: http://blah.domain.com/xmlrpc
Last-Modified: Thu, 14 Nov 2009 19:27:05 GMT

You could be a bit more thorough and write it for instance:

curl -sIL http://qa-mod-rewrite.domain.com |
  tr -d '\r' |
  awk -v RS= '
    NR == 1 && $2 == "301" && /\nLocation: [^\n]*domain\.com/ {redir=1}
    $2 == "200" {end=1}
    END {exit !(redir*end)}'

With variable data:

url=$1
EXPECTED_REDIRECTION=$2
EXPECTED_REDIRECTION_CODE=$3
EXPECTED_TERMINAL_CODE=$4
export EXPECTED_REDIRECTION EXPECTED_REDIRECTION_CODE EXPECTED_TERMINAL_CODE

curl -sIL "$url" |
  tr -d '\r' |
  awk -v RS= '
    BEGIN {
      re = ENVIRON["EXPECTED_REDIRECTION"]
      gsub(/[][^.$+?\\()]/, "\\&",re)
      re = "\nLocation: [^\n]*" re
    }
    NR == 1 && $2 == ENVIRON["EXPECTED_REDIRECTION_CODE"] && $0 ~ re {redir=1}
    $2 == $ENVIRON["EXPECTED_TERMINAL_CODE"] {end=1}
    END {exit !(redir*end)}'
1
  • That is insanely handy. Thanks. Is there a ready way that I can pass in a command line parameter I'm picking up from the script ($2) for the spot domain\.com is at? I always get messed up with Awk and the single quotes. Nov 14, 2013 at 22:23
3

You were close with pcregrep. You need to explicitly include \n along with . in your pattern

pcregrep -M '301(.|\n)*domain\.com(.|\n)*200'
1
  • See also: pcregrep -M '(?s)301.*domain\.com.*200' Nov 14, 2013 at 21:15
1

You could also do this directly in Perl and paragraph mode (-000):

perl -000ne 'print if /301/' file

This gives the expected output when run on the exact file you posted but will probably need to be tweaked for your actual input.

1
  • That doesn't answer the question though. And your probably want slurp mode (-0777) instead of paragraph mode here. Nov 14, 2013 at 20:53
0

Similar to perl's slurp mode sed can read the entire file or string into the hold buffer ("hold space") so that the file or string contents can be treated as a single line in "pattern space" (see "sed and Multi-Line Search and Replace").

str='
HTTP/1.1 301 Moved Permanently
Via: 1.1 GREGORY
Connection: close
Proxy-Connection: close
Date: Thu, 14 Nov 2013 16:35:19 GMT
Location: http://qa.domain.com/
Content-Type: text/html; charset=iso-8859-1
Server: Apache/2.2.3 (CentOS)

HTTP/1.1 200 OK
Via: 1.1 GREGORY
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 56772
Expires: -1
Date: Thu, 14 Nov 2013 16:35:03 GMT
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
Cache-Control: no-cache, no-store
Pragma: no-cache
X-AspNet-Version: 2.0.50727
Set-Cookie: cfcausa.qa#sc_wede=1; path=/
Set-Cookie: ASP.NET_SessionId=i4z1c4ahqoiw13552z413hbs; path=/; HttpOnly
X-Powered-By: ASP.NET
'

# GNU sed
# using [^\n] (to match any character except the newline character \n)
# cf. http://austinmatzko.com/2008/04/26/sed-multi-line-search-and-replace/
echo "$str" | 
  tr -d '\r' |
  sed -n -e '
     # if the first line copy the pattern to the hold buffer
     1h
     # if not the first line then append the pattern to the hold buffer
     1!H
     # if the last line then ...
     $ {
        # copy from the hold to the pattern buffer
        g
        # do the search and print all matching lines if there are the specified three matches in the given order
        /^.*\n\(HTTP\/[^\n]* 301 [^\n]*\).*\(\nLocation:[^\n]*domain\.com[^\n]*\).*\(\nHTTP\/[^\n]* 200 [^\n]*\).*$/s//\1\2\3/p;
     }'

# output:
# HTTP/1.1 301 Moved Permanently
# Location: http://qa.domain.com/
# HTTP/1.1 200 OK


# FreeBSD sed
# using [^[:cntrl:]] (to match any character except control characters, especially the newline character \n)
# cf. http://en.wikipedia.org/wiki/Regular_expression#Character_classes
echo "$str" | 
  tr -d '\r' |
  sed -n -e '1h;1!H;${;g; /^.*\n\(HTTP\/[^[:cntrl:]]* 301 [^[:cntrl:]]*\).*\(\nLocation:[^[:cntrl:]]*domain\.com[^[:cntrl:]]*\).*\(\nHTTP\/[^[:cntrl:]]* 200 [^[:cntrl:]]*\).*$/s//\1\2\3/p; }'


# output:
# HTTP/1.1 301 Moved Permanently
# Location: http://qa.domain.com/
# HTTP/1.1 200 OK
0

Thanks to all that responded. Another alternative I was working on was the following.

TMPFILE=$(mktemp)
curl $1 -i -I -L -s > $TMPFILE
cat $TMPFILE
echo ===================
pcregrep -M "1.1 301 Moved(.|\n)*Location: http(|s)://$2(.|\n)*1.1 200 OK" $TMPFILE > /dev/null 2>&1
STATUS=$?
echo $STATUS
rm $TMPFILE
exit $STATUS

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .