Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

What is a quick and easy way to validate if the pattern "301 domaname.com 200" is present within BASH when spread across multiple lines of output?

I was using something like the following:

 awk '/301|domain.com|200/'
 pcregrep -M '301|domain\.com|200'

but the order doesn't matter. I'm not sure how to state it so it does. What I thought would work apparently isn't catching the line endings.

pcregrep -M '301.*domain\.com.*200'

Background:

I'm building a small mod_rewrite server and I need a way to monitor what domains are getting redirected to what destinations.

As a result I'm assembling a small Nagios check script that will handle this for me.

What I have so far is the following:

curl qa-mod-rewrite.domain.com -i -I -L

HTTP/1.1 301 Moved Permanently
Via: 1.1 GREGORY
Connection: close
Proxy-Connection: close
Date: Thu, 14 Nov 2013 16:35:19 GMT
Location: http://qa.domain.com/
Content-Type: text/html; charset=iso-8859-1
Server: Apache/2.2.3 (CentOS)

HTTP/1.1 200 OK
Via: 1.1 GREGORY
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 56772
Expires: -1
Date: Thu, 14 Nov 2013 16:35:03 GMT
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
Cache-Control: no-cache, no-store
Pragma: no-cache
X-AspNet-Version: 2.0.50727
Set-Cookie: cfcausa.qa#sc_wede=1; path=/
Set-Cookie: ASP.NET_SessionId=i4z1c4ahqoiw13552z413hbs; path=/; HttpOnly
X-Powered-By: ASP.NET
share|improve this question

5 Answers 5

up vote 4 down vote accepted

There's scope for false positive there as 301.*domain\.com.*200 would match for instance on:

HTTP/1.1 404 Not found
Content-Length: 3010
X-Pingback: http://blah.domain.com/xmlrpc
Last-Modified: Thu, 14 Nov 2009 19:27:05 GMT

You could be a bit more thorough and write it for instance:

curl -sIL http://qa-mod-rewrite.domain.com |
  tr -d '\r' |
  awk -v RS= '
    NR == 1 && $2 == "301" && /\nLocation: [^\n]*domain\.com/ {redir=1}
    $2 == "200" {end=1}
    END {exit !(redir*end)}'

With variable data:

url=$1
EXPECTED_REDIRECTION=$2
EXPECTED_REDIRECTION_CODE=$3
EXPECTED_TERMINAL_CODE=$4
export EXPECTED_REDIRECTION EXPECTED_REDIRECTION_CODE EXPECTED_TERMINAL_CODE

curl -sIL "$url" |
  tr -d '\r' |
  awk -v RS= '
    BEGIN {
      re = ENVIRON["EXPECTED_REDIRECTION"]
      gsub(/[][^.$+?\\()]/, "\\&",re)
      re = "\nLocation: [^\n]*" re
    }
    NR == 1 && $2 == ENVIRON["EXPECTED_REDIRECTION_CODE"] && $0 ~ re {redir=1}
    $2 == $ENVIRON["EXPECTED_TERMINAL_CODE"] {end=1}
    END {exit !(redir*end)}'
share|improve this answer
    
That is insanely handy. Thanks. Is there a ready way that I can pass in a command line parameter I'm picking up from the script ($2) for the spot domain\.com is at? I always get messed up with Awk and the single quotes. –  Tim Brigham Nov 14 '13 at 22:23

You were close with pcregrep. You need to explicitly include \n along with . in your pattern

pcregrep -M '301(.|\n)*domain\.com(.|\n)*200'
share|improve this answer
    
See also: pcregrep -M '(?s)301.*domain\.com.*200' –  Stéphane Chazelas Nov 14 '13 at 21:15

You could also do this directly in Perl and paragraph mode (-000):

perl -000ne 'print if /301/' file

This gives the expected output when run on the exact file you posted but will probably need to be tweaked for your actual input.

share|improve this answer
    
That doesn't answer the question though. And your probably want slurp mode (-0777) instead of paragraph mode here. –  Stéphane Chazelas Nov 14 '13 at 20:53

Similar to perl's slurp mode sed can read the entire file or string into the hold buffer ("hold space") so that the file or string contents can be treated as a single line in "pattern space" (see "sed and Multi-Line Search and Replace").

str='
HTTP/1.1 301 Moved Permanently
Via: 1.1 GREGORY
Connection: close
Proxy-Connection: close
Date: Thu, 14 Nov 2013 16:35:19 GMT
Location: http://qa.domain.com/
Content-Type: text/html; charset=iso-8859-1
Server: Apache/2.2.3 (CentOS)

HTTP/1.1 200 OK
Via: 1.1 GREGORY
Connection: Keep-Alive
Proxy-Connection: Keep-Alive
Content-Length: 56772
Expires: -1
Date: Thu, 14 Nov 2013 16:35:03 GMT
Content-Type: text/html; charset=utf-8
Server: Microsoft-IIS/7.5
Cache-Control: no-cache, no-store
Pragma: no-cache
X-AspNet-Version: 2.0.50727
Set-Cookie: cfcausa.qa#sc_wede=1; path=/
Set-Cookie: ASP.NET_SessionId=i4z1c4ahqoiw13552z413hbs; path=/; HttpOnly
X-Powered-By: ASP.NET
'

# GNU sed
# using [^\n] (to match any character except the newline character \n)
# cf. http://austinmatzko.com/2008/04/26/sed-multi-line-search-and-replace/
echo "$str" | 
  tr -d '\r' |
  sed -n -e '
     # if the first line copy the pattern to the hold buffer
     1h
     # if not the first line then append the pattern to the hold buffer
     1!H
     # if the last line then ...
     $ {
        # copy from the hold to the pattern buffer
        g
        # do the search and print all matching lines if there are the specified three matches in the given order
        /^.*\n\(HTTP\/[^\n]* 301 [^\n]*\).*\(\nLocation:[^\n]*domain\.com[^\n]*\).*\(\nHTTP\/[^\n]* 200 [^\n]*\).*$/s//\1\2\3/p;
     }'

# output:
# HTTP/1.1 301 Moved Permanently
# Location: http://qa.domain.com/
# HTTP/1.1 200 OK


# FreeBSD sed
# using [^[:cntrl:]] (to match any character except control characters, especially the newline character \n)
# cf. http://en.wikipedia.org/wiki/Regular_expression#Character_classes
echo "$str" | 
  tr -d '\r' |
  sed -n -e '1h;1!H;${;g; /^.*\n\(HTTP\/[^[:cntrl:]]* 301 [^[:cntrl:]]*\).*\(\nLocation:[^[:cntrl:]]*domain\.com[^[:cntrl:]]*\).*\(\nHTTP\/[^[:cntrl:]]* 200 [^[:cntrl:]]*\).*$/s//\1\2\3/p; }'


# output:
# HTTP/1.1 301 Moved Permanently
# Location: http://qa.domain.com/
# HTTP/1.1 200 OK
share|improve this answer

Thanks to all that responded. Another alternative I was working on was the following.

TMPFILE=$(mktemp)
curl $1 -i -I -L -s > $TMPFILE
cat $TMPFILE
echo ===================
pcregrep -M "1.1 301 Moved(.|\n)*Location: http(|s)://$2(.|\n)*1.1 200 OK" $TMPFILE > /dev/null 2>&1
STATUS=$?
echo $STATUS
rm $TMPFILE
exit $STATUS
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.