1

We have two httpd servers running on RHEL 6 behind a hardware load balancer

We are getting timeout errors in our httpd error log.

[Fri Feb 19 15:06:49.823372 2016] [proxy_http:error] [pid 15760] (70007)The timeout specified has expired: [client 172.X.X.187:48453] AH01095: prefetch request body failed to 172.X.X.210:21080 (host-x15) from 172.X.X.187 (), referer: http://www.example.com.au/secure/Book?cb=30938956188 [Fri Feb 19 15:07:37.113599 2016] [proxy_http:error] [pid 16312] (70007)The timeout specified has expired: [client 172.X.X.187:51600] AH01095: prefetch request body failed to 172.X.X.209:21080 (host-x14) from 172.X.X.187 (), referer: http://www.example.com.au/secure/Book?cb=182574845707719466206

In our case , apache is working as a load balancer using mod_balancer_mananger. There were tomcat servers behind the apache load balancer.

The worry for me is the timeout errors I am seeing . Is my apache is dropping connections by any chance. In the above scenario , 172.X.X.187 is the hardware loadbalancer ip . 172.X.X.210/209 are the application server IPs which are behind apache load balancer.

Checking the system , I can see lots of TIME_WAIT connections from the load balancer IP. Sometimes it is more than 600 connections.

$ sudo netstat -anp | grep -i time | grep 172.X.X.187  | wc -l
  621

$ sudo netstat -anp | grep -i estab  | wc -l
  681

Is our connection limit is getting exhausted because of the default TCP kernel parameters as per this https://stackoverflow.com/questions/410616/increasing-the-maximum-number-of-tcp-ip-connections-in-linux

$ cat /proc/sys/net/ipv4/tcp_fin_timeout
 60
$ cat /proc/sys/net/ipv4/ip_local_port_range
32768   61000
$ cat /proc/sys/net/ipv4/tcp_max_syn_backlog
4096

apache load balancer configuration is as follows

BalancerMember host-x14:21080 min=1 max=1000 loadfactor=1  retry=1 timeout=240 route=host-x4
BalancerMember host-x15:21080 min=1 max=1000 loadfactor=1  retry=1 timeout=240 route=host-x5

Please suggest. If you need further information , please let me know .

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.