2

I have a VM running CentOS 6. On this Linux box, I run 4 curl download concurrently.

In some case, one of the curl download hangs. ps shows that curl process is still there. And netstat shows the connection's RECV_Q is very long:

tcp   878104      0 10.161.238.20:46130         10.162.220.12:80            ESTABLISHED 

The curl version 7.38.0.

Any idea? Why curl hangs? If increasing TCP buffer would help?

4
  • It seems you are not too "rapid" to get data from Recv_q. Could you please use the --trace option to see what it gets out? It seems your host is not able to get data from the socket. Curl parallel execution are for the same data or are they different calls ?
    – Echoes_86
    Commented Sep 19, 2016 at 14:26
  • 3
    Could be any number of reasons, but my first guess would be PMTU fun caused by somebody blocking ICMPs between you and the server. Commented Sep 19, 2016 at 14:27
  • @SatoKatsura can you please give more details? How to resolve this issue?
    – TieDad
    Commented Sep 20, 2016 at 10:05
  • If you don't know what PMTU discovery is and what that has to do with ICMPs I'm afraid I don't have a quick solution for you. Sorry about that. Commented Sep 20, 2016 at 18:51

2 Answers 2

1

Large and static receive queue indicates data has been received on the socket but curl is not reading it.

There's no reason increasing TCP buffers would help.

It is a pathological case here. For an HTTP request, curl is meant to send a request (or possibly requests if chunked) and read the response. If it's not reading the response, it's either it's suspended or its busy doing something else.

That could be for instance because it's blocked writing the response to its standard output, for instance because it's a full pipe, or it's writing it to a file on an unreliable network filesystem.

ps -o state,wchan,args -C curl

Or

pgrep -x curl | xargs -IP cat /proc/P/stack

Might give you a clue as to what curl is doing that prevents it from reading the response (also /proc/$pid/task/*/stack for multithreaded applications).

0

You could try running curl with strace and/or try with the latest version of curl (version 7.50.3).

2
  • Coz this is a production environment, I cannot run curl with strace. And since this issue is hard to reproduce, it's hard to debug with single test.
    – TieDad
    Commented Sep 20, 2016 at 10:07
  • 1
    You can statically compile strace and copy it over to the production environment, unless there are weird company policies in place.
    – Alexander
    Commented Sep 20, 2016 at 11:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .