Török Edwin
2013-01-24 15:46:17 UTC
Hi,
I am using the curl multi interface and I see a lot of timeouts on slowish links (all fine on fast links of course).
I have concluded that CURLOPT_CONNECTTIMEOUT sets not only the timeout for connect(), but also the timeout
for starting to receive a reply for a pipelined request.
Due to how pipelining works this means that all N-1 pipelined requests must finish during the CONNECTTIMEOUT
of the Nth request, otherwise all N (and later) requests are considered as timed out.
Is this intended behaviour?
I would've expected CURLOPT_CONNECTTIMEOUT to set only the timeout for connect(). For pipelined connections
as long as _any_ easy handle is receiving data on that connection I don't want the 'connect' timeout
of other pipelined requests to trigger. Is there a way to do that without modifying curl?
Alternatively I could queue up all requests that would go to the same host and only add the easy handle
to a multi handle when previous request is finished, but it'd be nice if curl could do this itself.
Note: I'm actually interested in persistent connections only, not necesarely pipelining (that is just a nice addition),
but in the multi interface I don't see a way to request that only: it is either both (MOPT_PIPELINING on), or neith
(MOPT_PIPELINING off, separate connections).
Here is how I encountered this issue:
rc = curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 60);
...
rc2 = curl_multi_setopt(x->multi, CURLMOPT_PIPELINING, 1);
....
If I add lots of easy handles to the multi handle (lets say 10 requests to same host each with a 1MB reply)
then the last requests tends to timeout with an error on a slowish link:
Timeout was reached, Operation timed out after 60009 milliseconds with 0 out of 0 bytes received
At first I thought there is some other timeout option that I'm not setting, but setting this didn't help
(its the default value anyway):
rc = curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 0);
Changing CURLOPT_CONNECCTIMEOUT to 10s will cause all the multi pipelined transfers to time out after ~10000 ms, so that
supports my original idea that CONNECTTIMEOUT influences timeout for pipelined requests.
Best regards,
--Edwin
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html
I am using the curl multi interface and I see a lot of timeouts on slowish links (all fine on fast links of course).
I have concluded that CURLOPT_CONNECTTIMEOUT sets not only the timeout for connect(), but also the timeout
for starting to receive a reply for a pipelined request.
Due to how pipelining works this means that all N-1 pipelined requests must finish during the CONNECTTIMEOUT
of the Nth request, otherwise all N (and later) requests are considered as timed out.
Is this intended behaviour?
I would've expected CURLOPT_CONNECTTIMEOUT to set only the timeout for connect(). For pipelined connections
as long as _any_ easy handle is receiving data on that connection I don't want the 'connect' timeout
of other pipelined requests to trigger. Is there a way to do that without modifying curl?
Alternatively I could queue up all requests that would go to the same host and only add the easy handle
to a multi handle when previous request is finished, but it'd be nice if curl could do this itself.
Note: I'm actually interested in persistent connections only, not necesarely pipelining (that is just a nice addition),
but in the multi interface I don't see a way to request that only: it is either both (MOPT_PIPELINING on), or neith
(MOPT_PIPELINING off, separate connections).
Here is how I encountered this issue:
rc = curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 60);
...
rc2 = curl_multi_setopt(x->multi, CURLMOPT_PIPELINING, 1);
....
If I add lots of easy handles to the multi handle (lets say 10 requests to same host each with a 1MB reply)
then the last requests tends to timeout with an error on a slowish link:
Timeout was reached, Operation timed out after 60009 milliseconds with 0 out of 0 bytes received
At first I thought there is some other timeout option that I'm not setting, but setting this didn't help
(its the default value anyway):
rc = curl_easy_setopt(curl, CURLOPT_CONNECTTIMEOUT, 0);
Changing CURLOPT_CONNECCTIMEOUT to 10s will cause all the multi pipelined transfers to time out after ~10000 ms, so that
supports my original idea that CONNECTTIMEOUT influences timeout for pipelined requests.
Best regards,
--Edwin
-------------------------------------------------------------------
List admin: http://cool.haxx.se/list/listinfo/curl-library
Etiquette: http://curl.haxx.se/mail/etiquette.html