Doubt Regarding the use of HTTP/2

[Vipin P R via curl-library]

Hi,
This is my first mail to the Libcurl Community. As part of it , I
request a small clarification in the use of the Libcurl multi interface.
When I set the Max connections to 3, and enable HTTP/2 with the multi
interface, I still see that all requests are being pipelined on the very
first connection.

In this specific example, I have given number of connections as 3 and
num of requests as 15. These are the parameters I specified.

634 curl_easy_setopt(handle,
635 CURLOPT_HTTP_VERSION , CURL_HTTP_VERSION_2_0);

962 if (res_cfg->ishttp2)
963 curl_multi_setopt(session->cm_handle,
964 CURLMOPT_PIPELINING, CURLPIPE_MULTIPLEX);

977 curl_multi_setopt(session->cm_handle, CURLMOPT_MAXCONNECTS,
978 session->num_connections);
979 curl_multi_setopt(session->cm_handle,
CURLMOPT_MAX_TOTAL_CONNECTIONS,
980 session->num_connections);
981 curl_multi_setopt(session->cm_handle,
CURLMOPT_MAX_HOST_CONNECTIONS,
982 session->num_connections);

session->num_connections = 3.

I also add 15 easy handles to the HTTP/2 multi handle. But from my curl
Verbose output, I can see that most of the requests are pipelined on a
single connection. Am I missing something here. Please help !

<IIRC earlier , when I used Libcurl with 15 requests, 3 connections with
the HTTP/2 enabled multihandle , I could see that it would be around 4 - 5
requests pipelined on a single connection.>

***@vipin-dev2:~/te_dp_bugs/avi-dev/test/TrafficEngine/te_dp$ tp
te_dp listening_on:0.0.0.0, port:43184 with pid:7129
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait
* No connections available.
* Trying 139.162.123.134...
* TCP_NODELAY set
* Connected to nghttp2.org (139.162.123.134) port 80 (#0)

GET / HTTP/1.1

Host: nghttp2.org
Accept: */*
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: AAMAAABkAARAAAAAAAIAAAAA

< HTTP/1.1 101 Switching Protocols
< Connection: Upgrade
< Upgrade: h2c
* Received 101
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
len=27
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (1)
* We can reuse, but we want a new connection anyway
* Hostname nghttp2.org was found in DNS cache
* Trying 139.162.123.134...
* TCP_NODELAY set
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.00144
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
* Connection #0 to host nghttp2.org left intact
* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Re-using existing connection! (#0) with host nghttp2.org
* Connected to nghttp2.org (139.162.123.134) port 80 (#0)
* Using Stream ID: 3 (easy handle 0x16e1ae0)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (1)
* We can reuse, but we want a new connection anyway
* Hostname nghttp2.org was found in DNS cache
* Trying 139.162.123.134...
* TCP_NODELAY set
* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (1)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 5 (easy handle 0x16f40c0)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (2)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 7 (easy handle 0x16fd3b0)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (3)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 9 (easy handle 0x17066a0)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (4)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: b (easy handle 0x170f990)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (5)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: d (easy handle 0x1718c80)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (6)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: f (easy handle 0x1721f70)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (7)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 11 (easy handle 0x172b260)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (8)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 13 (easy handle 0x1734550)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (9)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 15 (easy handle 0x173d840)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (10)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 17 (easy handle 0x1746b30)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Found bundle for host nghttp2.org: 0x16bbbd0 [can multiplex]
* Conn: 0 (0x1759110) Receive pipe weight: (-1/0), penalized: FALSE
* Multiplexed connection found!
* Found connection 0, with requests in the pipe (11)
* Re-using existing connection! (#0) with host nghttp2.org
* Using Stream ID: 19 (easy handle 0x174fe20)

GET / HTTP/2

Host: nghttp2.org
Accept: */*

* Connected to nghttp2.org (139.162.123.134) port 80 (#1)

GET / HTTP/1.1

Host: nghttp2.org
Accept: */*
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: AAMAAABkAARAAAAAAAIAAAAA

* Connected to nghttp2.org (139.162.123.134) port 80 (#2)

GET / HTTP/1.1

Host: nghttp2.org
Accept: */*
Connection: Upgrade, HTTP2-Settings
Upgrade: h2c
HTTP2-Settings: AAMAAABkAARAAAAAAAIAAAAA

< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001984
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001813
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001757
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001696
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001651
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001602
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001443
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/1.1 101 Switching Protocols
< Connection: Upgrade
< Upgrade: h2c
* Received 101
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
len=27
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001054
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
* Connection #1 to host nghttp2.org left intact
< HTTP/1.1 101 Switching Protocols
< Connection: Upgrade
< Upgrade: h2c
* Received 101
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade:
len=27
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.000543
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
* Connection #2 to host nghttp2.org left intact
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001678
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001557
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.00152
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.00145
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
< HTTP/2 200
< date: Fri, 21 Sep 2018 06:04:34 GMT
< content-type: text/html
< last-modified: Sun, 02 Sep 2018 06:50:48 GMT
< etag: "5b8b8848-19d8"
< accept-ranges: bytes
< content-length: 6616
< x-backend-header-rtt: 0.001839
< server: nghttpx
< via: 2 nghttpx
< x-frame-options: SAMEORIGIN
< x-xss-protection: 1; mode=block
< x-content-type-options: nosniff
<
* Connection #0 to host nghttp2.org left intact
^***@vipin-dev2:~/te_dp_bugs/avi-dev/test/TrafficEngine/te_dp$

[Daniel Stenberg via curl-library]

Post by Vipin P R via curl-library
When I set the Max connections to 3, and enable HTTP/2 with the multi
interface, I still see that all requests are being pipelined on the very
first connection.

When libcurl is about to start a new transfer there are several factors
involved that together are used to make the decision between an existing
connection or start a new one. It will generally prioritise using an existing
one if it (without delay) can, since that's typically the faster way and sort
of a lot of the point with HTTP/2.

Post by Vipin P R via curl-library
I have given number of connections as 3 and num of requests as 15.

"Maximum number of connections" doesn't mean that it will spread out over
three automatically. It just means that it won't use more than 3.

Post by Vipin P R via curl-library
I also add 15 easy handles to the HTTP/2 multi handle. But from my curl
Verbose output, I can see that most of the requests are pipelined on a
single connection. Am I missing something here. Please help !

That works exactly as intended. Except if you added all those handles at once
without CURLOPT_PIPEWAIT set, as then it is likely to create more connections
since then it deems creating a new connection is likely to be faster than
waiting for another transfer's connection to get ready.

Post by Vipin P R via curl-library
* Found bundle for host nghttp2.org: 0x16bbbd0 [serially]
* Server doesn't support multi-use yet, wait

This implies CURLOPT_PIPEWAIT is set. You asked for it to wait for a
connection!

[Vipin P R via curl-library]

Noted. Didn;t notice that, forwarding to the mailing list

Thanks,



Vipin P R <***@gmail.com>
2:36 PM (23 minutes ago)
to daniel
Hi,
Thanks for the above info. Is there some way to control the
distribution of requests per connection, other than using separate multi
handles each set to use 1 connection at max.
I mean is there some way this can be accomplished using libcurl, like
any configuration parameters I need to use, please help me



Hi,
This is what I am trying to achieve.

1. Have a certain number of easy handles, let's say 15.

2. Adding them to the multi handle sequentially with no delay.

3. Setting the max number of connections and the size of Connection
cache to 3.

4. Have 4-5 easy handles multiplexed on Conn #0, similarly to Conn #1
and to Conn #2.

How do we achieve this control over the distribution of requests among
connections.

This is what I am trying to achieve.

You are emailing this to me personally, not the mailing list.
If you send this email to the mailing list instead, I will respond.
--
/ daniel.haxx.se

[Daniel Stenberg via curl-library]

Post by Vipin P R via curl-library
Thanks for the above info. Is there some way to control the
distribution of requests per connection, other than using separate multi
handles each set to use 1 connection at max.
I mean is there some way this can be accomplished using libcurl, like
any configuration parameters I need to use, please help me

No, you can't affect this much.

In fact, even creating an API for allowing an application to have that sort of
control would be a tricky and probably require a quite complex API. We simply
never (before) had users asking for such abilities so we've not implemented
any.