Adding OCSP Features to Curl

[VanL via curl-library]

(Disclaimer: Working for a client)



We currently have a requirement to perform certificate revocation checks on
all the HTTPS connections our device performs. Curl is a key library we use
to perform these HTTPS connections. Curl currently supports an option to
request and verify a stapled OCSP response on an HTTPS connection
(CURLOPT_SSL_VERIFYSTATUS). However, not all servers support stapled OCSP.
In that case, Curl will terminate the connection since it was not able to
verify the status of the server certificate.



We would like to expand Curl's support for OCSP verification beyond OCSP
stapling to include online OCSP. If a server does not return an OCSP stapled
response, Curl can be configured to request the OCSP status of the
certificate from the OCSP responder information listed in the certificate.
This would allow more servers to be supported by Curl if OCSP verification
is desired. This is the behavior of Firefox, which has led the browser
market in certificate revocation support.



We wanted to get commentary on this as a feature, as well as the proposed
flow, before creating an issue with a proposed pull request.



Current logic:



Request stapled OCSP response

-> Connect to server

-> Receive response?

Y -> Check signature and time. Valid?

Y -> Status OK?

Y -> CONTINUE CONNECT

N -> Abort

N -> Abort

N -> Abort



Proposed logic:

Request stapled OCSP response

-> Connect to server

-> Receive response?

Y -> Check signature and time. Valid?

(NEW) OPTION: OCSP_CACHE?

Y -> Store response in cache, continue

N -> Continue

Y -> Status OK?

Y -> CONTINUE CONNECT

N -> Abort

N -> Abort

N -> (NEW) Certificate is must staple?

Y -> Abort

N -> OPTION: OCSP_CACHE?

Y -> Previous response in cache?

Y -> Check signature and time. Valid?

Y -> CONTINUE CONNECT

N -> Abort

N -> OPTION: OCSP_ONLINE?

Y -> Request OCSP response. Received?

Y -> Check signature and time. Valid?

Y -> (Goto OCSP_CACHE Option) and Response
Status OK?

Y -> CONTINUE CONNECT

N -> Abort

N -> Abort

N -> Abort

N -> OPTION: OCSP_SOFT_FAIL?

Y -> CONTINUE CONNECT

N -> Abort

N -> (Goto OCSP_ONLINE Option)



We would like feedback on the desirability of adding this to curl as opposed
to putting it in a separate library. If there is interest, we can provide
suggested semantics for each of the options.



Thanks,
Van








OSPOCO





Van Lindberg

<https://ospo.co/> Open Source Program Office as a Service

<mailto:***@ospo.co> ***@ospo.co | 210.201.2024



This message and any attachments may contain privileged or confidential
information.
All applicable rights and privileges, including attorney-client privilege,
are reserved and
asserted. If you are not the intended recipient or believe that you have
received this
communication in error, please do not print, copy, share, or otherwise use
the information.

[Daniel Stenberg via curl-library]

Post by VanL via curl-library
We would like to expand Curl's support for OCSP verification beyond OCSP
stapling to include online OCSP.
We wanted to get commentary on this as a feature, as well as the proposed
flow, before creating an issue with a proposed pull request.

Awesome! I was previously in conversations with a customer about OCSP as well,
so I've had a chance to think about this already before.

If I would add something already now to your little "requirement spec" it
would probably be:

- cutomizable timeout for the OCSP response for when someone drops all the
packets to that server
- possibly an option of how to act on such time-outs (abort or continue)
- decent error reporting so that users can figure out what went wrong when
a connection is rejected

Challenges I think you might find in your road to landing this:

- tests - I propose you try to write this with functions that can be
unit-tested
- don't bury yourself in a single TLS library's ways of doing things so that
there's a chance for other TLS backends to follow suit later on (because I
just presume you'll aim at doing this work for a single TLS library first)
- needless to say perhaps, but everything needs to be done non-blocking

Qeustions:

- Are you thinking memory-only cache for the OCSP responses or on-disk too?
- For the OCSP caching, will you parse HTTP caching headers for that?
- Considered a CURL_LOCK_DATA_OCSP to share OCSP details between easy
handles?

Proposal:

- This is a rather big work. Try to split it up so that commits can be
reviewed without require super humans to do it.

Good luck!

[VanL via curl-library]

Post by Daniel Stenberg via curl-library
If I would add something already now to your little "requirement spec" it
- cutomizable timeout for the OCSP response for when someone drops all the
packets to that server
- possibly an option of how to act on such time-outs (abort or continue)
- decent error reporting so that users can figure out what went wrong when
a connection is rejected

We can take these as requirements.

Post by Daniel Stenberg via curl-library
- Are you thinking memory-only cache for the OCSP responses or on-disk too?

The cache could be implemented as a runtime cache or also a persistent
cache. One approach for the interface could be:
- CURLOPT_SSL_VERIFYCACHING. Pass 1 to enable runtime caching of OCSP
responses. Pass 0 to disable OCSP caching.
- CURLOPT_SSL_VERIFYCACHING_LOAD. Pass a data structure to inject a
starting cache.
- CURLINFO_SSL_VERIFYCACHING_GET. Get a data structure of the cache that
can be saved off.
This would allow the persistent cache to be implemented by the user
application.

Another approach for the interface could be:
- CURLOPT_SSL_VERIFYCACHING. Pass 1 to enable runtime caching of OCSP
responses. Pass 2 to enable persistent caching. Pass 0 to disable OCSP
caching.
- CURLOPT_SSL_VERIFYCACHING_DIR. Pass a directory to store the OCSP cache
into. Pass NULL to disable persistent caching.
This would have the OCSP persistent caching implemented within Curl.

Thoughts on whether one approach or another would be preferable for curl?

One other possibility would be full chain verification - just looping over
all the intermediate certificates as well.

Post by Daniel Stenberg via curl-library
- tests - I propose you try to write this with functions that can be
unit-tested
- don't bury yourself in a single TLS library's ways of doing things so that
there's a chance for other TLS backends to follow suit later on (because I
just presume you'll aim at doing this work for a single TLS library first)
- needless to say perhaps, but everything needs to be done non-blocking
- For the OCSP caching, will you parse HTTP caching headers for that?
- Considered a CURL_LOCK_DATA_OCSP to share OCSP details between easy
handles?

We haven't thought about these yet, but will. As a practical matter, we will
start with OpenSSL.

Post by Daniel Stenberg via curl-library
- This is a rather big work. Try to split it up so that commits can be
reviewed without require super humans to do it.

We will do our best!

Thanks,
Van

Van Lindberg
OSPOCO: Open Source Program Office as a Service
***@ospo.co | 210.201.2024

This message and any attachments may contain privileged or confidential
information.
All applicable rights and privileges, including attorney-client privilege,
are reserved and
asserted. If you are not the intended recipient or believe that you have
received this
communication in error, please do not print, copy, share, or otherwise use
the information.

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-l

[Daniel Stenberg via curl-library]

Post by VanL via curl-library
Thoughts on whether one approach or another would be preferable for curl?

I think starting with an in-memory only cache is a good start, then storing
that could be done in a later stage.

Post by VanL via curl-library
One other possibility would be full chain verification - just looping over
all the intermediate certificates as well.

I too could imagine such an option.