difficulties with SSL certs

[Dennis Clarke via curl-library]

This is likely a problem with OpenSSL and not curl but regardless I was
surprised to see :


europa$
europa$ curl -4 -L --url 'https://gitlab.com/' -o /dev/null
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
europa$

europa$ curl --version
curl 7.75.0 (x86_64-unknown-freebsd13.0) libcurl/7.75.0 OpenSSL/1.1.1k
zlib/1.2.11 libidn2/2.3.0 libssh2/1.9.0
Release-Date: 2021-02-03
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt
pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HTTPS-proxy IDN Largefile libz NTLM NTLM_WB
SSL TLS-SRP UnixSockets
europa$

So I looked into the location where the ssl certs "should" be given my
curl config :

$ ./configure --prefix=/opt/bw --disable-dependency-tracking \
--disable-silent-rules --without-gnu-ld --enable-shared \
--enable-static \
--with-libidn=/usr/local --with-libidn2=/usr/local \
--with-ssl=/opt/bw --with-ca-path=/opt/bw/ssl/certs \
--enable-tls-srp --with-libssh2

So I expect that the cacert.pem file at

https://curl.se/docs/caextract.html

would solve all my problems however :

europa$ ls -lapb /opt/bw/ssl/certs/
total 350
drwxr-xr-x 2 root wheel 3 Apr 8 02:35 ./
drwxr-xr-x 5 root wheel 9 Apr 7 00:14 ../
-rw-r--r-- 1 root wheel 208075 Jan 19 04:12 cacert.pem
europa$

This does not help at all and even OpenSSL seems confused.

So I try to debug this with the s_client sub-command and see something
obvious right away :


europa# /opt/bw/bin/openssl s_client -4 -showcerts -connect gitlab.com:443
CONNECTED(00000003)
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST
Network, CN = USERTrust RSA Certification Authority
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo
Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = gitlab.com
verify return:1
---
Certificate chain
0 s:CN = gitlab.com
i:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited,
CN = Sectigo RSA Domain Validation Secure Server CA
-----BEGIN CERTIFICATE-----
MIIGBzCCBO+gAwIBAgIQP8Jvo234xjti44c2dq2FnDANBgkqhkiG9w0BAQsFADCB
jzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
A1UEBxMHU2FsZm9yZDEYMBYGA1UEChMPU2VjdGlnbyBMaW1pdGVkMTcwNQYDVQQD
.
.
.

etc etc etc

The "unable to get local issuer certificate" is the problem.

However the default FreeBSD system openssl here seems to have no issues:

CONNECTED(00000004)
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST
Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo
Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = gitlab.com
verify return:1
---
Certificate chain
0 s:CN = gitlab.com
i:C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited,
CN = Sectigo RSA Domain Validation Secure Server CA
-----BEGIN CERTIFICATE-----
MIIGBzCCBO+gAwIBAgIQP8Jvo234xjti44c2dq2FnDANBgkqhkiG9w0BAQsFADCB
jzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
A1UEBxMHU2FsZm9yZDEYMBYGA1UEChMPU2VjdGlnbyBMaW1pdGVkMTcwNQYDVQQD
.
.
.
etc etc etc

So there must be a problem with the location of the ssl certs or the
contents of that cacert.pem file. So I dump out all the contents :

openssl crl2pkcs7 -nocrl -certfile /opt/bw/ssl/certs/cacert.pem | \
openssl pkcs7 -print_certs -text > /var/tmp/dclarke/pile



I think the data I need is in there :

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
5c:8b:99:c5:5a:94:c5:d2:71:56:de:cd:89:80:cc:26
Signature Algorithm: ecdsa-with-SHA384
Issuer: C=US, ST=New Jersey, L=Jersey City, O=The USERTRUST
Network, CN=USERTrust ECC Certification Authority
Validity
Not Before: Feb 1 00:00:00 2010 GMT
Not After : Jan 18 23:59:59 2038 GMT
Subject: C=US, ST=New Jersey, L=Jersey City, O=The USERTRUST
Network, CN=USERTrust ECC Certification Authority
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (384 bit)
.
.
.etc etc etc


Also this :

Certificate:
Data:
Version: 3 (0x2)
Serial Number:
01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d
Signature Algorithm: sha384WithRSAEncryption
Issuer: C=US, ST=New Jersey, L=Jersey City, O=The USERTRUST
Network, CN=USERTrust RSA Certification Authority
Validity
Not Before: Feb 1 00:00:00 2010 GMT
Not After : Jan 18 23:59:59 2038 GMT
Subject: C=US, ST=New Jersey, L=Jersey City, O=The USERTRUST
Network, CN=USERTrust RSA Certification Authority
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (4096 bit)


Those exact serial numbers also exist in the FreeBSD default cert pile :

europa$ ls -la /usr/share/certs/trusted/USERTrust_*
-r--r--r-- 1 root wheel 3238 Apr 4 03:34
/usr/share/certs/trusted/USERTrust_ECC_Certification_Authority.pem
-r--r--r-- 1 root wheel 7669 Apr 4 03:34
/usr/share/certs/trusted/USERTrust_RSA_Certification_Authority.pem
europa$

I checked them.

So I am baffled. What do I need in the SSL certs directory for both
OpenSSL and curl to be happy ? Is this an OpenSSL question and not
really a curl question ?

[Geoff Beier via curl-library]

Post by Dennis Clarke via curl-library
This is likely a problem with OpenSSL and not curl but regardless I was
europa$
europa$ curl -4 -L --url 'https://gitlab.com/' -o /dev/null
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0
curl: (60) SSL certificate problem: unable to get local issuer certificate

<snip>

Post by Dennis Clarke via curl-library
So I looked into the location where the ssl certs "should" be given my

<more snipped>

It's probably worth running with verbose output to see what is actually going on. For example:

https://paste.sr.ht/~tuxpup/54de3fe431cf595519490535a52b2d7421e18db6

Does the verbose output offer any additional clues?

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/

[Daniel Stenberg via curl-library]

Post by Dennis Clarke via curl-library
So I looked into the location where the ssl certs "should" be given my
$ ./configure ...
--with-ca-path=/opt/bw/ssl/certs \

Note that this is the *ca path* where OpenSSL expects to find individual certs
stored.

You use --with-ca-bundle to specify a "bundle" as a single file.

OpenSSL supports both setups.

Post by Dennis Clarke via curl-library
So I expect that the cacert.pem file at
https://curl.se/docs/caextract.html
europa$ ls -lapb /opt/bw/ssl/certs/
total 350
drwxr-xr-x 2 root wheel 3 Apr 8 02:35 ./
drwxr-xr-x 5 root wheel 9 Apr 7 00:14 ../
-rw-r--r-- 1 root wheel 208075 Jan 19 04:12 cacert.pem
europa$
This does not help at all and even OpenSSL seems confused.

Exactly, because you now put the bundle in the directory where OpenSSL expects
a directory setup.

You should rather try your downloaded bundle like this:

$ curl --cacert /opt/bw/ssl/certs/cacert.pem -4 -L https://gitlab.com/ -o
/dev/null

... it certainly works for me!

[Dennis Clarke via curl-library]

Post by Daniel Stenberg via curl-library

Post by Dennis Clarke via curl-library
So I looked into the location where the ssl certs "should" be given my
$ ./configure ...
 --with-ca-path=/opt/bw/ssl/certs \

Note that this is the *ca path* where OpenSSL expects to find individual
certs stored.
You use --with-ca-bundle to specify a "bundle" as a single file.
OpenSSL supports both setups.

ah ha. Well that makes sense and now I can rebuild curl with
better/other config options to specify the ca cert bundle.

For the sake of being verbose this is what I see :

europa$
europa$ /opt/bw/bin/curl -vvvvv -4 -L --url 'https://gitlab.com/' -o
/dev/null
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Trying 172.65.251.78:443...
* Connected to gitlab.com (172.65.251.78) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
* CApath: /opt/bw/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [25 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4542 bytes data]
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
} [2 bytes data]
* SSL certificate problem: unable to get local issuer certificate
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
europa$


Well there we see CAfile: none.

Post by Daniel Stenberg via curl-library

Post by Dennis Clarke via curl-library
So I expect that the cacert.pem file at
   https://curl.se/docs/caextract.html
europa$ ls -lapb /opt/bw/ssl/certs/
total 350
drwxr-xr-x  2 root  wheel       3 Apr  8 02:35 ./
drwxr-xr-x  5 root  wheel       9 Apr  7 00:14 ../
-rw-r--r--  1 root  wheel  208075 Jan 19 04:12 cacert.pem
europa$
This does not help at all and even OpenSSL seems confused.

Exactly, because you now put the bundle in the directory where OpenSSL
expects a directory setup.
$ curl --cacert /opt/bw/ssl/certs/cacert.pem -4 -L https://gitlab.com/
-o /dev/null
... it certainly works for me!

Well I really don't want to have to specify a cert path on every command
line so it would be best to just build a new curl. Speaking of which
there is a patch in the works for 7.75.1 ?

[Ray Satiro via curl-library]

Post by Dennis Clarke via curl-library
europa$
europa$ curl -4 -L --url 'https://gitlab.com/' -o /dev/null
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here:https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
europa$
europa$ curl --version
curl 7.75.0 (x86_64-unknown-freebsd13.0) libcurl/7.75.0 OpenSSL/1.1.1k
zlib/1.2.11 libidn2/2.3.0 libssh2/1.9.0
Release-Date: 2021-02-03
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt
pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HTTPS-proxy IDN Largefile libz NTLM NTLM_WB
SSL TLS-SRP UnixSockets
europa$
So I looked into the location where the ssl certs "should" be given my
$ ./configure --prefix=/opt/bw --disable-dependency-tracking \
--disable-silent-rules --without-gnu-ld --enable-shared \
--enable-static \
--with-libidn=/usr/local --with-libidn2=/usr/local \
--with-ssl=/opt/bw --with-ca-path=/opt/bw/ssl/certs \
--enable-tls-srp --with-libssh2
So I expect that the cacert.pem file at
https://curl.se/docs/caextract.html
europa$ ls -lapb/opt/bw/ssl/certs/
total 350
drwxr-xr-x 2 root wheel 3 Apr 8 02:35 ./
drwxr-xr-x 5 root wheel 9 Apr 7 00:14 ../
-rw-r--r-- 1 root wheel 208075 Jan 19 04:12 cacert.pem
europa$
This does not help at all and even OpenSSL seems confused.

What is your curl -V version and the verbose output? Your CA path is
supposed to contain the certificate files named by hash value [1], which
it doesn't. Putting a single file with a bundle of certificates in the
path won't help. You can use configure option --with-ca-bundle=FILE.
Note your OS may have a packaged and maintained directory containing
certificates or a bundle of certificates, I would use that if you can.
Also see [2] for scanned paths

[1]:
https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_load_verify_locations.html
[2]: https://github.com/curl/curl/blob/curl-7_76_0/acinclude.m4#L2182-L2192

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: http

[Dennis Clarke via curl-library]

Post by Ray Satiro via curl-library

Post by Dennis Clarke via curl-library
europa$
europa$ curl -4 -L --url 'https://gitlab.com/' -o /dev/null
   % Total    % Received % Xferd  Average Speed   Time    Time     Time
Current
                                  Dload  Upload   Total   Spent    Left
Speed
   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
     0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here:https://curl.se/docs/sslcerts.html
curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this
situation and
how to fix it, please visit the web page mentioned above.
europa$
europa$ curl --version
curl 7.75.0 (x86_64-unknown-freebsd13.0) libcurl/7.75.0 OpenSSL/1.1.1k
zlib/1.2.11 libidn2/2.3.0 libssh2/1.9.0
Release-Date: 2021-02-03
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt
pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HTTPS-proxy IDN Largefile libz NTLM NTLM_WB
SSL TLS-SRP UnixSockets
europa$
So I looked into the location where the ssl certs "should" be given my
$ ./configure --prefix=/opt/bw --disable-dependency-tracking \
  --disable-silent-rules --without-gnu-ld --enable-shared \
  --enable-static \
  --with-libidn=/usr/local --with-libidn2=/usr/local \
  --with-ssl=/opt/bw --with-ca-path=/opt/bw/ssl/certs \
  --enable-tls-srp --with-libssh2
So I expect that the cacert.pem file at
     https://curl.se/docs/caextract.html
europa$ ls -lapb/opt/bw/ssl/certs/
total 350
drwxr-xr-x  2 root  wheel       3 Apr  8 02:35 ./
drwxr-xr-x  5 root  wheel       9 Apr  7 00:14 ../
-rw-r--r--  1 root  wheel  208075 Jan 19 04:12 cacert.pem
europa$
This does not help at all and even OpenSSL seems confused.

What is your curl -V version and the verbose output? Your CA path is
supposed to contain the certificate files named by hash value [1], which
it doesn't.

I just dropped the correct certs into that directory and they are still
not found :

europa$ grep '89:80:cc:26' /opt/bw/ssl/certs/*
/opt/bw/ssl/certs/USERTrust_ECC_Certification_Authority.pem:
5c:8b:99:c5:5a:94:c5:d2:71:56:de:cd:89:80:cc:26
europa$ grep '0e:35:03:2d' /opt/bw/ssl/certs/*
/opt/bw/ssl/certs/USERTrust_RSA_Certification_Authority.pem:
01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d


Those are exactly what is needed.

europa$ /opt/bw/bin/curl -V
curl 7.75.0 (x86_64-unknown-freebsd13.0) libcurl/7.75.0 OpenSSL/1.1.1k
zlib/1.2.11 libidn2/2.3.0 libssh2/1.9.0
Release-Date: 2021-02-03
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt
pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS HTTPS-proxy IDN Largefile libz NTLM NTLM_WB
SSL TLS-SRP UnixSockets
europa$

So I should think that the certs are found in that directory now :


europa$ /opt/bw/bin/curl -vvvv -4 -L https://gitlab.com/ -o /dev/null
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0* Trying 172.65.251.78:443...
* Connected to gitlab.com (172.65.251.78) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
* CApath: /opt/bw/ssl/certs
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [25 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [4542 bytes data]
* TLSv1.3 (OUT), TLS alert, unknown CA (560):
} [2 bytes data]
* SSL certificate problem: unable to get local issuer certificate
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:--
0
* Closing connection 0
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
europa$

Post by Ray Satiro via curl-library
Putting a single file with a bundle of certificates in the
path won't help. You can use configure option --with-ca-bundle=FILE.

That is exactly what I am going to do. Once a week or so I can check if
the cacert bundle has been updated and then otherwise forget about it.

Post by Ray Satiro via curl-library
Note your OS may have a packaged and maintained directory containing
certificates or a bundle of certificates, I would use that if you can.
Also see [2] for scanned paths

Yes, I was thinking of that also but felt it should not be black magic
to just point to a cacert bundle.

Post by Ray Satiro via curl-library
https://www.openssl.org/docs/man1.0.2/man3/SSL_CTX_load_verify_locations.html
[2]: https://github.com/curl/curl/blob/curl-7_76_0/acinclude.m4#L2182-L2192

Thank you and I will look over all of that.

[Patrick Monnerat via curl-library]

Post by Dennis Clarke via curl-library

Post by Ray Satiro via curl-library

Post by Dennis Clarke via curl-library
So I looked into the location where the ssl certs "should" be given my
$ ./configure --prefix=/opt/bw --disable-dependency-tracking \
  --disable-silent-rules --without-gnu-ld --enable-shared \
  --enable-static \
  --with-libidn=/usr/local --with-libidn2=/usr/local \
  --with-ssl=/opt/bw --with-ca-path=/opt/bw/ssl/certs \
  --enable-tls-srp --with-libssh2
So I expect that the cacert.pem file at
     https://curl.se/docs/caextract.html
europa$ ls -lapb/opt/bw/ssl/certs/
total 350
drwxr-xr-x  2 root  wheel       3 Apr  8 02:35 ./
drwxr-xr-x  5 root  wheel       9 Apr  7 00:14 ../
-rw-r--r--  1 root  wheel  208075 Jan 19 04:12 cacert.pem
europa$
This does not help at all and even OpenSSL seems confused.

What is your curl -V version and the verbose output? Your CA path is
supposed to contain the certificate files named by hash value [1], which
it doesn't.

I just dropped the correct certs into that directory and they are still
europa$ grep '89:80:cc:26' /opt/bw/ssl/certs/*
5c:8b:99:c5:5a:94:c5:d2:71:56:de:cd:89:80:cc:26
europa$ grep '0e:35:03:2d' /opt/bw/ssl/certs/*
01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d
Those are exactly what is needed.

If you use ca-path rather than ca-bundle with openssl, certificates in
that directory have to be named according to a hash in order to be found
by openssl.

To create a soft link properly named use:

ln -s <certname> `openssl x509 -in <certname> -noout -hash`.0

There is also a command to hash a whole directory:
https://www.openssl.org/docs/manmaster/man1/c_rehash.html

-------------------------------------------------------------------
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/m

[Dennis Clarke via curl-library]

On 4/8/21 5:41 AM, Patrick Monnerat via curl-library wrote:
.
.
.

Post by Patrick Monnerat via curl-library

Post by Dennis Clarke via curl-library
europa$ grep '89:80:cc:26' /opt/bw/ssl/certs/*
5c:8b:99:c5:5a:94:c5:d2:71:56:de:cd:89:80:cc:26
europa$ grep '0e:35:03:2d' /opt/bw/ssl/certs/*
01:fd:6d:30:fc:a3:ca:51:a8:1b:bc:64:0e:35:03:2d
Those are exactly what is needed.

If you use ca-path rather than ca-bundle with openssl, certificates in
that directory have to be named according to a hash in order to be found
by openssl.
ln -s <certname> `openssl x509 -in <certname> -noout -hash`.0
https://www.openssl.org/docs/manmaster/man1/c_rehash.html

Brilliant! Thank you. For quite some time I did wonder where those
file numbers/names were coming from :


alpha$ cd /opt/bw/ssl/certs
alpha$ ls -lapb
total 1802
drwxr-xr-x 2 root root 595 Jul 12 2020 ./
drwxr-xr-x 5 root root 9 Mar 27 00:35 ../
lrwxrwxrwx 1 root root 26 Jan 20 2019 00673b5b.0 ->
thawte_Primary_Root_CA.pem
lrwxrwxrwx 1 root root 45 Jan 20 2019 02265526.0 ->
Entrust_Root_Certification_Authority_-_G2.pem
lrwxrwxrwx 1 root root 23 Jan 20 2019 02756ea4.0 ->
Certplus_Root_CA_G1.pem
lrwxrwxrwx 1 root root 31 Jan 20 2019 02b73561.0 ->
Comodo_Secure_Services_root.pem
.
.
.
etc etc etc