There’s some confusion on the Internet about how to control which certificates are used for server (and non-server) TLS sockets and why client certs just don’t seem to work right (see here, here, here, here, etc.). I’ll explain how the process is supposed to happen, explain why it doesn’t necessarily work easily with Java, and how to work around the problem. Though this article generally applies to SSL as well as TLS, I’ll refer to just TLS from now on. Also, the testing and bug hunting in this article were done against Sun/Oracle’s JDK 6u25. I have not confirmed whether or not these issues are fixed in Java 7.
Basic terminology
Certificate or cert
The public half of a public/private key pair, though it’s not generally referred to as a key. This part is freely given to anyone.
Private key
A private key is never given out publicly. It is used to sign or encrypt data. A private key can be used to verify that its corresponding certificate was used to sign or encrypt things and vice versa.
Certificate Signing Request or CSR
A file that you generate with your private key. You can send just the CSR to your CA and they will create a signed certificate for you.
Certificate Authority or CA
These are places like Thawte that you pay in order to get a certificate that browsers will accept. You can also use someone like CACert.org to get a free certificate that browsers will not accept. You can also generate your own simple CA using openssl. A CA uses its private key to digitally sign a CSR and create a signed cert so that browsers can use the CA’s cert to tell that your cert is approved by that CA.
Distinguished Name or DN
This is defined by LDAP. It’s a grouping of RDNs (Relative Distinguished Names). A RDN is something like “CN=your name”. This one means that the Common Name is set to the string “your name”. A DN would be something like “CN=your name,OU=Engineering,O=Initech”. In this case, OU means Organizational Unit and O means Organization.
X509
A specification governing the format and usage of certificates.
How client certs are supposed to work
This is a greatly simplified explanation of the TLS 1.0 protocol. Check out the RFC for more details at around section 7.4. (There are newer versions of TLS, but 1.0 is what Java 6 supports.) I’m showing the case where client certificates have been configured on the server side, so this isn’t exactly what happens when you do normal server-only TLS.
- ClientHello: client informs server what ciphers and compression methods it supports
- ServerHello
- Server picks a cipher and compression that both it and the client support and tells the client about its choices, as well as some other things like a session id
- It presents its certificate (this is what the client needs to validate as being signed by a trusted CA)
- It presents a list of certificate authority DNs that client certs may be signed by
- Client response
- The client continues the key exchange protocol necessary to set up a TLS session
- The client presents a certificate that was signed by one of the CAs described in the Server hello
- The server accepts the cert that the client presented and all is well
Note that the ServerHello does not ask for specific client certificates. It just provides info about CAs (in the form of DNs) and expects the client to figure out an appropriate cert that was signed by one of those CAs.
File formats for certs and keys
There are many different formats used for storing keys and certs on-disk, but the most common ones are probably PEM, PKCS12, and JKS. The formats are not treated equally by Java, so it’s important to understand the different formats. It’s not obvious how to manipulate these formats, so I’ve also included sample commands for working them them.
PEM
PEM is just DER that’s been Base64 encoded. It looks like this for a certificate:
-----BEGIN CERTIFICATE----- (base 64 encoded stuff) -----END CERTIFICATE-----
or for a private key:
-----BEGIN PRIVATE KEY----- (base 64 encoded stuff) -----END PRIVATE KEY-----
Sometimes PEM files will have a human-readable block of text above the Base64 encoded block. You can safely remove this human-readable text. (It confuses Java’s keytool.) Some applications prefer the cert PEM and the private key PEM to be in one file. (Apache httpd is one, if I remember correctly.) Since PEM files are just plain text, you can do that with cat: cat cert.pem key.pem > cert-with-key.pem. While you could arbitrarily combine as many PEM blocks as you wanted into one file, typically they are kept separate except for this one case. You can get a human-readable description of a cert in PEM format with openssl x509 -in cert.pem -noout -text. (Certs are X509 formatted, hence the ‘x509′ subcommand to openssl.) For keys, the command is openssl rsa -in key.pem -text -noout. Private keys can also be encrypted, in which case the marker block will say BEGIN ENCRYPTED PRIVATE KEY. You can create the decrypted form of the key with openssl rsa -in key-encrypted.pem -out key-decrypted.pem.
PKCS12
PKCS12 is a password-protected format that can contain multiple certificates and keys.
You can view the contents of a PKCS12 file (typically .p12 is used for PKCS12 files) with openssl pkcs12 -in file.p12. Add -info for a little bit more metadata. Note that if the file includes a private key, openssl will ask you for another password after asking for the decryption password for the PKCS12 file. This second password is used to encrypt the private key before displaying its PEM data to you. You could put this data in a separate file and decrypt it as shown above if you want the decrypted form.
You can create PKCS12 files with or without private keys or CA certs.
- Cert and key:
openssl pkcs12 -export -out cert-and-key.p12 -in cert.pem -inkey key.pem - Cert and key that includes the CA cert that signed the cert:
openssl pkcs12 -export -out cert-and-key-with-ca.p12 -in cert.pem -inkey key.pem -CAfile /path/to/cacert.pem -chain - Cert without key (useful for CA certs):
openssl pkcs12 -export -out cacert.p12 -in cacert.pem -nokeys
JKS
A JKS keystore stores multiple certs and keys like PKCS12, but it’s just a Java thing, not a widespread standard like PKCS12. The tool to manage JKS files is ‘keytool’ which ships with the JDK. Entries in a JKS file have an “alias” that must be unique. If you don’t specify an alias, it will use “mycert” by default. This is fine if you’re only putting one thing in a keystore, but if you add another thing you’ll get an error because it will try to use the same (default) alias twice. JKS keystores also have a password, just like PKCS12. You can use keytool to add PEM and PKCS12 files.
- Create JKS with cert, key and CA cert from PKCS12:
keytool -importkeystore -destkeystore cert-and-key-with-ca.jks -srckeystore cert-and-key-with-ca.p12 -srcstoretype PKCS12 - Add a CA cert, then add the cert without a key:
keytool -keystore cacert-added-then-cert-nokey.jks -import -file cacert.pem -alias cacert(Say yes when it asks if you want to trust the CA)keytool -keystore cacert-added-then-cert-nokey.jks -import -file cert.pem -alias cert - Add a CA cert, then add the cert with a key:
keytool -keystore cacert-added-then-cert-withkey.jks -import -file cacert.pem -alias cacert(Say yes when it asks if you want to trust the CA)keytool -destkeystore cacert-added-then-cert-withkey.jks -importkeystore -srckeystore cert-and-key.p12 -srcstoretype PKCS12
TLS with Java
There are going to be a lot of certs involved in configuring TLS, so let’s first decide on some names.
- server-cert: the cert that the server presents to clients
- server-key: the private key that corresponds to server-cert
- server-ca-cert: the cert of the CA that signed server-cert
- client-cert: the cert that the client presents to the server (if asked to)
- client-key: the private key that corresponds to client-cert
- client-ca-cert: the cert of the CA that signed client-cert
When client certs aren’t enabled, the server presents server-cert to the client. If the client has server-ca-cert designated as a trusted CA, the connection can proceed. When client certs are enabled, the server presents server-cert as its cert and also sends the DN of client-ca-cert. The client checks server-cert against its trusted CA certs as before. It then looks for any certs that it has that are signed by client-ca-cert, finds client-cert, and sends that back. The server-key and client-key keys are used in other parts of the TLS protocol. To be as general as possible, I’m going to assume that client-ca-cert and server-ca-cert are not the same cert and are also not in the system-wide set of trusted CAs. Oracle’s JSSE Reference Guide is useful when figuring out how all these crypto classes fit together, so you may want to have that open as a reference.
If you want to examine the internals of the SSLSocket/SSLServerSocket implementation, set the system property javax.net.debug to “all” for maximum verbosity. You can download the OpenJDK code and step through it by looking for where the debug statements are printed. It’s not as good as a debugger, but there’s not much code and the debug statements are frequent enough that it’s not hard to follow. Wireshark is also extremely useful.
In Java, you use SSLSocketFactory to get SSLSockets and and SSLServerSocketFactory to get SSLServerSocket instances. The simplest usage looks like this:
/*
* The static getDefault() methods return the non-SSL
* factory classes, so they have to be cast.
*/
SSLServerSocketFactory serverSocketFactory =
(SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
SSLServerSocket serverSocket =
(SSLServerSocket) serverSocketFactory.createServerSocket(8443);
SSLSocketFactory socketFactory =
(SSLSocketFactory) SSLSocketFactory.getDefault();
SSLSocket socket =
(SSLSocket) socketFactory.createSocket("localhost", 8443);
// do the standard socket stuff with byte streams, etc.
This isn’t really going to work, though, since we haven’t told the server socket what cert to use. Time for more terminology!
- A keystore has certs and keys in it and defines what is going to be presented to the other end of a connection.
- A truststore has just certs in it and defines what certs that the other end will send are to be trusted. You could put keys in a truststore, but they wouldn’t be used for anything.
Confusingly, the Java classjava.security.KeyStoreis used in the process of creating both keystores and truststores. I will be careful to capitalize asKeyStorewhen I mean the class as opposed to the conceptual items. For the server socket, we need to specify a keystore containing server-cert and server-key. We also need a truststore containing client-ca-cert. For the client socket, we need a keystore containing the client cert and key and a truststore containing the server-ca-cert. To get these keystores and truststores, we need to constructKeyStoreinstances with the appropriate certificate and key data.KeyStores can be created for JKS or PKCS12 files. This code creates aKeyStoreand loads data from an input stream. Afterload()has been called, theKeyStoreis ready for use.
// keyStoreType is either "JKS" or "PKCS12"
KeyStore keyStore = KeyStore.getInstance(keyStoreType);
keyStore.load(inputStream, keyStorePassword.toCharArray());
A KeyStore is just an intermediate step, though. Once we have a KeyStore with the keystore data and a KeyStore with the truststore data, the next step is a TrustManager (for a truststore) and a KeyManager (for a keystore).
TrustManagerFactory trustManagerFactory =
TrustManagerFactory.getInstance("PKIX", "SunJSSE");
trustManagerFactory.init(trustStore);
Now we have a TrustManagerFactory instance. JSSE is fairly agnostic towards cryptosystems, so it can, at least in theory, support things beyond X509. In practice, X509 is all we care about, and looking in the OpenJDK source code will give the impression that X509 is all it’s built to support anyway. The “PKIX” algorithm implements cert-chain validation for X509 certs. A TrustManagerFactory can create a TrustManager[], one for each type of “trust material”. We only care about the X509TrustManager instance.
X509TrustManager x509TrustManager = null;
for (TrustManager trustManager : trustManagerFactory.getTrustManagers()) {
if (trustManager instanceof X509TrustManager) {
x509TrustManager = (X509TrustManager) trustManager;
break;
}
}
if (x509TrustManager == null) {
throw new NullPointerException();
}
Now we have the X509TrustManager instance we want. A similar approach will get you the X509KeyManager.
KeyManagerFactory keyManagerFactory =
KeyManagerFactory.getInstance("SunX509", "SunJSSE");
keyManagerFactory.init(keyStore, password.toCharArray());
X509KeyManager x509KeyManager = null;
for (KeyManager keyManager : keyManagerFactory.getKeyManagers()) {
if (keyManager instanceof X509KeyManager) {
x509KeyManager = (X509KeyManager) keyManager;
break;
}
}
if (x509KeyManager == null) {
throw new NullPointerException();
}
Now you can construct an SSLContext. Here’s the code to create a SSLServerSocket.
// load in the appropriate keystore and truststore for the server
// get the X509KeyManager and X509TrustManager instances
SSLContext sslContext = SSLContext.getInstance("TLS");
// the final null means use the default secure random source
sslContext.init(new KeyManager[]{keyManager},
new TrustManager[]{trustManager}, null);
SSLServerSocketFactory serverSocketFactory =
sslContext.getServerSocketFactory();
SSLServerSocket serverSocket =
(SSLServerSocket) serverSocketFactory.createServerSocket(PORT);
serverSocket.setNeedClientAuth(true);
// prevent older protocols from being used, especially SSL2 which is insecure
serverSocket.setEnabledProtocols(new String[]{"TLSv1"});
// you can now call accept() on the server socket, etc
And here’s how to construct an SSLSocket. Make sure you don’t use the same keystore and truststore that you did for the server! They almost certainly need to be different.
// load in the appropriate keystore and truststore for the client
// get the X509KeyManager and X509TrustManager instances
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(new KeyManager[]{keyManager},
new TrustManager[]{trustManager}, null);
SSLSocketFactory socketFactory = sslContext.getSocketFactory();
SSLSocket socket =
(SSLSocket) socketFactory.createSocket("localhost", SslServer.PORT);
socket.setEnabledProtocols(new String[]{"TLSv1"});
// read from the socket, etc
It should now work. Unfortunately, Java’s default KeyStore implementation has some bugs, so depending on how you set up your server-side trust store, it may or may not work.
But it doesn’t work when I try that!
There’s a bug in the way cert chains are handled for X509TrustManager objects in the Sun/Oracle implementation. A quick look in the OpenJDK code in sun.security.ssl.X509TrustManagerImpl and sun.security.validator.KeyStores shows that the logic used to get issuers is simply wrong. In the case where the KeyStore entry in a KeyStore is a key entry (not a bare cert), it unconditionally uses the first cert in the chain of certs for that key, regardless of whether or not it is even a CA cert or the actual issuing cert in the chain. In fact, the documentation for KeyStore.getCertificateChain() says that the root cert is the last cert in the chain, not the first. This code was probably tested using self-signed certs (which only have one cert in the chain, so it will always work) and not using separate CA certs.
There’s also a separate bug in the way PKCS12 files are loaded. It looks like maybe the PKCS12 parsing code can’t figure out what to do when there isn’t a private key. I haven’t looked into the source of that bug yet. The SunJSSE description in the JSSE Reference Guide has this terse note: “Storing trusted anchors in PKCS12 is not supported. Users should store trust anchors in JKS format and save private keys in PKCS12 format.” It’s unfortunate that this isn’t broadcast more clearly in the documentation. If you load a PKCS12 file containing just client-ca-cert, you get nothing in the server’s X509TrustManager KeyStore. This is the PKCS12 loading bug. When you connect a client, you get java.net.SocketException: Broken pipe on the client side and javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: No trusted certificate found on the server. This happens because the server has sent a CertificateRequest in the ServerHello, but has not included any DNs for the client to look up certs by.
Construct a PKCS12 file containing client-cert, client-key, and client-ca-cert and then import that in one step (using the -importkeystore option to keytool). If you load that PKCS12 file or the resulting JKS, you get a chain of client-cert and client-ca-cert in the KeyStore. This is correct. However, X509TrustManager.getIssuers() mistakenly returns client-cert as an issuer (which it is not) and does not return client-ca-cert. The correct behavior would be to return only client-ca-cert. (An “issuer” is a CA.) In this case, you get javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate on the client and javax.net.ssl.SSLHandshakeException: null cert chain on the server. The server sends the DN of client-cert in the CertificateRequest part of the ServerHello. The client (correctly) does not find any certs signed by that cert, so it returns no certificates. The server rejects the connection with error code 42 for “bad certificate” (see the TLS RFC section A.3 for error codes) and dies with its own error that (accurately) says there is a null certificate chain from the client.
If you load a JKS file that was created by separately adding client-ca-cert in step 1 (using -import -file client-ca-cert.pem) and then the client-cert (with -import -file client-cert.pem or -importkeystore with a p12 containing client-cert and client-pem) in step 2, the JKS will have two separate entries instead of one entry containing a chain of certs. This will lead to getIssuers() returning both client-cert and client-ca-cert, so both certs will have their DNs in the ServerHello. This is technically incorrect (client-cert is not a CA cert) but does work.
The good news
Fortunately, the JKS implementation of KeyStore is not suffering from the same bug as the PKCS12 code. A KeyStore loaded from a JKS containing only client-ca-cert does end up with a cert in it. Since it’s just a cert, not a cert in a chain attached to a key, it avoids the buggy code path in KeyStores, so the correct DN gets sent to the client in the ServerHello and all proceeds normally.
| Key contents | Key type | Result of getIssuers() |
|---|---|---|
| client-cert, client-key, client-ca-cert | PKCS12 | client-cert |
| client-cert, client-key, client-ca-cert | JKS | client-cert |
| client-cert, client-key | PKCS12 | client-cert |
| client-cert, client-key | JKS | client-cert |
| client-cert, client-ca-cert | PKCS12 | (empty) |
| client-ca-cert | PKCS12 | (empty) |
| client-ca-cert added first, then client-cert & client-key | JKS | client-cert and client-ca-cert |
| client-ca-cert added first, then client-cert | JKS | client-cert and client-ca-cert |
| client-ca-cert | JKS | client-ca-cert (what you want) |
The result of all this analysis: For your SSLServerSocket’s TrustManager’s KeyStore, use a JKS containing only the CA cert for the client certs. If you us a PKCS12, you’ll get no certs. If you include the cert and key that you’re looking for as well as the CA cert and you create the JKS keystore from one PKCS12 containing all three entities, you’ll get the wrong cert. If you create a JKS keystore using the CA cert and then add the client cert (with or without key) later, you’ll get too many certs.