New Feature in Payara Server 5.184: Allow Use of Different Security Providers via JCE API

Java SE comes with many security primitives upon which Java EE builds. JAAS for instance is a well known API in this context, from which Java EE uses the Principal, Subject and to some degree the LoginModule types. The Java EE security APIs JACC and JASPIC make direct use of these types.

Another API (Framework) within the Java SE library is JCE (Java Cryptography Extension) which was originally a separate extension to JCA, the Java Cryptography Architecture (not to be confused with JCA, the Java EE Connection Architecture). The difference between JCE and JCA is somewhat cosmetic these days, as JCE isn't an external extension anymore ever since it was included in Java 1.4 all the way back in 2002. The reason it was first separate and then later included has to do with the US export restrictions on encrypting/decrypting technology. Before, all the encrypting/decrypting functionality was put in JCE, which had to be downloaded separately. The base JDK, free of these functions, could therefore be freely exported. Later on these restrictions were relaxed, and JCE could be added to the JDK itself.

Well known Java SE security types such as MessageDigest, KeyStore, SecureRandom, Certificate, Cipher and Mac are part of JCA/JCE.

JCA/JCE is a pluggable service/provider architecture, meaning that for most services offered by the platform, the using code requests a (named) service of some kind via the bootstrap classes being offered, and the runtime queries its registered providers of type Provider to see which one offers that service. Each such provider is essentially a Map containing one or more services and every JDK ships with one of more of these providers (for instance SUN for the Oracle JVM, IBMJCE for the IBM one, etc). Essential is that users can plug-in new, external, providers of their own. Well known examples of such external providers are the Bouncy Castle provider and the original IAIK-JCE provider.

There's essentially two use cases where those multiple providers come into play. The first is that providers can provide different algorithms. For instance, provider 1 can provide algorithm FOO and provider 2 can provide algorithm BAR. This is largely opaque to the end user, who just requests say algorithm BAR and the runtime will retrieve an implementation from provider 2 in this case. Another option is that both provider 1 and provider 2 provide an implementation of FOO, but with different characteristics, for instance strength. Here it's important that the runtime maintains a set order of providers. The user can alter this ordering, and thereby favour a specific provider.

Services as mentioned are named, and consist of two components; a type and an algorithm. As is typical for security, there are many alternative terms in use. For instance, a type is of course not always called a type, but for instance an engine as well (a kind of internal security through obscurity). For registering these services those two are often combined in a single string. For instance, a service with type "CertificateFactory" and algorithm "X.509" is registered via the String "CertificateFactory.X.509".

Servers, such as Payara Server, often need a lot of the functionality offered by JCA/JCE. Specifically when working with TLS (Transport Level Security) such as SSL/HTTPS access to key stores and the handling of certificates is needed. Java EE also requires support for client-cert authentication and for encrypting/decrypting and signing/verifying SOAP messages (MLS, Message Level Security). However, a Java EE server is under no obligation to actually use the JCA/JCE APIs for all of this or use it in a strict way. GlassFish for instance did use the JCA/JCE APIs, but the original developers for some reason felt the need to cast and store several types obtained via these APIs as Sun specific types. We can only guess at why this was originally done, but what we do know is that GlassFish breaks when alternative JCE providers are installed at a more preferred position (lower number) than the SUN provider, or when run on a JDK that doesn't have these SUN types at all.

How to Use a Custom JCE Provider With Payara Server

This is now fixed in Payara 5.184, as all assumptions of JCA/JCE classes being SUN have been removed.

To demonstrate how such a custom JCE provider can be used with Payara Server we'll present a somewhat contrived but working example of how to augment the certificate factory and use it for principal mapping.

While a custom provider can be installed statically in the JDK or via the service loader mechanism, we'll use the programmatic API here for simplicity. Note that it is often not the best way to go about this for an application server, as it changes the provider while the server is running, and this can influence other processes that already use the previous preferred provider. For this example we're also going to be using an existing provider (Bouncy Castle), to which we add a custom service.

We'll start with the certificate factory:

public class MyJCECertificateFactory extends CertificateFactory {
 
    @Override
    public Certificate engineGenerateCertificate(InputStream in) throws CertificateException {
        Certificate certificate = super.engineGenerateCertificate(in);
 
        if (certificate instanceof X509Certificate == false) {
            return certificate;
        }
 
        return new MyJCEX509Certificate((X509Certificate) certificate);
    }
 
}

The factory is based on the CertificateFactorySpi type, but instead of implementing it fully we've re-used Bouncy Castle's CertificateFactory here. What we're doing is just wrapping the returned Certificate with our own one. Our own certificate type then looks as follows:

public class MyJCEX509Certificate extends X509Certificate {
 
    private final X509Certificate certificate;
 
    public MyJCEX509Certificate(X509Certificate certificate) {
        this.certificate = certificate;
    }
     
    @Override
    public X500Principal getSubjectX500Principal() {
         
        X500Principal principal = certificate.getSubjectX500Principal();
 
        if ("C=UK,ST=lak,L=zak,O=kaz,OU=bar,CN=lfoo".equals(principal.getName())) {
            return new X500Principal("CN=u1");
        }
         
        return principal;
    }
     
    @Override
    public Principal getSubjectDN() {
         
        Principal principal = certificate.getSubjectDN();
 
        if ("CN=lfoo,OU=bar,O=kaz,L=zak,ST=lak,C=UK".equals(principal.getName())) {
            return new X500Principal("CN=u1");
        }
         
        return principal;
    }
 
    // Other methods omitted for brevity
}

This custom certificate is where we do our actual "principal mapping". For the example we only map "CN=lfoo,OU=bar,O=kaz,L=zak,ST=lak,C=UK" to "CN=u1" though. Note that we had to override two methods here. The getSubjectDN() one is actually deprecated (or denigrated as in the Certificate's terminology), but it's still being used all over the place. getSubjectX500Principal() is the preferred method to use. Finally, we install the factory via a Servlet that we can easily ping (as mentioned, this would not normally be the advisory way to do this):

@WebServlet(urlPatterns = { "/BouncyServlet" })
public class BouncyServlet extends HttpServlet {
 
    private static final long serialVersionUID = 1L;
 
    @Override
    protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
         
        BouncyCastleProvider provider = new BouncyCastleProvider();
        provider.put("CertificateFactory.X.509", MyJCECertificateFactory.class.getName());
                 
        int pos = Security.insertProviderAt(provider, 1);
                 
        response.getWriter().print("pos:" + pos);
    }
 
}

What we do in the above code is instantiating the bouncy castle provider, and registering our custom certificate factory with it. When we said above a Provider is like a Map we weren't exaggerating, the used put() method here is actually directly from the Map interface. We also see here an example of type and algorithm being combined into the single key for registering our factory class. The modified provider is then inserted at provider position 1 (the first, there's no 0), using a static method of the Security class.

After pinging this Servlet, and requesting another Servlet over HTTPS that's protected with the client-cert authentication mechanism, a principal with name "CN=lfoo,OU=bar,O=kaz,L=zak,ST=lak,C=UK" will be replaced at nearly the lowest level in the system with "CN=u1", and Payara will only see this principal name. We can use this principal name for instance to map groups and roles to it:

<glassfish-web-app>
 
    <security-role-mapping>
        <role-name>g1</role-name>
        <group-name>g1</group-name>
        <principal-name>CN=u1</principal-name>
    </security-role-mapping>
 
</glassfish-web-app>

A fully working sample demonstrating exactly this can be found in the Java EE 7 samples repo.

Give it a Try: