Using Kubernetes Secrets with Payara Micro

In this blog, I'll discuss the basics around Kubernetes ConfigMaps and Secrets and give two examples of how you can use them in combination with Payara Micro.

Just as with your application, we want to keep the configuration values for the (Docker) Containers we are running within our Kubernetes Cluster separate from the image itself.

When our application needs to connect to a database or make use of values that could change over time, like a reference date, it is best to define them outside of the Docker Image. That way, we do not need to recreate the image, but instead, only need adjust those configuration values.

About Kubernetes Configmaps and Secrets

Kubernetes has two concepts for passing configuration values to the Containers: ConfigMaps, and Secrets. Configmaps and Secrets are identical other than the applied security.

You define key-value pairs holding your configuration values with them, and you can pass them to the container in two ways: as files or as environment values.

The concept of key-value pairs is probably known by everyone. The key defines what the value is about, like databasename=test. So whenever we want to know the value which is defined for the database name, we can request the configuration key databasename.

Within a Container, we can use some well-known concepts to retrieve those values, files, and environment variables and thus we do not need to integrate with some specially designed and implemented service within Kubernetes to retrieve those values. This makes retrieving those configuration values very easy. Keep reading to see examples of both ways of accessing the configuration values.

In the rest of this blog we will concentrate on using Kubernetes Secrets, but keep in mind using ConfigMaps is very similar.

Creating your First Secret

Let's execute some commands to generate some Kubernetes Secrets. For this blog, I assume you have access to a Kubernetes Cluster using the kubectl CLI command.

You can create a secret based on key-value pairs on the command line or defined within a file (use the --from-file option for that)

kubectl create secret generic test --from-literal=key=value

Get the list of all secrets known by Kubernetes (within the current namespace by default):

kubectl get secrets

Which gives as a result:

Get the content of the Secret:

kubectl get secret test -o yaml

which gives as output:

apiVersion: v1
data:
  key: dmFsdWU=
kind: Secret
metadata:
  creationTimestamp: 2019-08-08T14:34:01Z
  name: test
  namespace: demo
  resourceVersion: "1461989"
  selfLink: /api/v1/namespaces/demo/secrets/test
  uid: 90d5e1bc-b9e9-11e9-84c8-025000000001
type: Opaque

How Secure is a Secret?

At first glance, your configuration values are nicely secured when you see the output of the kubectl get secret command. Until you realise that `dmFsdWU=` is just the base64 encoding of `value`. So how secure are Secrets?

If you look at this subject within the documentation, you will learn that those Secrets can be encrypted at rest. So when stored within Kubernetes, it is encrypted. But once you request the value, it is decrypted. And this makes sense because the Secret must be readable as an environment variable, for example, so that it can be used by your scripts and applications.

To obscure it a bit, it is just base64 encoded when you request the value through the kubectl command. So make sure that you assign the appropriate roles (See Kubernetes RBAC system for more information on this) so that the secret can only be read by authorized people.

Secrets are kept "secret" in the best possible way, but eventually, the configuration value needs to be accessed readable by your Container.

Mounted File Example

There are various scenarios where you need a sensitive value as contents of a file in the running Container. Those sensitive values should not be defined with the Docker build script, as the values are then exposed to unauthorized people.

One of the scenarios which are a valid use case for this is the definition of the private key for some encryption done by your application. The application can read this file and use it.

Let's have a look at the yaml definition of the pod (which is also possible with a deployment which is preferred for production scenarios) how the mapping from a Kubernetes secret to a file can be done.

pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: payara-secret
  labels:
    name: payara
spec:
  containers:
    - name: payara-secret
      image: secret
      imagePullPolicy: IfNotPresent
      volumeMounts:
        - name: secret-volume
          mountPath: "/opt/payara/mounted"
          readOnly: true
  volumes:
    - name: secret-volume
      secret:
        secretName: aeskey
        items:
          - key: key
            path: secret

First, we define the mounted volume. We specify the path where it will be mounted, /opt/payara/mounted in the example, and the name of the volume that needs to be mounted at that location.

The volume, however, is not a real volume on a disk of your Kubernetes provider for example but is based on a Secret. The secretName defines the name of the secret and the path is the name of the file which needs to be created.

So the above example takes the value of key in the Secret called aeskey and places it in the /opt/payara/mounted/secret file.

The command to create the Secret is described earlier and could be like

kubectl create secret generic aeskey --from-literal=key=Dx5DfWnBDuCYPdFQX02HTg

The value, in this case, is a base64 encoded byte array which will be used for an AES symmetric encryption.

Environment Variable Example

Defining database values is a typical example where you can use environment values. With Payara Micro you can define the data sources through post-boot scripts.

These scripts contain the asadmin commands which needs to be executed when the Payara Micro instance is started and before the application is deployed. References to environment variables are possible within those commands. For more information on those scripts, you can have a look at the Payara Micro Documentation.

Creating a JDBC connection pool can be done through the following asadmin commands:

create-jdbc-connection-pool --datasourceclassname com.mysql.jdbc.jdbc2.optional.MysqlDataSource --restype javax.sql.DataSource rubus-pool

set resources.jdbc-connection-pool.rubus-pool.property.password=${ENV=password} resources.jdbc-connection-pool.rubus-pool.property.databaseName=${ENV=database-name} resources.jdbc-connection-pool.rubus-pool.property.serverName=${ENV=database-server} resources.jdbc-connection-pool.rubus-pool.property.user=${ENV=user} resources.jdbc-connection-pool.rubus-pool.property.portNumber=3306

The environment variables can be defined by the Kubernetes yaml file to create your deployment or pod like this

pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: payara-env-secret
  labels:
    name: payara
spec:
  containers:
    - name: payara-env-secret
      image: env-secret
      imagePullPolicy: IfNotPresent
      env:
        - name: database-server
          valueFrom:
            secretKeyRef:
              name: database-config
              key: database-server
        - name: database-name
          valueFrom:
            secretKeyRef:
              name: database-config
              key: database-name
        - name: user
          valueFrom:
            secretKeyRef:
              name: database-config
              key: user
        - name: password
          valueFrom:
            secretKeyRef:
              name: database-config
              key: password

This config refers to the database-config secret which we can create with the following command

kubectl create secret generic database-config --from-literal=database-server=10.0.1.15 --from-literal=database-name=test --from-literal=user=app-user --from-literal=password=secret

When starting your Container, you can see in the log that those asadmin commands are executed with the Environment references replaced by the values defined in the Kubernetes Secret.

Using Kubernetes Secrets

You should not place configuration values within your Image so that you can easily change them without the need to rebuild your Container image. Kubernetes Configmaps and Secrets are designed for this purpose and allow you to 'map' these configuration values to Environment variables and files.

Kubernetes Secrets should be preferred when those configuration values contain sensitive information. They are encrypted within the Kubernetes system and when requested by a user, they are still base64 encoded so that the real content is not that easy to spot.

Payara Micro can easily be used in combination with Kubernetes Secrets and Environment variables as it has support for Environment Variables within the post-boot script file. Defining some database connection properties becomes quite easily in that case as I showed you in the example.

And of course, files are easy to use by any application and application server so there are many scenarios where the combination of Kubernetes Secrets and Volume mounting can be handy.

Download Payara Micro and Give it a Try:

Learn More: