Configure mTLS for Spinnaker Services

Overview of mTLS

mTLS is a transport level security measure. When a client connects to a server, as in a TLS connection:

• The server responds with its certificate. Additionally, the server sends a certificate request and a list of Distinguished Names the server recognizes.
• The client verifies the certificate of the server and responds with its own certificate and the Distinguished Name its certificate was (in)directly signed with.
• The server verifies the certificate of the client.

To set up TLS, provide the following:

1. For a server:

   • Certificate and private key
   • Chain of certificates to validate clients

2. For a client:

   • Certificate and private key to present to the server
   • Chain of certificates to validate the server (if self signed)

For information about TLS, see Configure TLS for Spinnaker Services.

What you need

The following table lists the Armory and Spinnaker services, their type (Java or Golang), and which certificates they need:

• Dinghy is the service for Pipelines as Code.
• Terraformer is the service for the Armory Terraform Integration.
• PaCRD is the service for Pipelines as Code in the form of Kubernetes Custom Resource Definitions.

Note: Gate may be handled differently if you already terminating SSL at Gate. If not, make sure the load balancer and ingress you are using supports self-signed certificates.

In the following sections, you need to have the following information available:

• ca.pem (all Golang servers): the CA certificate in PEM format
• [service].crt (each Golang server): the certificate and (optionally) the private key of the Golang server in PEM format
• [service].key (each Golang server): the private key of the Golang server if not bundled with the certificate you’re using
• [GOSERVICE]_KEY_PASS (each Golang server): the password to the private key of the server
• truststore.p12 (all Java clients): a PKCS12 truststore with CA certificate imported
• TRUSTSTORE_PASS (all Java clients): the password to the truststore you’re using
• [service].p12 (each Java server): a PKCS12 keystore containing the certificate and private key of the server
• [SERVICE]_KEY_PASS (each Java server): the password to the keystore you’re using

The server certificate will serve as its client certificate to other services. You can generate different certificates and use them in ok-http-client.key-store* (Java) and http.key* (Golang).

To learn how to generate these files, see the Generate Certificates for Spinnaker guide.

Configuring Java services

Add the following to each Java service profile: <deploy>/profiles/<service>-local.yml in Halyard or under profiles in the SpinnakerService’s profiles:

# Only needed for "server" role
server:
  ssl:
    enabled: true
    key-store: <reference to [service].p12>
    key-store-type: PKCS12
    key-store-password: <[SERVICE]_KEY_PASS>
    trust-store: <reference to ca.p12>
    trust-store-type: PKCS12
    trust-store-password: <TRUSTSTORE_PASS>
    # Roll out with "want" initially
    client-auth: need

# Needed for all Java services
ok-http-client:
  key-store: <reference to [service].p12>
  key-store-type: PKCS12
  key-store-password: [SERVICE]_KEY_PASS
  trust-store: <reference to truststore.p12>
  trust-store-type: PKCS12
  trust-store-password: <TRUSTSTORE_PASS>

Configuring Golang services

server:
  ssl:
    enabled: true
    certFile: <reference to [service].crt>
    keyFile: <reference to [service].key if not included in the certFile's PEM>
    keyPassword: <[GOSERVICE]_KEY_PASS if required>
    cacertFile: <reference to ca.pem>
    # Roll out with "want" initially
    clientAuth: need

http:
  cacertFile: <reference to ca.pem>
  clientCertFile: <reference to [service].crt>
  clientKeyFile: <reference to [service.key]>
  clientKeyPassword: <[GOSERVICE]_KEY_PASS if required>

Changing service endpoints

Use either the Operator or Halyard to change Spinnaker’s service endpoints.

Operator

Change the SpinnakerService custom resource:

kind: SpinnakerService
...
spec:
  spinnakerConfig:
    service-settings:
      clouddriver:
        baseUrl: https://spin-clouddriver.<NAMESPACE>:7002
      dinghy:
        baseUrl: https://spin-dinghy.<NAMESPACE>:8081
      echo:
        baseUrl: https://spin-echo.<NAMESPACE>:8089
      fiat:
        baseUrl: https://spin-fiat.<NAMESPACE>:7003
      front50:
        baseUrl: https://spin-front50.<NAMESPACE>:8080
      gate:
        baseUrl: https://spin-gate.<NAMESPACE>:8084
      kayenta:
        baseUrl: https://spin-kayenta.<NAMESPACE>:8090
      orca:
        baseUrl: https://spin-orca.<NAMESPACE>:8083
      igor:
        baseUrl: https://spin-igor.<NAMESPACE>:8088
      rosco:
        baseUrl: https://spin-rosco.<NAMESPACE>:8087
      terraformer:
        baseUrl: https://spin-terraformer.<NAMESPACE>:7088

Halyard

Change each Java or Golang service endpoints by adding the following in <hal directory>/<deployment>/service-settings/<service>.yml:

baseUrl: https://spin-<SERVICE>.<NAMESPACE>:<SERVICE PORT>

Changing readiness probe

Change the readiness probe used by Kubernetes from an HTTP request to a TCP probe.

Operator

Add the following snippet to each service in SpinnakerService manifest:

apiVersion: spinnaker.armory.io/v1alpha2
kind: SpinnakerService
metadata:
  name: spinnaker
spec:
  spinnakerConfig:  
    service-settings:
      <service>:
        kubernetes:
          useTcpProbe: true

Halyard

This can be done by adding the following to each service under <deploy>/service-settings/<service>.yml:

kubernetes:
  useTcpProbe: true

Important: Version 1.8.2 or later of Halyard Armory is required to support TCP probe.

Deployment

Apply your changes to your Spinnaker deployment:

Operator

kubectl -n <spinnaker namespace> apply -f <SpinnakerService manifest>

Halyard

hal deploy apply

If Spinnaker services are already using HTTPS, you can roll out mTLS without interruption by making the client certificate optional (want) in server.ssl.client-auth (Java) and server.ssl.clientAuth. Then once all the services are stable, rolling out a new configuration with that value set to need.