Armory Agent for Kubernetes Installation

Learn how to install the Armory Agent in your Kubernetes and Armory Enterprise environments.

Proprietary

Early Access

The information below is written for an Early Access feature. Contact us if you are interested in using this feature! Your feedback will help shape the development of this feature.

Do not use Early Access features for production environments.

This installation guide is designed for installing the Agent in a test environment. It does not include mTLS configuration, so the Agent service and plugin do not communicate securely.

Before you begin

You deployed Armory Enterprise using the Armory Operator and Kustomize patches.
You have configured Clouddriver to use MySQL or PostgreSQL. See the Configure Clouddriver to use a SQL Database guide for instructions. The Agent plugin uses the SQL database to store cache data.
You have a running Redis instance. The Agent plugin uses Redis to coordinate between Clouddriver replicas. Note: you need Redis even if you only have one Clouddriver instance.
You have read the Armory Agent overview.
If you are running multiple Clouddriver instances, you have a running Redis instance. The Agent uses Redis to coordinate between Clouddriver replicas.
You have an additional Kubernetes cluster to serve as your deployment target cluster.

Networking requirements

Communication from the Agent service to the Clouddriver plugin occurs over gRPC port 9091. Communication between the service and the plugin must be http/2. http/1.1 is not compatible and causes communication issues between the Agent service and Clouddriver plugin.

Compatibility matrix

The Armory Agent is in early access. For more information about using this feature, contact us.

Armory Enterprise (Spinnaker) Version	Armory Agent Plugin Version	Armory Agent Version
2.24.x (1.24.x)	0.7.25	0.5.25
2.25.x (1.25.x)	0.8.24	0.5.25
2.26.x (1.26.x)	0.9.16	0.5.25

Your Clouddriver service must use a MySQL-compatible database. See the Configure Clouddriver to use a SQL Database guide for instructions.

Database compatibility:

MySQL	PostgreSQL
5.7; AWS Aurora	10+

Installation overview

In this guide, you deploy the Agent service to your target cluster.

Installation steps:

Install the Clouddriver plugin. You do this in the cluster where you are running Armory Enterprise.
1. Create the plugin manifest as a Kustomize patch.
2. Create a LoadBalancer service Kustomize patch to expose the plugin on gRPC port 9091.
3. Apply the manifests.
Install the Agent service in the deployment target cluster.

Install the Clouddriver plugin

Create the plugin manifest

Create a new armory-agent directory in your Kustomize patches directory. Add the following agent-config.yaml manifest to your new armory-agent directory.

Change the value for name if your Armory Enterprise service is called something other than “spinnaker”.
Update the kubesvc-plugin value to the Armory Agent Plugin Version that is compatible with your Armory Enterprise version. See the compatibility matrix.

apiVersion: spinnaker.armory.io/v1alpha2
kind: SpinnakerService
metadata:
  name: spinnaker
spec:
  spinnakerConfig:
    profiles:
      clouddriver:
        spinnaker:
          extensibility:
            pluginsRootPath: /opt/clouddriver/lib/plugins
            plugins:
              Armory.Kubesvc:
                enabled: true
        # Plugin config
        kubesvc:
          cluster: redis
#          eventsCleanupFrequencySeconds: 7200
#          localShortCircuit: false
#          runtime:
#            defaults:
#              onlySpinnakerManaged: true
#            accounts:
#              account1:
#                customResources:
#                  - kubernetesKind: MyKind.mygroup.acme
#                    versioned: true
#                    deployPriority: "400"
  kustomize:
    clouddriver:
      deployment:
        patchesStrategicMerge:
          - |
            spec:
              template:
                spec:
                  initContainers:
                  - name: kubesvc-plugin
                    image: docker.io/armory/kubesvc-plugin:<version> # must be compatible with your Armory Enterprise version
                    volumeMounts:
                      - mountPath: /opt/plugin/target
                        name: kubesvc-plugin-vol
                  containers:
                  - name: clouddriver
                    volumeMounts:
                      - mountPath: /opt/clouddriver/lib/plugins
                        name: kubesvc-plugin-vol
                  volumes:
                  - name: kubesvc-plugin-vol
                    emptyDir: {}

Then include the file under the patchesStrategicMerge section of your kustomization file.

bases:
  - agent-service
patchesStrategicMerge:
  - armory-agent/agent-config.yaml

Expose Clouddriver as a LoadBalancer

To expose Clouddriver as a Kubernetes-type LoadBalancer, add the following manifest to your Kustomize directory. Then include the file in the resources section of your kustomization file.

Various cloud providers may require additional annotations for LoadBalancer. Consult your cloud provider’s documentation.

# This LoadBalancer service exposes the gRPC port on Clouddriver for the remote Agents to connect to
# Look for the LoadBalancer service IP address that is exposed on 9091
apiVersion: v1
kind: Service
metadata:
  labels:
  name: spin-agent-clouddriver
spec:
  ports:
    - name: grpc
      port: 9091
      protocol: TCP
      targetPort: 9091
  selector:
    app: spin
    cluster: spin-clouddriver
  type: LoadBalancer

Apply the manifests

After you have configured both manifests, apply the updates.

Get the LoadBalancer IP address

Use kubectl get svc spin-agent-cloud-driver -n spinnaker to make note of the LoadBalancer IP external address. You need this address when you configure the Agent.

Confirm Clouddriver is listening

Use netcat to confirm Clouddriver is listening on port 9091 by executing nc -zv [LB address] 9091. Perform this check from a node in your Armory Enterprise cluster and one in your target cluster.

Install the Agent service

You can either install the Agent service manually using kubectl or with a Helm chart that Armory provides. Note that the Helm chart currently points to the latest image, which is not necessarily a stable release.

kubectl
Helm

Create a namespace

In the deployment target cluster, execute kubectl create ns spin-agent to create a namespace for the Agent service.

Configure permissions

Create a ClusterRole, ServiceAccount, and ClusterRoleBinding for the Agent by applying the following manifest in your spin-agent namespace:

Show me the manifest

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: spin-cluster-role
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - pods/log
  - ingresses/status
  - endpoints
  verbs:
  - get
  - list
  - update
  - patch
  - delete
- apiGroups:
  - ""
  resources:
  - services
  - services/finalizers
  - events
  - configmaps
  - secrets
  - namespaces
  - ingresses
  - jobs
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
  - delete
- apiGroups:
  - batch
  resources:
  - jobs
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
- apiGroups:
  - apps
  - extensions
  resources:
  - deployments
  - deployments/finalizers
  - deployments/scale
  - daemonsets
  - replicasets
  - replicasets/finalizers
  - replicasets/scale
  - statefulsets
  - statefulsets/finalizers
  - statefulsets/scale
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
  - delete
- apiGroups:
  - monitoring.coreos.com
  resources:
  - servicemonitors
  verbs:
  - get
  - create
- apiGroups:
  - spinnaker.armory.io
  resources:
  - '*'
  - spinnakerservices
  verbs:
  - create
  - get
  - list
  - update
  - watch
  - patch
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - '*'
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: spin-agent
  name: spin-sa
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: spin-cluster-role-binding
subjects:
  - kind: ServiceAccount
    name: spin-sa
    namespace: spin-agent
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: spin-cluster-role

Configure the Agent service

Configure the Agent service using a ConfigMap. Define armory-agent.yml in the data section:

apiVersion: v1
kind: ConfigMap
metadata:
  name: armory-agent-config
  namespace: spin-agent
data:
  armory-agent.yml: |  
  server:
    port: 8082

Clouddriver plugin LoadBalancer

Replace [LoadBalancer Exposed Address] with the IP address you obtained in the Get the LoadBalancer IP address section.

apiVersion: v1
kind: ConfigMap
metadata:
  name: armory-agent-config
  namespace: spin-agent
data:
  armory-agent.yaml: |
    clouddriver:
      grpc: [LoadBalancer Exposed Address]:9091
      insecure: true

Kubernetes account

Add your Kubernetes account configuration for your cluster:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kubesvc-config
  namespace: spin-agent
data:
  armory-agent.yaml: |
    clouddriver:
      grpc: <LoadBalancer Exposed Address>:9091
      insecure: true
    kubernetes:
     accounts:
     - name:
       kubeconfigFile:
       insecure:
       context:
       oAuthScopes:
       serviceAccount: true
       serviceAccountName: spin-sa
       namespaces: []
       omitNamespaces: []
       onlyNamespacedResources:
       kinds: []
       omitKinds: []
       customResourceDefinitions: [{kind:}]
       metrics:
       permissions: []
       maxResumableResourceAgeMs:
       onlySpinnakerManaged:
       noProxy:

See the Agent options for field explanations.

Apply the manifest to your spin-agent namespace.

Deploy the Agent service

Apply the following Agent deployment manifest in your spin-agent namespace:

Show me the manifest

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: spin
    app.kubernetes.io/name: armory-agent
    app.kubernetes.io/part-of: spinnaker
    cluster: spin-armory-agent
  name: spin-armory-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: spin
      cluster: spin-armory-agent
  template:
    metadata:
      labels:
        app: spin
        app.kubernetes.io/name: armory-agent
        app.kubernetes.io/part-of: spinnaker
        cluster: spin-armory-agent
    spec:
      serviceAccount: spin-sa
      containers:
      - image: armory/kubesvc:<version> # must be compatible with your Armory Enterprise version
        imagePullPolicy: IfNotPresent
        name: armory-agent
        ports:
          - name: health
            containerPort: 8082
            protocol: TCP
          - name: metrics
            containerPort: 8008
            protocol: TCP
        readinessProbe:
          httpGet:
            port: health
            path: /health
          failureThreshold: 3
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /opt/spinnaker/config
          name: volume-armory-agent-config
        # - mountPath: /kubeconfigfiles
        #   name: volume-armory-agent-kubeconfigs
      restartPolicy: Always
      volumes:
      - name: volume-armory-agent-config
        configMap:
          name: armory-agent-config
      # - name: volume-armory-agent-kubeconfigs
      #   secret:
      #     defaultMode: 420
      #     secretName: kubeconfigs-secret

On the Kubernetes cluster where you want to install the Agent Service, perform the following steps:

Add the Armory charts repo:

helm repo add armory-charts http://armory.jfrog.io/artifactory/charts

If you have previously added the chart repo, update it with the following commands:

helm repo update 
helm upgrade armory-agent armory-charts/agent-k8s

Create a namespace in the Kubernetes cluster where you are installing the Agent Service. In Agent mode, this is the same cluster as the deployment target for your app.
```
kubectl create namespace <agent-namespace>
```
If you plan to run the Agent in something other than Agent mode, such as Infrastructure mode, you need to create a kubeconfig file that grants access to the deployment target cluster. For example, run the following command if you use Amazon EKS: aws eks update-kubeconfig --name <target-cluster> .
Decide whether or not you want to connect to Armory Cloud services. Armory Cloud services is required for some features to function and affects how you configure the Agent service installation. You must either disable the connection or provide credentials.

If you do not want to connect to Armory Cloud services, you must include the following parameters when running the helm command to install the Agent service:
- --set cloudEnabled=false
- --set grpcUrl=localhost:9090
These parameters control whether or not Agent attempts to connect to Armory Cloud services. They are required if you do not have a clientID and clientSecret and do not want to use Armory Cloud services.

If you want to connect to Armory Cloud services, you must include the following parameters with the values that Armory provided when running the helm command to install the Agent service:
- --set clientId
- --set clientSecret
These are parameters are used to authenticate you to Armory Cloud services.

Run one of the following Helm commands:

Install with default configs in Agent mode:

helm install armory-agent armory-charts/agent-k8s \
--set accountName=<my-k8s-cluster> \ # Provide a descriptive name. This name gets used in the UI and Armory Cloud API. 
--set mode=agent \
--namespace=<agent-namespace> # Namespace where you want to install the Agent.

Depending on your environment and usage, set one or more of the following parameters:

# Disable the connection to Armory Cloud 
--set cloudEnabled=false    
--set grpcUrl=localhost:9090
   
# Authenticate to Armory Cloud
--set clientId=<your-clientId>  
--set clientSecret=<your-Armory-Cloud-secret> 
   
# Custom config options for Kubernetes
--set kubernetes=<kubernetes-options>

# If TLS is disabled in your environment
--set insecure=true

# If you are pulling from a private registry
--set imagePullSecrets=<secret>

For information about additional options, see the Agent config options.

Show me an example

The following examples use the imagePullSecrets and insecure parameters, which may or may not be needed depending on your environment.

This example installs Agent service without a connection to Armory Cloud:

helm install armory-agent --set accountName=demo-account,imagePullSecrets=regcred,grpcUrl=spin-clouddriver-grpc:9091,insecure=true,cloudEnabled=false --namespace dev armory-charts/agent-k8s

This example installs Agent service with a connection to Armory Cloud:

helm install armory-agent --set accountName=hubaccount1,imagePullSecrets=regcred,grpcUrl=agents.staging.cloud.armory.io:443,audience=https://api.cloud.armory.io,tokenIssuerUrl=https://auth.cloud.armory.io/oauth/token,clientId=************,clientSecret=************ --namespace dev armory-charts/agent-k8s

Install with default configs in Infrastructure mode:

helm install armory-agent armory-charts/agent-k8s \
--set accountName=<my-k8s-cluster> \ # Provide a descriptive name. This name gets used in the UI and Armory Cloud API. 
--set mode=infrastructure \
--namespace=<agent-namespace> # Namespace where you want to install the Agent.

Depending on your environment and usage, set one or more of the following parameters:

# Disable the connection to Armory Cloud 
--set cloudEnabled=false    
--set grpcUrl=localhost:9090
   
# Authenticate to Armory Cloud
--set clientId=<your-clientId> 
--set clientSecret=<your-Armory-Cloud-secret> 
   
# Custom config options for Kubernetes
--set kubernetes=<kubernetes-options> 

# If TLS is disabled in your environment
--set insecure=true

# If you are pulling from a private registry
--set imagePullSecrets=<secret>

For information about additional options, see the Agent config options.

Show me an example

The following examples use the imagePullSecrets and insecure parameters, which may or may not be needed depending on your environment.

This example installs Agent service without a connection to Armory Cloud:

helm install armory-agent --set mode=infrastructure,accountName=demo-account,imagePullSecrets=regcred,grpcUrl=spin-clouddriver-grpc:9091,insecure=true,cloudEnabled=false --set-file kubeconfig=$HOME/.kube/config --namespace dev armory-charts/agent-k8s

This example installs Agent service with a connection to Armory Cloud:

helm install armory-agent --set accountName=hubaccount1,imagePullSecrets=regcred,grpcUrl=agents.staging.cloud.armory.io:443,audience=https://api.cloud.armory.io,tokenIssuerUrl=https://auth.cloud.armory.io/oauth/token,clientId==************,clientSecret=************ --namespace dev armory-charts/agent-k8s

Install with custom settings:

Use helm template to generate a manifest.

helm template armory-agent armory-charts/agent-k8s \
--set-file kubeconfig=<path-to-your-kubeconfig>,armoryagent.yml=<path-to-agent-options>.yml \ 
--namespace=<agent-namespace> # Namespace where you want to install the Agent.

For armoryagentyml, create the file and customize it to meet your needs. For information about the options, see the Agent config options.

Install the helm chart using your template:

helm install armory-agent <local-helm-chart-name>

Confirm success

Create a pipeline with a Deploy manifest stage. You should see your target cluster available in the Accounts list. Deploy a static manifest.

What’s next

Troubleshoot the Armory Agent Service and Plugin page if you run into issues.
Learn how to Monitor the Armory Agent with Prometheus. Agent CPU usage is low, but the amount of memory depends on the size of the cluster the Agent is monitoring. The gRPC buffer consumes about 4MB of memory.
Configure Mutual TLS Authentication
Read about Kubernetes Permissions for the Armory Agent

Feedback

Was this page helpful?

Thank you for letting us know!

Sorry to hear that. Please tell us how we can improve.

Last modified September 1, 2021: (4731d13)