Kubernetes

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

Overview

We've utilised Kubernetes' CRD implementation to allow you to define fine grained control over your credentials. As with our Terraform provider, we allow you to filter out projects, resources and specific credentials you want to load.

Configuration

Manifold’s Kubernetes integration is done in 5 steps:

  1. Create an auth token
  2. Create Kubernetes secret
  3. Define RBAC rules (Optional)
  4. Configure Kubernetes credentials controller
  5. Reference credentials

Auth Token

An API key is required for your user or team, so that we can securely look up and decrypt credentials in Terraform. API keys are currently created through the Manifold CLI. For the Terraform Provider you must grant read-credentials access to the token.

To get started with API tokens refer to the authentication documentation.

Create Kubernetes Secret

Once you have the token, you'll want to create a new Kubernetes Secret:

$ kubectl create namespace manifold-system

$ kubectl create --namespace=manifold-system secret generic manifold-api-secrets --from-literal=api_token=<AUTH_TOKEN> --from-literal=team=<MANIFOLD_TEAM>

Replace <AUTH_TOKEN> with the newly created token.

The team value is optional. If a team is provided in the controller (see below), only resources that define this team will be picked up and used to load the credentials. If no team is defined, this is ignored.

RBAC Rules

If your cluster uses role-based access control, you need to give access to manifold-system.

For Google Kubernetes Engine, you need to create an cluster-admin first:

$ kubectl create clusterrolebinding manifold:credentials --clusterrole=cluster-admin --user=<YOUR_GKE_EMAIL>

Example manifold-rbac.yml:

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: manifold:credentials
rules:
  - apiGroups: ["manifold.co"]
    resources: ["projects", "resources"]
    verbs: ["*"]
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["*"]
  - apiGroups: ["apiextensions.k8s.io"]
    resources: ["customresourcedefinitions"]
    verbs: ["*"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: manifold:credentials
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
  - name: manifold-credentials
    namespace: manifold-system
    kind: ServiceAccount

To apply:

$ kubectl apply -f manifold-rbac.yml

Credentials Controller

The controller takes care of monitoring your Resource Definitions and populating the correct Kubernetes Secrets with Manifold Credentials.

An example of a controller manifest credentials-controller.yml:

apiVersion: v1
kind: Namespace
metadata:
  name: manifold-system

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: manifold-credentials
  namespace: manifold-system

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: credentials-controller
  namespace: manifold-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: "manifold-k8s-credentials-controller"
    spec:
      serviceAccountName: manifold-credentials
      containers:
        - name: credentials-controller
          image: manifoldco/kubernetes-credentials:v0.1.1
          env:
            - name: MANIFOLD_API_TOKEN
              valueFrom:
                secretKeyRef:
                  name: manifold-api-secrets
                  key: api_token
            - name: MANIFOLD_TEAM
              valueFrom:
                secretKeyRef:
                  name: manifold-api-secrets
                  key: team
                  optional: true
          resources:
            requests:
              cpu: 25m
              memory: 10Mi

This is a general purpose Deployment, you can customise to fit your needs. To apply:

$ kubectl apply -f credentials-controller.yml

Reference Credentials

Once you've set up the controller, it will start looking for the resources defined earlier and write the values from Manifold to the respective Kubernetes secret. This means that when a credential changes, the secret will also be updated automatically with the new value.

An example Deployment referencing Manifold secrets:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-service
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: my-service
    spec:
      containers:
        - name: my-service
          image: manifoldco/my-service:latest
          env:
            - name: CUSTOM_TOKEN_ID
              valueFrom:
                secretKeyRef:
                  name: manifold-api-secrets
                  key: TOKEN_ID
            - name: RESOURCE2_USERNAME
              valueFrom:
                secretKeyRef:
                  name: manifold-api-secrets
                  key: USERNAME
            - name: CUSTOM_TOKEN_SECRET
              valueFrom:
                secretKeyRef:
                  name: manifold-api-secrets
                  key: alias-name
            - name: NON_EXISTING_TOKEN
              valueFrom:
                secretKeyRef:
                  name: manifold-api-secrets
                  key: NON_EXISTING

You can use the same pattern to access reference secrets from your Deployment:

- name: SECRET_NAME_FOR_DEPLOYMENT
  valueFrom:
   secretKeyRef:
     name: manifold-api-secrets
     key: SECRET_KEY_ON_MANIFOLD