GitHub Flow for Kubernetes

Heighliner is an open-source system for automating application deployment with Kubernetes.

Overview

In this guide, we will show you how to install Heighliner in your own cluster and stand up the hello hlnr application which is able to:

  • Generate preview deployments for pull requests when they are opened
  • Stage release candidates when a pre-release is made
  • Deploy to production when a release is made
  • Reuse as much configuration as possible

What is Heighliner?

Application deployment within Kubernetes is difficult. To solve this we’ve built an opinionated Kubernetes plugin that leverages your existing GitHub workflow to enable simple deployment on Kubernetes. Heighliner generates preview deployments for Pull Requests when they are opened, stages release candidates when a pre-release is made, and deploys to production when a final release is made.

In addition to improving deployment workflow, we want to improve the experience of writing and managing yaml files. We did this to encourage reusability, reduce yaml necessary to run apps in, and simplify the configuration necessary for high availability.

For more on our motivations, opinions on high availability and security, and the design behind Heighliner see: Introducing Heighliner.

Installation

In the Introducing Heighliner blog post, we highlighted some of the components used by Heighliner. In this section, we’ll explain how you can install Heighliner in your cluster with a view towards using Heighliner. For detailed installation and troubleshooting documentation see installation

Assumptions

To facilitate early progress with this project we have made assumptions about what your production Kubernetes cluster might look like. These assumptions stem from our own Kubernetes experience and will not apply to everyone, yet.

Heighliner also assumes you are using GitHub for source control. The system also expects that you are building Docker images of your Pull Requests and Releases and storing them using a specific naming scheme (see Preview Service) in a DockerHub registry.

Prerequisites

In order use Heighliner you will need to be using the following services with Kubernetes:

Installation

We have made installation as easy as possible with a simple command that will generate all the manifests you need configured to utilize your DNS provider and GitHub callback you provide for setting up GitHub webhooks.

If you encounter any issues through installation please leave your feedback here

Once you have cloned the repo you can run:

make bins && ./bin/heighliner install --version <HEIGHLINER_VERSION> \
    --github-callback-domain <GITHUB_CALLBACK_DOMAIN> \
    --dns-provider <EXTERNAL_DNS_PROVIDER> \
    | kubectl apply -f -

We recommend you use a specific version, like 0.1.0 instead of latest. This will prevent unexpected breaking changes if your Deployment gets rescheduled. The DNS Provider type should match what is provisioned through External DNS for example Route53 or DNSimple. The GitHub Callback Domain is used to set up webhooks with GitHub. We only need the domain name here, not the schema. We set up SSL by default through CertManager. More information can be found here.

You can check on the various controllers here:

kubectl get deploy -n hlnr-system

Using Heighliner

It’s part of our mission with Heighliner to provide a remedy for YAML fatigue by significantly reducing the amount of YAML you would have to write. An average service is 200 LOC, with Heighliner this can be as little as 50 LOC. With that in mind, let’s write some YAML to use Heighliner to deploy a hello world app.

Github Repository

Now that we have Heighliner installed, we can set up a GitHub Repository CRD. Our GitHub Repository CRD declares the GitHub repo we want to connect to and populate release events from. First we will need a Github API token for the repo that lets us set up and access webhooks for the repo we want to listen. Create a token according to these instructions.

# This secret is required to connect to your GitHub repository.
apiVersion: v1
kind: Secret
metadata:
  name: hello-hlnr
type: Opaque
data:
  # Example: "GH API token" Base64 encoded
  GITHUB_AUTH_TOKEN: YmFzZTY0IGVuY29kZWQgR0ggQVBJIHRva2Vu

---

apiVersion: hlnr.io/v1alpha1
kind: GitHubRepository
metadata:
  name: Heighliner
  namespace: repos
spec:
  maxAvailable: 1
  repo: Heighliner
  owner: manifoldco
  configSecret:
    name: hello-hlnr # this is your secret where you have your GITHUB_AUTH_TOKEN defined

Once installed we can have a look at our GitHub Repository CRD with the alias ghr using kubectl.

kubectl get ghr -n repos -o yaml

Once you run this command you will see under the status: keys, the various pull requests, pre-releases, and releases you have made. It will also show the configured webhook.

Preview Service

Now we can write our Preview Service for previewing Pull Requests.

# Create a namespace that can hold all of out preview goodness.
apiVersion: v1
kind: Namespace
metadata:
  name: previews

---

# Create a versioning policy that tells us the versions we want to deploy from.
# Currently we have major, minor, and patch for tag versions and preview, candidate, and release for release levels.
# Example of tag versions: manifold.co/hello-hlnr:1.2.3 (1 is the major number, 2 the minor, and 3 is the patch)
# For more information see: https://semver.org/
# Preview refers to a Github pull request and candidate is a Github pre-release and release is a Github release
apiVersion: hlnr.io/v1alpha1
kind: VersioningPolicy
metadata:
  name: preview-minor
  namespace: previews
spec:
  semVer:
    version: minor
    level: preview

---

# Create the microservice that uses a Network Policy to expose and Image to the outside world
# The microservice crd can link a [whole lot more to make your app highly available and secure](https://github.com/manifoldco/Heighliner/blob/master/_examples/microservice.yaml)
apiVersion: hlnr.io/v1alpha1
kind: Microservice
metadata:
  name: hello-hlnr
  namespace: previews
spec:
  networkPolicy:
    name: hello-hlnr
  imagePolicy:
    name: hello-hlnr

---

# Create the image policy that defines which container we should use and which repo it is linked to.
apiVersion: hlnr.io/v1alpha1
kind: ImagePolicy
metadata:
  name: hello-hlnr
  namespace: previews
spec:
  image: manifoldco/hello-hlnr
  imagePullSecrets:
  - name: manifold-docker-registry
  versioningPolicy:
    name: preview-minor
  filter:
    github:
      name: Heighliner
      namespace: repos

---

# Create the NetworkPolicy that auto-configures services and ingresses with a templated domain name for our previews
apiVersion: hlnr.io/v1alpha1
kind: NetworkPolicy
metadata:
  name: hello-hlnr
  namespace: previews
spec:
  microservice:
    name: hello-hlnr
  ports:
  - name: headless
    port: 80
    targetPort: 8080
  externalDNS:
  - domain: "{{.StreamName}}.arigato.tools"
    port: headless
    disableTLS: true # lets disable TLS for out previews :)
  updateStrategy:
    latest: {}

Monitoring Heighliner Status

Using aliases and kubectl we can monitor Heighliner status. Once applied, we can look at the various CRDs:

Image Policy

Use ip for image policy, which declares which image and repo to use to make releases; reports releases in status.

$ kubectl get ip -n previews -o yaml
# an image policy spec will be displayed here ...
  status:
    releases:
    - image: manifoldco/hello-hlnr:4028fdd8c416f5ffd29150425d899a0d95d5aa2e
      level: preview
      releaseTime: 2018-08-14T13:46:04Z
      semVer:
        name: my-first-pr
        version: 4028fdd8c416f5ffd29150425d899a0d95d5aa2e
    - image: manifoldco/hello-hlnr:3c6d5830f7bce8c774f4b07069a16e4a2470e280
      level: preview
      releaseTime: 2018-08-14T15:12:44Z
      semVer:
        name: my-second-pr
        version: 3c6d5830f7bce8c774f4b07069a16e4a2470e280

GitHub Repository

Use ghr for GitHub repository, which declares which repo to use; reports GitHub previews, pre-releases and releases in status.

$ kubectl get ghr -n previews -o yaml
# a github policy spec will be displayed here...
  status:
    releases:
    -  level: preview
      name: my-first-pr
      releaseTime: 2018-08-14T13:46:04Z
      tag: 4028fdd8c416f5ffd29150425d899a0d95d5aa2e
    - level: my-second-pr
      name: mack/integrations-meta-image-fix
      releaseTime: 2018-08-14T15:12:44Z
      tag: 3c6d5830f7bce8c774f4b07069a16e4a2470e280

Heighliner Network Policy

Use hnp for Heighliner network policy, which declares which domain to point the application at; reports those domains attached to releases

$ kubectl get hnp -n previews -o yaml
# a Heighliner network policy will be displayed here
  domains:
  - semVer:
      Name: my-first-pr
      version: 4028fdd8c416f5ffd29150425d899a0d95d5aa2e
    url: https://hello-hlnr-pr-murlfjkj.previews.mywebsite
  - semVer:
      name: my-second-pr
      version: 3c6d5830f7bce8c774f4b07069a16e4a2470e280
    url: https://hello-hlnr-pr-q52nabm3.previews.mywebsite

Microservice and Versioned Microservice

Use msvc and vsvc to refer to Microservice and Versioned Microservice. Thse CRDs are explained in a lot more depth in Introducing Heighliner.

They are useful to check up on with kubectl because they provide a summary of various aggregated releases that are going to be made (msvc) and the fully configured deployment that is going to be made out of those releases (vscv).

Staging and Production

The beauty of Heighliner is its simplicity of configuration, use, reuse, and defaults. To move our application between stage and production we just change the version policy from previews to candidate (in the case of pre-releases) or release (in the cases of releases).

To see what these would look like in action go to staging example or production example.

Limitations

Currently, Heighliner is limited to HTTP applications only there are a few other minor issues that you can look at in the issues section of our repo that we are actively working on. Please feel free to raise issues with any feedback you have https://github.com/manifoldco/Heighliner/issues.