Automate App Deployment updates on Kubernetes using Keel

Posted on 143 views

Kubernetes usually as k8s or Kube is a perfect container orchestration system used for automation and management. It was initially developed by Google but is currently a community project. Recently, the popularity of Kubernetes and its ecosystem has grown immensely due to its ability to its workload types, design patterns, and behavior.

The basic resource in Kubernetes is called a pod. This is the smallest deployable unit which is a wrapper around containers. A pod may be made up of one or more containers each bearing its own configurations.

There are 3 different resources provided when deploying pods. These are:

  • Deployments: the easiest and most used resource. They are usually used for stateless applications. However, the application can be made stateful by attaching a persistent volume to it.
  • StatefulSets: This resource is used to manage the deployment and scaling of a set of Pods. It provides the guarantee about ordering and uniqueness of these Pods.
  • DaemonSets: This controller ensures all the pod runs on all the nodes of the cluster. In case a node is added/removed from the cluster, DaemonSet automatically adds or removes the pod.

Updating deployments can be done manually by:

  • Update image tag in deployment.yaml
  • Apply the manifest kubectl apply -f deployment.yaml

This method is time-consuming and does not give ample time for the user to focus on other important tasks like writing code, tests e.t.c. Today, there are many tools that can be used to build container images, these include Kaniko, Img image builder e.t.c. But there has been a missing gap on how to automatically apply these updates when new images are available.

Keel aims to provide a simple, robust, background service that automatically updates deployments, allowing users to focus on some other tasks. This is important, especially in situations where your application is re-build and needs to be deployed each time the code changes.

The below image will help you understand how Keel works.

Automate-App-Deployment-updates-on-Kubernetes-using-Keel-8-1024x428

In this guide, we will discuss how to automate app deployment updates on Kubernetes using Keel.

Getting Started.

Since this guide demonstrates how to automatically update deployments on Kubernetes, you need to be familiar with how to build and push images to your container registry such as Docker Hub.

Therefore you will require:

  • Github repository with a Dockerfile: we will use the repo URL as the path of the Dockerfile
  • Docker hub account: to be able to authenticate and push the Docker image.
  • Access to Kubernetes cluster: to be able to deploy the Kaniko pod and create the docker registry secret.
  • Webhook Relay – will relay public webhooks to our internal Kubernetes environment so we don’t have to expose Keel to the public internet

You also need a Kubernetes cluster set up. This page provides several options on how to set a Kubernetes cluster:

You need kubectl as well.

curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
sudo mv kubectl /usr/local/bin

Enable access to the cluster:

# For k0s
export KUBECONFIG=/var/lib/k0s/pki/admin.conf

# For Vanilla Kubernetes
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Confirm kubectl works by listing available namespaces:

$ kubectl get ns
NAME              STATUS   AGE
default           Active   18d
demo              Active   2m3s
kube-flannel      Active   18d
kube-node-lease   Active   18d
kube-public       Active   18d
kube-system       Active   18d
metallb-system    Active   18d
web               Active   18d

Step 1 – Set up GitHub repository

For this guide, we will have a simple Go application that will be built and updated automatically. The private git repository will be created with the 2 files below:

  • Dockerfile
FROM golang:1.17-alpine as build-env
 
# Set environment variable
ENV APP_NAME sample-dockerize-app
ENV CMD_PATH main.go
 
# Copy application data into image
COPY . $GOPATH/src/$APP_NAME
WORKDIR $GOPATH/src/$APP_NAME
 
# Budild application
RUN CGO_ENABLED=0 go build -v -o /$APP_NAME $GOPATH/src/$APP_NAME/$CMD_PATH
 
# Run Stage
FROM alpine:3.14
 
# Set environment variable
ENV APP_NAME sample-dockerize-app
 
# Copy only required data into this image
COPY --from=build-env /$APP_NAME .
 
# Expose application port
EXPOSE 8081
 
# Start app
CMD ./$APP_NAME
  • main.go
package main

import (
    "fmt"
    "log"
    "net/http"
)

var version = "version1"

func main() 
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) 
        fmt.Fprintf(w, "Welcome to my website! Version %s", version)
    )
    fmt.Printf("App is starting, version: %s \n", version)
    log.Fatal(http.ListenAndServe(":8081", nil))

This application will be used to print the version to help us identify the automatic image update. The files should be created on Github.

An example showing how the files should appear:

Automate-App-Deployment-updates-on-Kubernetes-using-Keel

Step 2 – Enable Webhook Relay forwarding

Triggers are entry points into the Keel. They collect information regarding updated images and send events to providers.

Creating a Webhook Relay operator allows us to create a public point and destination where webhooks are forwarded.

To keep things isolated we will run them in a separate namespace. Create the namespace with the command:

kubectl create namespace push-workflow

Now set the created namespace as the current context.

kubectl config set-context $(kubectl config current-context) --namespace=push-workflow

Install Helm charts using the aid in the below guide:

Add the repository:

helm repo add webhookrelay https://charts.webhookrelay.com
helm repo update

Obtain access tokens from the webhook Relay page. While on this page, navigate to your profile and click ‘Create Token‘. You will have the token generated.

Export the token as variables

export RELAY_KEY=0***9cc-1765-4***-b968-397***1c6
export RELAY_SECRET=Qsr1R****U3

Install the tokens using Helm:

helm upgrade --install webhookrelay-operator --namespace=push-workflow webhookrelay/webhookrelay-operator \
  --set credentials.key=$RELAY_KEY --set credentials.secret=$RELAY_SECRET

Check if the application has been installed.

$ helm list
NAME                 	NAMESPACE    	REVISION	UPDATED                                	STATUS  	CHART                      APP VERSION
webhookrelay-operator	push-workflow	1       	2022-06-17 11:10:18.377937954 +0000 UTC	deployed	webhookrelay-operator-0.3.10.5.1      

Now create a manifest that will allow you receive and forward hooks.

vim webhookrelay_cr.yaml

Add the below lines to the file.

apiVersion: forward.webhookrelay.com/v1
kind: WebhookRelayForward
metadata:
  name: keel-forward
spec:
  buckets:
  - name: dockerhub-to-keel
    inputs:
    - name: dockerhub-endpoint
      description: "Public endpoint"
      responseBody: "OK"
      responseStatusCode: 200
    outputs:
    - name: keel
      destination: http://keel:9300/v1/webhooks/dockerhub
      internal: true

Apply the manifest

kubectl apply -f webhookrelay_cr.yaml

You should have the 2 pods running;

# kubectl get pods
NAME                                           READY   STATUS      RESTARTS   AGE
keel-forward-whr-deployment-5d8db5b4b7-lwkbr   1/1     Running     0          21s
webhookrelay-operator-c694f6c8b-msv86          1/1     Running     0          112s

Obtain the endpoint by describing the pod, say:

$ kubectl describe webhookrelayforwards.forward.webhookrelay.com keel-forward
...
...
Status:
  Agent Status:  Running
  Public Endpoints:
    https://n5jmxkbviuucl395nwjxr7.hooks.webhookrelay.com
  Ready:           true
  Routing Status:  Configured

Alternatively, obtain the public endpoint from the buckets page.

Automate-App-Deployment-updates-on-Kubernetes-using-Keel-7

Step 3 – Configure DockerHub

Now create a new repository on DockerHub. This can be public or private depending on your preference.

Automate-App-Deployment-updates-on-Kubernetes-using-Keel-1

Now configure the repository Webhooks by creating a new one (dockerhub-to-keel) using the public endpoint.

Automate-App-Deployment-updates-on-Kubernetes-using-Keel-2-1024x430

Step 4 – Build and Push Container Image

There are several guides to help you build and push docker images to your container registry. In this guide, we will use Kaniko deployed using the aid of the guide below.

You can use the guide to deploy the DockerHub secrets and proceed as below.

Create the manifest:

vim pod.yml

The file will contain the lines:

apiVersion: v1
kind: Pod
metadata:
  name: kaniko
spec:
  containers:
  - name: kaniko
    image: gcr.io/kaniko-project/executor:latest
    args:
    - "--context=git://@github.com/computingpost/kubernetes-kaniko.git#refs/heads/master"
    - "--destination=/keel-demo-image:latest"
    volumeMounts:
    - name: kaniko-secret
      mountPath: /kaniko/.docker
  restartPolicy: Never
  volumes:
  - name: kaniko-secret
    secret:
      secretName: dockercred
      items:
        - key: .dockerconfigjson
          path: config.json

This manifest will help us create and push the image to Dockerhub with the tag keel-demo-image:latest. Apply the manifest.

kubectl apply -f pod.yml

Follow the build process.

$ kubectl logs kaniko --follow
Enumerating objects: 28, done.
Counting objects: 100% (28/28), done.
Compressing objects: 100% (25/25), done.
Total 28 (delta 4), reused 0 (delta 0), pack-reused 0
INFO[0001] Resolved base name golang:1.17-alpine to build-env 
INFO[0001] Retrieving image manifest golang:1.17-alpine 
INFO[0001] Retrieving image golang:1.17-alpine from registry index.docker.io 
INFO[0002] Retrieving image manifest alpine:3.14        
INFO[0002] Retrieving image alpine:3.14 from registry index.docker.io 
INFO[0004] Built cross stage deps: map[0:[/sample-dockerize-app]] 
INFO[0004] Retrieving image manifest golang:1.17-alpine 
INFO[0004] Returning cached image manifest              
INFO[0004] Executing 0 build triggers                   
.......
INFO[0019] ENV APP_NAME sample-dockerize-app            
INFO[0019] COPY --from=build-env /$APP_NAME .           
INFO[0019] Taking snapshot of files...                  
INFO[0019] EXPOSE 8081                                  
INFO[0019] cmd: EXPOSE                                  
INFO[0019] Adding exposed port: 8081/tcp                
INFO[0019] CMD ./$APP_NAME                              
INFO[0019] Pushing image to klinsmann1/keel-demo-image:latest 
INFO[0023] Pushed index.docker.io/klinsmann1/[email protected]:51d44171d79919df45e27c61ef929bf48fb2cabcade724c5a76cda501ad4303b 

Now you should have the image pushed to Dockerhub and tagged as latest

Automate-App-Deployment-updates-on-Kubernetes-using-Keel-3

Step 5 – Deploy Keel on Kubernetes.

Using Helm charts, we can easily deploy keel with the below steps:

Add the chart repo:

helm repo add keel https://charts.keel.sh 
helm repo update

When working regularly with Kubernetes manifests, you need to install Keel without Helm provider support as shown:

helm upgrade --install keel --namespace=push-workflow keel/keel --set helmProvider.enabled="false" --set service.enabled="true" --set service.type="ClusterIP"

Step 6 – Using Keel to Automate App Deployment updates.

When using Keel, policies are used to define how and when you want the application update to occur. Although providers may have different methods of getting the configuration for your applications, policies are similar across them all. The available Keel policies are;

  • all: this is used to initiate updates whenever a new version bump or a new prerelease is created. For example 1.0.0 -> 1.0.1-rc1
  • major: this updates major, minor, and patch versions
  • minor: updates minor and patch versions but ignores major versions.
  • patch: updates patch versions only and ignore minor and major versions.
  • force: this forces the update even when the tag is not semver. For example latest. You can also enforce a tag say keel.sh/match-tag=true
  • glob: use wildcards to match versions

Policies are specified with a special annotation. For example:

...........
  annotations:
      keel.sh/policy: minor     # update policy (available: patch, minor, major, all, force)
      keel.sh/trigger: poll     # enable active repository checking (webhooks and GCR would still work)
      keel.sh/approvals: "1"    # required approvals to update
      keel.sh/match-tag: "true" # only makes a difference when used with 'force' policy, will only update if tag matches :dev->:dev, :prod->:prod 
      keel.sh/pollSchedule: "@every 1m"
      keel.sh/notify: chan1,chan2  # chat channels to sent notification to

Deploy Your Application

Create a manifest to use the docker image pushed to DockerHub:

vim deployment.yaml

Add the below lines to the file specifying the keel policies as required.

apiVersion: apps/v1
kind: Deployment
metadata: 
  name: demo  
  labels: 
    name: "demo"
  annotations:
    # force policy will ensure that deployment is updated
    # even when tag is unchanged (latest remains)
    keel.sh/policy: force
spec:
  replicas: 1
  revisionHistoryLimit: 5
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      name: demo
      labels:
        app: demo
    spec:     
      containers:                    
        - image: klinsmann1/keel-demo-image:latest
          imagePullPolicy: Always # this is required to force pull image     
          name: demo
          ports:
            - containerPort: 8081
          livenessProbe:
            httpGet:
              path: /
              port: 8081
            initialDelaySeconds: 10
            timeoutSeconds: 5

Apply the manifest.

kubectl create -f deployment.yaml

Verify if everything is okay.

$ kubectl get pods
NAME                                           READY   STATUS      RESTARTS   AGE
demo-66db46dfd-7x6bw                           1/1     Running     0          9s
kaniko                                         0/1     Completed   0          103s
keel-65fcb79c6d-n7vz4                          1/1     Running     0          6m23s
keel-forward-whr-deployment-5d8db5b4b7-kzdzn   1/1     Running     0          11m
webhookrelay-operator-c694f6c8b-4vrpc          1/1     Running     0          13m

$ kubectl logs demo-66db46dfd-7x6bw 
App is starting, version: version1 

Step 7 – Update and Push Container Image

Now we will edit our main.go code on Github, and build and push a newer image with the program’s version string to version2

package main

import (
    "fmt"
    "log"
    "net/http"
)

var version = "version2"

func main() 
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) 
        fmt.Fprintf(w, "Welcome to my website! Version %s", version)
    )
    fmt.Printf("App is starting, version: %s \n", version)
    log.Fatal(http.ListenAndServe(":8080", nil))

Build and push the image by running the same kaniko manifest in step 4. The commands used here are:

kubectl delete pod kaniko
kubectl apply -f pod.yml

The new image will be pushed to Dockerhub. You can fill the process with the command:

kubectl logs kaniko --follow

Step 8 – Verify Updates on your Kubernetes App

Once a new version is built and pushed, Keel automatically updates the application. To verify this, view the deployment history of the application.

kubectl rollout history deployment/demo

Sample Output:

Automate-App-Deployment-updates-on-Kubernetes-using-Keel-5

Check if the pod is running.

$ kubectl get pods
NAME                                           READY   STATUS      RESTARTS   AGE
demo-86df5f4b76-p76th                          1/1     Running     0          103s
kaniko                                         0/1     Completed   0          2m9s
keel-65fcb79c6d-n7vz4                          1/1     Running     0          11m
keel-forward-whr-deployment-5d8db5b4b7-kzdzn   1/1     Running     0          16m
webhookrelay-operator-c694f6c8b-4vrpc          1/1     Running     0          18m

Now verify if the update was automated.

kubectl logs demo-86df5f4b76-p76th

Automate-App-Deployment-updates-on-Kubernetes-using-Keel-6

Conclusion.

It is that simple!

Now you can easily automate app deployment updates on Kubernetes using Keel. I hope this was of value.

coffee

Gravatar Image
A systems engineer with excellent skills in systems administration, cloud computing, systems deployment, virtualization, containers, and a certified ethical hacker.