Kubernetes usually as k8s or Kube is a perfect container orchestration system used for automation and management. It was initially developed by Google but is currently a community project. Recently, the popularity of Kubernetes and its ecosystem has grown immensely due to its ability to its workload types, design patterns, and behavior.
The basic resource in Kubernetes is called a pod. This is the smallest deployable unit which is a wrapper around containers. A pod may be made up of one or more containers each bearing its own configurations.
There are 3 different resources provided when deploying pods. These are:
- Deployments: the easiest and most used resource. They are usually used for stateless applications. However, the application can be made stateful by attaching a persistent volume to it.
- StatefulSets: This resource is used to manage the deployment and scaling of a set of Pods. It provides the guarantee about ordering and uniqueness of these Pods.
- DaemonSets: This controller ensures all the pod runs on all the nodes of the cluster. In case a node is added/removed from the cluster, DaemonSet automatically adds or removes the pod.
Updating deployments can be done manually by:
- Update image tag in
- Apply the manifest
kubectl apply -f deployment.yaml
This method is time-consuming and does not give ample time for the user to focus on other important tasks like writing code, tests e.t.c. Today, there are many tools that can be used to build container images, these include Kaniko, Img image builder e.t.c. But there has been a missing gap on how to automatically apply these updates when new images are available.
Keel aims to provide a simple, robust, background service that automatically updates deployments, allowing users to focus on some other tasks. This is important, especially in situations where your application is re-build and needs to be deployed each time the code changes.
The below image will help you understand how Keel works.
In this guide, we will discuss how to automate app deployment updates on Kubernetes using Keel.
Since this guide demonstrates how to automatically update deployments on Kubernetes, you need to be familiar with how to build and push images to your container registry such as Docker Hub.
Therefore you will require:
- Github repository with a Dockerfile: we will use the repo URL as the path of the Dockerfile
- Docker hub account: to be able to authenticate and push the Docker image.
- Access to Kubernetes cluster: to be able to deploy the Kaniko pod and create the docker registry secret.
- Webhook Relay – will relay public webhooks to our internal Kubernetes environment so we don’t have to expose Keel to the public internet
You also need a Kubernetes cluster set up. This page provides several options on how to set a Kubernetes cluster:
- Install Kubernetes Cluster on Rocky Linux 8 with Kubeadm & CRI-O
- Install Kubernetes Cluster on Ubuntu using K3s
- Deploy Kubernetes Cluster on Linux With k0s
- Run Kubernetes on Debian with Minikube
You need kubectl as well.
curl -LO "https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl" chmod +x kubectl sudo mv kubectl /usr/local/bin
Enable access to the cluster:
# For k0s export KUBECONFIG=/var/lib/k0s/pki/admin.conf # For Vanilla Kubernetes mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl works by listing available namespaces:
$ kubectl get ns NAME STATUS AGE default Active 18d demo Active 2m3s kube-flannel Active 18d kube-node-lease Active 18d kube-public Active 18d kube-system Active 18d metallb-system Active 18d web Active 18d
Step 1 – Set up GitHub repository
For this guide, we will have a simple Go application that will be built and updated automatically. The private git repository will be created with the 2 files below:
FROM golang:1.17-alpine as build-env # Set environment variable ENV APP_NAME sample-dockerize-app ENV CMD_PATH main.go # Copy application data into image COPY . $GOPATH/src/$APP_NAME WORKDIR $GOPATH/src/$APP_NAME # Budild application RUN CGO_ENABLED=0 go build -v -o /$APP_NAME $GOPATH/src/$APP_NAME/$CMD_PATH # Run Stage FROM alpine:3.14 # Set environment variable ENV APP_NAME sample-dockerize-app # Copy only required data into this image COPY --from=build-env /$APP_NAME . # Expose application port EXPOSE 8081 # Start app CMD ./$APP_NAME
package main import ( "fmt" "log" "net/http" ) var version = "version1" func main() http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) fmt.Fprintf(w, "Welcome to my website! Version %s", version) ) fmt.Printf("App is starting, version: %s \n", version) log.Fatal(http.ListenAndServe(":8081", nil))
This application will be used to print the version to help us identify the automatic image update. The files should be created on Github.
An example showing how the files should appear:
Step 2 – Enable Webhook Relay forwarding
Triggers are entry points into the Keel. They collect information regarding updated images and send events to providers.
Creating a Webhook Relay operator allows us to create a public point and destination where webhooks are forwarded.
To keep things isolated we will run them in a separate namespace. Create the namespace with the command:
kubectl create namespace push-workflow
Now set the created namespace as the current context.
kubectl config set-context $(kubectl config current-context) --namespace=push-workflow
Install Helm charts using the aid in the below guide:
Add the repository:
helm repo add webhookrelay https://charts.webhookrelay.com helm repo update
Obtain access tokens from the webhook Relay page. While on this page, navigate to your profile and click ‘Create Token‘. You will have the token generated.
Export the token as variables
export RELAY_KEY=0***9cc-1765-4***-b968-397***1c6 export RELAY_SECRET=Qsr1R****U3
Install the tokens using Helm:
helm upgrade --install webhookrelay-operator --namespace=push-workflow webhookrelay/webhookrelay-operator \ --set credentials.key=$RELAY_KEY --set credentials.secret=$RELAY_SECRET
Check if the application has been installed.
$ helm list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION webhookrelay-operator push-workflow 1 2022-06-17 11:10:18.377937954 +0000 UTC deployed webhookrelay-operator-0.3.10.5.1
Now create a manifest that will allow you receive and forward hooks.
Add the below lines to the file.
apiVersion: forward.webhookrelay.com/v1 kind: WebhookRelayForward metadata: name: keel-forward spec: buckets: - name: dockerhub-to-keel inputs: - name: dockerhub-endpoint description: "Public endpoint" responseBody: "OK" responseStatusCode: 200 outputs: - name: keel destination: http://keel:9300/v1/webhooks/dockerhub internal: true
Apply the manifest
kubectl apply -f webhookrelay_cr.yaml
You should have the 2 pods running;
# kubectl get pods NAME READY STATUS RESTARTS AGE keel-forward-whr-deployment-5d8db5b4b7-lwkbr 1/1 Running 0 21s webhookrelay-operator-c694f6c8b-msv86 1/1 Running 0 112s
Obtain the endpoint by describing the pod, say:
$ kubectl describe webhookrelayforwards.forward.webhookrelay.com keel-forward ... ... Status: Agent Status: Running Public Endpoints: https://n5jmxkbviuucl395nwjxr7.hooks.webhookrelay.com Ready: true Routing Status: Configured
Alternatively, obtain the public endpoint from the buckets page.
Step 3 – Configure DockerHub
Now create a new repository on DockerHub. This can be public or private depending on your preference.
Now configure the repository Webhooks by creating a new one (dockerhub-to-keel) using the public endpoint.
Step 4 – Build and Push Container Image
There are several guides to help you build and push docker images to your container registry. In this guide, we will use Kaniko deployed using the aid of the guide below.
You can use the guide to deploy the DockerHub secrets and proceed as below.
Create the manifest:
The file will contain the lines:
apiVersion: v1 kind: Pod metadata: name: kaniko spec: containers: - name: kaniko image: gcr.io/kaniko-project/executor:latest args: - "--context=git://
@github.com/computingpost/kubernetes-kaniko.git#refs/heads/master" - "--destination= /keel-demo-image:latest" volumeMounts: - name: kaniko-secret mountPath: /kaniko/.docker restartPolicy: Never volumes: - name: kaniko-secret secret: secretName: dockercred items: - key: .dockerconfigjson path: config.json
This manifest will help us create and push the image to Dockerhub with the tag keel-demo-image:latest. Apply the manifest.
kubectl apply -f pod.yml
Follow the build process.
$ kubectl logs kaniko --follow Enumerating objects: 28, done. Counting objects: 100% (28/28), done. Compressing objects: 100% (25/25), done. Total 28 (delta 4), reused 0 (delta 0), pack-reused 0 INFO Resolved base name golang:1.17-alpine to build-env INFO Retrieving image manifest golang:1.17-alpine INFO Retrieving image golang:1.17-alpine from registry index.docker.io INFO Retrieving image manifest alpine:3.14 INFO Retrieving image alpine:3.14 from registry index.docker.io INFO Built cross stage deps: map[0:[/sample-dockerize-app]] INFO Retrieving image manifest golang:1.17-alpine INFO Returning cached image manifest INFO Executing 0 build triggers ....... INFO ENV APP_NAME sample-dockerize-app INFO COPY --from=build-env /$APP_NAME . INFO Taking snapshot of files... INFO EXPOSE 8081 INFO cmd: EXPOSE INFO Adding exposed port: 8081/tcp INFO CMD ./$APP_NAME INFO Pushing image to klinsmann1/keel-demo-image:latest INFO Pushed index.docker.io/klinsmann1/[email protected]:51d44171d79919df45e27c61ef929bf48fb2cabcade724c5a76cda501ad4303b
Now you should have the image pushed to Dockerhub and tagged as latest
Step 5 – Deploy Keel on Kubernetes.
Using Helm charts, we can easily deploy keel with the below steps:
Add the chart repo:
helm repo add keel https://charts.keel.sh helm repo update
When working regularly with Kubernetes manifests, you need to install Keel without Helm provider support as shown:
helm upgrade --install keel --namespace=push-workflow keel/keel --set helmProvider.enabled="false" --set service.enabled="true" --set service.type="ClusterIP"
Step 6 – Using Keel to Automate App Deployment updates.
When using Keel, policies are used to define how and when you want the application update to occur. Although providers may have different methods of getting the configuration for your applications, policies are similar across them all. The available Keel policies are;
- all: this is used to initiate updates whenever a new version bump or a new prerelease is created. For example 1.0.0 -> 1.0.1-rc1
- major: this updates major, minor, and patch versions
- minor: updates minor and patch versions but ignores major versions.
- patch: updates patch versions only and ignore minor and major versions.
- force: this forces the update even when the tag is not semver. For example latest. You can also enforce a tag say keel.sh/match-tag=true
- glob: use wildcards to match versions
Policies are specified with a special annotation. For example:
........... annotations: keel.sh/policy: minor # update policy (available: patch, minor, major, all, force) keel.sh/trigger: poll # enable active repository checking (webhooks and GCR would still work) keel.sh/approvals: "1" # required approvals to update keel.sh/match-tag: "true" # only makes a difference when used with 'force' policy, will only update if tag matches :dev->:dev, :prod->:prod keel.sh/pollSchedule: "@every 1m" keel.sh/notify: chan1,chan2 # chat channels to sent notification to
Deploy Your Application
Create a manifest to use the docker image pushed to DockerHub:
Add the below lines to the file specifying the keel policies as required.
apiVersion: apps/v1 kind: Deployment metadata: name: demo labels: name: "demo" annotations: # force policy will ensure that deployment is updated # even when tag is unchanged (latest remains) keel.sh/policy: force spec: replicas: 1 revisionHistoryLimit: 5 selector: matchLabels: app: demo template: metadata: name: demo labels: app: demo spec: containers: - image: klinsmann1/keel-demo-image:latest imagePullPolicy: Always # this is required to force pull image name: demo ports: - containerPort: 8081 livenessProbe: httpGet: path: / port: 8081 initialDelaySeconds: 10 timeoutSeconds: 5
Apply the manifest.
kubectl create -f deployment.yaml
Verify if everything is okay.
$ kubectl get pods NAME READY STATUS RESTARTS AGE demo-66db46dfd-7x6bw 1/1 Running 0 9s kaniko 0/1 Completed 0 103s keel-65fcb79c6d-n7vz4 1/1 Running 0 6m23s keel-forward-whr-deployment-5d8db5b4b7-kzdzn 1/1 Running 0 11m webhookrelay-operator-c694f6c8b-4vrpc 1/1 Running 0 13m $ kubectl logs demo-66db46dfd-7x6bw App is starting, version: version1
Step 7 – Update and Push Container Image
Now we will edit our main.go code on Github, and build and push a newer image with the program’s version string to version2
package main import ( "fmt" "log" "net/http" ) var version = "version2" func main() http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) fmt.Fprintf(w, "Welcome to my website! Version %s", version) ) fmt.Printf("App is starting, version: %s \n", version) log.Fatal(http.ListenAndServe(":8080", nil))
Build and push the image by running the same kaniko manifest in step 4. The commands used here are:
kubectl delete pod kaniko kubectl apply -f pod.yml
The new image will be pushed to Dockerhub. You can fill the process with the command:
kubectl logs kaniko --follow
Step 8 – Verify Updates on your Kubernetes App
Once a new version is built and pushed, Keel automatically updates the application. To verify this, view the deployment history of the application.
kubectl rollout history deployment/demo
Check if the pod is running.
$ kubectl get pods NAME READY STATUS RESTARTS AGE demo-86df5f4b76-p76th 1/1 Running 0 103s kaniko 0/1 Completed 0 2m9s keel-65fcb79c6d-n7vz4 1/1 Running 0 11m keel-forward-whr-deployment-5d8db5b4b7-kzdzn 1/1 Running 0 16m webhookrelay-operator-c694f6c8b-4vrpc 1/1 Running 0 18m
Now verify if the update was automated.
kubectl logs demo-86df5f4b76-p76th
It is that simple!
Now you can easily automate app deployment updates on Kubernetes using Keel. I hope this was of value.