CloudWatch is a service which collects operational and monitoring data in the form of logs, metrics, and events in AWS Cloud platform. It then visualizes the data by using automated dashboards so you can get a unified view of your AWS resources, applications, and services that run in AWS and on-premises.
After deployment of EKS or Kubernetes cluster, there is a need to configure Container Insights to send data to CloudWatch.
A pre-requisite for this setup is a functional Amazon EKS or Kubernetes cluster. If you need reference installation posts we have them available in our website.
Easily Setup Kubernetes Cluster on AWS with EKS
Install Kubernetes Cluster on Ubuntu with kubeadm
Install Kubernetes Cluster on CentOS 7 with kubeadm
Check to confirm kubectl can connect to the Kubernetes API Server.
$ kubectl get nodes NAME STATUS ROLES AGE VERSION ip-192-168-138-244.eu-west-1.compute.internal Ready
9d v1.17.9-eks-4c6976 ip-192-168-176-247.eu-west-1.compute.internal Ready 9d v1.17.9-eks-4c6976
Create and Attach a policy to the IAM role of your worker nodes
Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/ and select one of the Worker nodes.
Choose the IAM role in the description. On the IAM role page, choose Attach policies.
Select CloudWatchAgentServerPolicy checkbox and attach the policy.
Setup Container Insights on Amazon EKS
Once the policy has been attached to the EKS worker nodes role proceed to deploy Container Insights on Amazon EKS.
Save your cluster name and region as variables.
Download installation manifest:
curl -O https://raw.githubusercontent.com/aws-samples/amazon-cloudwatch-container-insights/latest/k8s-deployment-manifest-templates/deployment-mode/daemonset/container-insights-monitoring/quickstart/cwagent-fluentd-quickstart.yaml
Update the file to set correct name of cluster and region.
sed -ie "s/cluster_name/$EKS_CLUSTER_NAME/;s/region_name/$EKS_CLUSTER_REGION/" cwagent-fluentd-quickstart.yaml
Install Container Insights on Amazon EKS with the following command.
kubectl apply -f ./cwagent-fluentd-quickstart.yaml
A number of resources are created as seen in the command execution output:
namespace/amazon-cloudwatch unchanged serviceaccount/cloudwatch-agent unchanged clusterrole.rbac.authorization.k8s.io/cloudwatch-agent-role unchanged clusterrolebinding.rbac.authorization.k8s.io/cloudwatch-agent-role-binding unchanged configmap/cwagentconfig unchanged daemonset.apps/cloudwatch-agent unchanged configmap/cluster-info unchanged serviceaccount/fluentd unchanged clusterrole.rbac.authorization.k8s.io/fluentd-role unchanged clusterrolebinding.rbac.authorization.k8s.io/fluentd-role-binding unchanged configmap/fluentd-config unchanged daemonset.apps/fluentd-cloudwatch unchanged
Get status of all resources in amazon-cloudwatch namespace.
$ kubectl get all -n amazon-cloudwatch NAME READY STATUS RESTARTS AGE pod/cloudwatch-agent-lqtfw 1/1 Running 0 5m21s pod/cloudwatch-agent-lsz92 1/1 Running 0 5m21s pod/fluentd-cloudwatch-6j2jf 1/1 Running 0 5m17s pod/fluentd-cloudwatch-dzrdq 1/1 Running 0 5m17s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/cloudwatch-agent 2 2 2 2 2
5m22s daemonset.apps/fluentd-cloudwatch 2 2 2 2 2 5m18s
We can see all pods are in running state.These were created by daemon sets.
Viewing Container Insights Metrics
After you have Container Insights set up and it is collecting metrics, you can view those metrics in the CloudWatch console.
Choose Performance Monitoring in the navigation pane from where you can select resource type metrics to view.