Introduction
In the dynamic world of Kubernetes, ensuring that critical applications run seamlessly across all nodes in a cluster is essential for maintaining system health, performance, and observability. Enter the DaemonSet, a powerful Kubernetes resource designed to streamline this process. In this blog, we will explore the concept of DaemonSets in detail, along with Jobs and CronJobs, complete with examples to illustrate their practical applications. Let's get started!
What is a DaemonSet?
A daemon set is another type of Kubernetes object that controls pods. Unlike deployment, the DS automatically deploys 1 pod to each available node. You don't need to update the replica based on demand; the DS takes care of it by spinning X number of pods for X number of nodes.
If you create a ds in a cluster of 5 nodes, then 5 pods will be created.
If you add another node to the cluster, a new pod will be automatically created on the new node.
Examples of daemonset
Logging daemons (e.g., Fluentd, Logstash)
Monitoring daemons (e.g., Prometheus Node Exporter, Datadog agent)
Network daemons (e.g., Flannel, Weave Net)
Storage daemons (e.g., Ceph, GlusterFS)
Key Features of DaemonSets
Automatic Deployment and Management:
- A DaemonSet automatically schedules a pod on all nodes that match the DaemonSet’s node selector, adding new pods as new nodes are added to the cluster, and removing pods as nodes are removed from the cluster.
Selective Deployment:
- DaemonSets can be configured to run on a subset of nodes by using node selectors or taints and tolerations. This is useful for running certain daemons only on specific types of nodes, like GPU nodes or nodes with specific hardware.
Pod Management:
- DaemonSets manage the lifecycle of the pods they create, ensuring that they are recreated if they fail and keeping the desired number of replicas running.
Creating a DaemonSet
A DaemonSet is defined using a YAML file (ds.yaml), similar to other Kubernetes objects. Here’s an example:
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: demo-ds
labels:
name: demo-ds
spec:
template:
metadata:
labels:
name: demo-ds
spec:
containers:
- name: nginx-ds
image: nginx
selector:
matchLabels:
name: demo-ds
Apply this configuration using:
kubectl apply -f ds.yaml
This will create a DaemonSet, and you will see one pod created per node.
By default, DaemonSets run pods on worker nodes and not on control plane nodes. To schedule pods on control plane nodes as well, add a toleration that allows the pods to be scheduled on nodes with the taint node-role.kubernetes.io/master:NoSchedule. We will cover this in future blogs.
To view all DaemonSets in the cluster:
kubectl get ds -A
To view DaemonSets in a specific namespace:
kubectl get ds -n <namespace-name>
CronJob and Job
Before diving into CronJobs, it’s useful to understand Cron and its syntax.
The cron utility in Unix-based operating systems is used to schedule tasks to run at specified intervals. These scheduled tasks, known as "cron jobs," can be set to run at any time or date, based on a flexible syntax. Kubernetes CronJobs use the same syntax to define the schedule.
Cron Syntax
The cron syntax consists of five fields, each representing a specific time unit. These fields are separated by spaces:
Field Values
Minute: (0 - 59) - The minute of the hour.
Hour: (0 - 23) - The hour of the day.
Day of the month: (1 - 31) - The day of the month.
Month: (1 - 12) - The month of the year.
Day of the week: (0 - 6) - The day of the week (Sunday to Saturday).
Special Characters
Asterisk (
*): Represents all possible values for the field. For example, an asterisk in the hour field means every hour.Comma (
,): Specifies a list of values. For example,1,2,5in the day of the week field means Monday, Tuesday, and Friday.Dash (
-): Specifies a range of values. For example,1-5in the hour field means every hour from 1 AM to 5 AM.Slash (
/): Specifies a step value. For example,*/15in the minute field means every 15 minutes.
Examples
Every Saturday
Every Saturday at 11 pm
Every Saturday at 11:45 pm
Every 5 minutes
CronJob in Kubernetes
A CronJob in Kubernetes is used to run jobs on a scheduled basis. It is similar to the Unix cron utility, allowing for periodic and recurring tasks such as backups, report generation, and system maintenance.
Here’s an example of a CronJob that prints "40daysofkubernetes" every minute
apiVersion: batch/v1
kind: CronJob
metadata:
name: demo-cronjob
spec:
schedule: "*/1 * * * *" # This schedule means the job will run every 1 minutes.
jobTemplate:
spec:
template:
metadata:
labels:
job: demo-cronjob
spec:
containers:
- name: print-message
image: busybox
command: ["sh", "-c", "echo '40daysofkubernetes'"]
restartPolicy: OnFailure
Create a file cronjob.yaml and apply it using:
kubectl apply -f cronjob.yaml
To view your CronJob:
kubectl get cronjobs
To see the output, check the logs of any of the pods:
kubectl logs <pod-name>
To delete the CronJob:
kubectl delete cronjob/<cronjob-name>
Job
A Job in Kubernetes is used for running a one-time task to completion. It ensures that a specified number of pods run to completion successfully. Jobs are typically used for batch processing tasks or other short-lived workloads.
Summary
Job: Runs immediately and performs a one-time task without scheduling.
CronJob: Runs Jobs on a schedule, ideal for recurring tasks.
Thank you for reading my blog. Happy learning!