1. Overview
In this tutorial, we’ll learn about the init containers in Kubernetes.
2. Init Containers in Kubernetes
In Kubernetes, each pod can have more than one container. These containers within the same pod work together to accomplish a common goal or provide a cohesive set of functionalities. For example, there can be a container that runs the main application and another “sidecar” container that ships the log to a remote host.
There are two types of containers that we can run within a pod, namely the main containers and the initialization containers (or init containers).
Specifically, within the Pod specification, there is a containers field that accepts a list of containers definition for the normal containers and the initContainers field that defines the list of initialization containers. For example, here’s a sample pod specification that defines a main container and an init container for the pod:
$ cat static-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: static-web
spec:
initContainers:
# an init containers that create directory
- name: init-create-dir
image: alpine
command: ['sh', '-c', 'sleep 5 && echo "creating dir" && mkdir -p opt/log']
volumeMounts:
- name: data
mountPath: /opt
containers:
# an app
- name: app
image: alpine
command: ['sh', '-c', 'echo "app is running" && sleep infinity']
volumeMounts:
- name: data
mountPath: /opt
volumes:
- name: data
emptyDir: {}
Note that both the initContainers and containers field shares the same Container specification.
2.1. Purpose of Init Containers
One common use case of the init containers is to run a script that blocks until certain preconditions are true. This is because the init containers will always be the first containers to run within the pods. Additionally, the main container of the pod will not start if the init containers do not complete successfully. For instance, we can ping a dependent service in init containers to make sure it’s alive before we start our main workload in the pod.
Besides that, we can also use init containers to configure the network stack and populate data in the volume for the main workload of the pod. This is doable thanks to the fact that containers within the same pod share the same network namespace and volume device. Therefore, the changes made by the init containers on the network namespace and volume device will be persisted throughout the whole lifecycle of a single pod.
For example, the Istio project, a popular open-source service mesh software, uses the init containers mechanism to configure the network stack of the pod as part of the initialization process. Specifically, it deploys an init container to configure the iptables of the pod to intercept all the incoming and outgoing traffic of the main application container.
2.2. Advantages of Init Containers
Firstly, using the init containers for initialization brings security benefits. Specifically, we can reduce the number of binaries we need for initialization purposes by placing them in the init containers instead of the main containers. This greatly reduces the attack surface due to the ephemeral nature of the init containers.
For example, as part of our initialization process, we might need to run a curl command. However, including the curl binary in our main application containers will not be ideal. By doing this, our main application will only be secure as the curl command. Any 0-day exploits discovered on the curl command would cause the main containers to be at risk as well. On the other hand, placing the curl command in the init containers would remove this concern as the init containers are only run in the startup phase and will not be a relatively huge security concern.
Besides that, an initialization task that requires elevated permissions can also greatly reduce the security risk by granting permission only to the init containers. For example, some initialization task that configures the iptables requires CAP_NET_ADMIN and CAP_NET_RAW permission. Granting these permissions on long-running, traffic-facing main containers will definitely raise red flags. By granting them only to the init containers for initialization purposes, we can effectively reduce the need to grant these permissions to the main containers.
3. Characteristics of Init Containers
There are a few characteristics of the init containers that differentiate them from the normal main containers. Let’s look at them in detail.
3.1. The Execution Order
One distinctive characteristic of the init containers is their execution order. Specifically, the list of init containers will run in a sequential manner according to the order they appear in the initContainer list. Additionally, failure in one of the init containers will either stop or restart the entire pod start-up sequence, depending on the restartPolicy of the pod.
On the other hand, the main containers will start concurrently without waiting for one another. Though the pod will only consider a pod ready if all of its main containers are running successfully.
3.2. Unsupported Fields of Container Specification
Despite sharing the same Container specification, the initContainers do not support the lifecycle, livenessProbe, readinessProbe, and startupProbe fields. Setting values for these fields on initContainers is a validation error and will invalidate the whole pod manifest:
$ kubectl apply -f init-container-with-livenessprobe.yaml
The Pod "static-web" is invalid: spec.initContainers[0].livenessProbe: Invalid value: core.Probe{ProbeHandler:core.ProbeHandler{Exec:(*core.ExecAction)(0xc0086d01c8), HTTPGet:(*core.HTTPGetAction)(nil), TCPSocket:(*core.TCPSocketAction)(nil), GRPC:(*core.GRPCAction)(nil)}, InitialDelaySeconds:0, TimeoutSeconds:1, PeriodSeconds:10, SuccessThreshold:1, FailureThreshold:3, TerminationGracePeriodSeconds:(*int64)(nil)}: must not be set for init containers
The command above applies the init-container-with-livenessprobe.yaml manifest, which set a value to the livenessProbe field of the initContainers. As we can see from the output, it results in a validation error with the error message “must not be set for init containers”.
This is because these fields are meant for configuring the behavior of the containers when the pod is in the Running state. Since the init containers are run to completion and discarded before the pod can be in the Running state, these fields do not carry any meaning for the init containers.
3.3. Restart Policy
The restartPolicy field of the pod defines on what condition should the containers within the pod restart themselves. There are three possible values: Always, OnFailure, and Never. When we set the restartPolicy of a pod to Always, the init containers’ restartPolicy will be set to OnFailure. This is because the Always policy restarts the containers even with a zero exit code.
However, the Always policy contradicts the expectation of an init container, which is to exit with a zero exit code. Therefore, both the Always and OnFailure restartPolicy values set on the pod level will map to the OnFailure value for init containers.
4. Lifecycle of Init Containers
Creating and running init containers are the first step for bringing a pod to life. Therefore, understanding the lifecycle of the init containers can help in understanding and debugging issues related to the readiness of a pod.
We’ll first look at the steps that an init container goes through, and then we perform a demonstration using a simple example.
4.1. Starting up the Init Containers
Firstly, the process begins when we create a pod resource on the Kubernetes cluster. The creation of the pod can be either directly using the Pod specification or indirectly through the Deployment, StatefulSet, or DaemonSet specification. Either way, the kube-scheduler will schedule the pod on a node. Then, the kubelet process on the node will create and starts the init containers. Due to the init containers’ sequential execution order, the process will run the first init container and observe its exit code before deciding on the next step.
4.2. On Init Containers Failure
When the init container returns a non-zero exit code, the startup process either fails the startup or restarts the process, depending on the restartPolicy of the pod. If we set the restartPolicy to Never, the pod status will change to Failed, and the start-up process stops at this point.
Alternatively, for the restartPolicy value of Always or OnFailure, the whole process will be restarted, starting from the first init containers in the list. Because of the possibility of multiple executions of init containers due to restarts, it’s important to design our init containers such that it’s idempotent.
Furthermore, if there’s a failure in the init containers, the status of the pod will turn to Init:Error. This tells us that the pod fails to startup due to errors in the init containers. When the same error occurs multiple times, the status will then turn into Init:CrashedLoopBackOff.
4.3. On Init Containers Success
On the other hand, a zero exit code of an init container signifies that the initialization task is successful. When that happens, the pod start-up process proceeds to the next init container in the list. Besides that, the kubelet process will update the status of the Pod to Init:N/M, where N is the number of init containers that have succeeded, and M is the total number of init containers defined. This process goes on until all the init containers run to completion with a zero exit code.
When all the init containers have been completed successfully, the pod status will be set to PodInitializing. Beyond this point, the responsibility of the init containers officially ends, and the main containers startup process will begin.
5. Init Container Lifecycle Demonstration
Let’s look at the details of the lifecycle of the init containers with an example:
$ cat static-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: static-web
spec:
initContainers:
# an init containers that create directories
- name: init-create-dir
image: alpine
command: ['sh', '-c', 'sleep 5 && echo "creating dir"']
# an init containers that create a config file
- name: init-create-config
image: alpine
command: ['sh', '-c', 'sleep 5 && echo "creating file"']
containers:
# an app
- name: app
image: alpine
command: ['sh', '-c', 'echo "app is running" && sleep infinity']
# a hypothetical log transformer
- name: log-transformer
image: alpine
command: ['sh', '-c', 'echo "transforming log" && sleep infinity']
The static-pod.yaml manifest above shows a simple example. It defines two init containers, init-create-dir and init-create-config, that simulate the initialization tasks. Additionally, we run two main containers on the pod, app and log-transformer. We apply the sleep command in the init containers so that we have enough time to observe the state change.
5.1. Creating the Pod
Firstly, we apply the pod spec using the command kubectl apply to create the pod:
$ kubectl apply -f static-pod.yaml
pod/static-web created
After we run the kubectl apply on the manifest, the controller will schedule the pod to a node and bring it up.
5.2. Observing Status Using kubectl describe
To get the status of the pod, we can use the kubectl describe command:
$ kubectl describe pod static-web
Name: static-web
Namespace: default
Priority: 0
...
Init Containers:
init-create-dir:
Container ID:
Image: alpine
...
init-create-config:
Container ID:
Image: alpine
...
Containers:
app:
Container ID:
Image: alpine
...
log-transformer:
Container ID:
Image: alpine
...
The output of the kubectl describe command contains all the information for the pod. For our demonstration, we’ll only focus on the InitContainers, Containers, and Conditions fields. Specifically, we’ll observe the State field of the InitContainers and Containers to observe the changes.
Right after we apply the pod manifest against the Kubernetes cluster, we’ll see that all of the containers in both fields are in the Waiting state.
Then, we’ll see the first init container, the init-create-dir start running with the other containers remaining in their Waiting state:
$ kubectl describe pod static-web
Name: static-web
...
Init Containers:
init-create-dir:
...
State: Running
Started: Sun, 21 May 2023 11:32:29 +0800
...
init-create-config:
...
State: Waiting
Reason: PodInitializing
...
Containers:
app:
...
State: Waiting
Reason: PodInitializing
...
log-transformer:
...
State: Waiting
Reason: PodInitializing
...
Once the first init containers run to completion, the state will change to Terminated, and the second init container, init-create-config, will start running:
$ kubectl describe pod static-web
Name: static-web
...
Init Containers:
init-create-dir:
...
State: Terminated
Reason: Completed
Exit Code: 0
...
init-create-config:
...
State: Running
Started: Sun, 21 May 2023 11:32:29 +0800
...
Finally, when both the init containers have successfully executed their tasks, both the main containers will run concurrently. Additionally, the Initialized field of the pod’s Condition will be set to True. This signifies the completion of the init containers lifecycle:
$ kubectl describe pod static-web
Name: static-web
...
Init Containers:
init-create-dir:
...
State: Terminated
Reason: Completed
Exit Code: 0
...
init-create-config:
...
State: Terminated
Reason: Completed
Exit Code: 0
...
Containers:
app:
...
State: Running
Started: Sun, 21 May 2023 11:32:40 +0800
...
log-transformer:
...
State: Running
Started: Sun, 21 May 2023 11:32:40 +0800
...
Furthermore, at the end of the init containers lifecycle, we can see that the Initialized field of the pod will change to True to indicate the completion of the initialization process:
Conditions:
Type Status
Initialized True
...
6. Conclusion
In this article, we’ve learned about the init containers in Kubernetes. Firstly, we’ve learned that init containers are used for initializing a pod. Then, we’ve looked at some benefits of using init containers for initialization, such as a reduced surface of attack. Additionally, we’ve explored the characteristics of init containers in detail. For instance, we’ve learned about the sequential execution manner of the init containers. Additionally, we’ve also seen how several fields of containers are not applicable for init containers.
Finally, we learned about the lifecycle of the init containers. Specifically, we’ve looked at an actual demonstration of a pod specification with two init containers and two main containers.