Immutable Configmaps Improve Performance

👋 Hi! I’m Bibin Wilson. In each edition, I share practical tips, guides, and the latest trends in DevOps and MLOps to make your day-to-day DevOps tasks more efficient. If someone forwarded this email to you, you can subscribe here to never miss out!

In Kubernetes, an Immutable ConfigMap is a regular ConfigMap with the immutable field set to true.

This setting prevents the ConfigMap from being modified after creation. If you try to update it, it will be rejected by the API server.

To make any changes, you must delete and recreate the ConfigMap.

To understand the performance improvement with immutable ConfigMaps, you should first understand how ConfigMaps and their updates are handled by the Kubelet.

Kubelet & ConfigMap

Kubelet, the agent that runs on each node, plays a important role in ensuring that the configurations defined in ConfigMaps are kept in sync with the pods that consume them.

When a pod is first created, Kubelet fetches the ConfigMap specified in the pod's configuration.

Kubelet periodically synchronizes the ConfigMap data with the pod's filesystem. This ensures that the data remains up to date if the ConfigMap is updated.

Kubelet checks ConfigMaps based on the syncFrequency setting in the KubeletConfiguration file.

By default, this is set to 1 minute, meaning Kubelet checks for updates to ConfigMaps every 1 minute. If any changes are detected, Kubelet updates the running containers with the new configurations.

Mutable Configmaps and Performance Impact

Now think of large production clusters with thousands of configmaps?

Kubelets has to continuously watch for changes which comes with a cost.

Normally, for regular (mutable) ConfigMaps, the kube-apiserver and Kubelet keep a continuous connection (via the kube-apiserver’s watch API) to detect updates. These watches consume resources, especially when there are many ConfigMaps or frequent updates.

If a Pod mounts a ConfigMap as a volume, the kubelet checks its local cache to determine the current state of the ConfigMap during the sync cycle.

If the cache is fresh (i.e., up-to-date), it uses the cached data and if the cache is stale, it queries the kube-apiserver for the latest version.

Performance Benefits With Immutable Configmaps

When a ConfigMap is marked as immutable, kubelet no longer needs to watch for changes to that ConfigMap.

By making a ConfigMap immutable, you can reduce the load on the kube-apiserver, as it doesn’t need to handle unnecessary update checks.

This means, the local cache can hold the immutable ConfigMap indefinitely without worrying about staleness, as the data will never change.

In small clusters with less workloads, this performance impact is negligible.

However, immutable ConfigMap is particularly beneficial in large clusters with many Pods consuming the same ConfigMap, as it minimizes resource usage.

Wrapping up

Hope I was able to help you learn something new this week!

If you have any questions, feedback, or suggestions, feel free to reach out at [email protected].

Happy Weekend! 🎉