Scaling up Without Slowing Down: Accelerating Pod Start Time

Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon North America in Salt Lake City from November 12 - 15, 2024. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at https://kubecon.io

Scaling up Without Slowing Down: Accelerating Pod Start Time - Ganeshkumar Ashokavardhanan, Microsoft & Yifan Yuan, AlibabaCloud

Cold start times of Kubernetes pods, particularly those with large container images, lead to slower scale up, inefficient deployments and increased costs. There are many open-source approaches to address this: on-demand image loading, peer-to-peer systems, pre-warming nodes and checkpoint and restore. We will show how the optimal approach varies depending on the workload type, runtime behavior and the scale of the system. For instance, deep learning inference that needs the entire model to be on the node can be optimized differently than ML training workloads, which have a gradual data access pattern. We will also discuss the latency tradeoffs during the entire pod lifecycle, and the impact of the solutions we propose on network congestion, node storage utilization and reliability. Join us as we navigate through the many open-source approaches available, to share the framework you need to decide the approach that is optimal for your Kubernetes workloads.