When Kubernetes first got here onto the scene, it was a significant turning level, a revision of the infrastructure and operations area that reworked the way in which builders and ops personnel construct, deploy, and keep purposes within the cloud. It has since grow to be the clear commonplace for the way trendy purposes are constructed and operated. Because the CNCF famous in its newest Annual Cloud Native Survey report, “Amongst container customers, 82% are utilizing Kubernetes in manufacturing in 2025, up from 66% in 2023. This represents near-universal adoption throughout the container ecosystem.”
Over the previous few years, one other revision within the area has occurred with Kubernetes’s evolution from a container orchestrator to an AI infrastructure platform. In keeping with the CNCF survey, “The rise of Kubernetes because the de facto AI platform represents a basic shift in how organizations method machine studying operations. . .[with Kubernetes] offering a unified orchestration layer that handles each conventional software workloads and compute-intensive AI duties.” The emergence of seismic applied sciences like generative AI and agentic AI has solely accelerated this transformation.
The intersection of AI with Kubernetes is undoubtedly one of the crucial impactful developments within the operations area. As Jonathan Johnson, software program architect at Dijure, observes, “AI on K8s may be very, crucial, and there may be not sufficient [resources] on the market.” Raju Gandhi, senior technical architect at Edward Jones, echoes this evaluation, noting that “operationalizing AI/ML on K8s is a giant subject, [and it’s only] getting greater. It is a subject that wants consideration.” However what are a few of the issues that it’s best to find out about this pattern to maintain abreast and keep forward within the sport?
Generative AI
Anybody with entry to a pc or a smartphone has seemingly used some iteration of generative AI, a surprising reality when you think about that GenAI was on the outer edges of mainstream discourse and consumption a scant 5 years in the past. However on the finish of 2022, the debut of ChatGPT marked the start of a technological revolution, one that may influence and reshape almost each side of our working and private lives. Unsurprisingly, there at the moment are 1000’s of generative AI fashions, a proliferation that naturally has its personal set of complexities. Deciding on a mannequin is easy, however if you happen to’re an software developer or MLOps engineer, how do you go about working that mannequin in a manufacturing system? Not solely do it’s a must to be cognizant of things like resilience, scalability, safety, and operational prices, however there’s the truth that bringing a mannequin from experimentation into manufacturing might be arduous if not performed correctly. That’s the place Kubernetes comes into play.
As Roland Huß and Daniele Zonca, distinguished engineers at Crimson Hat, word, “GenAI/LLM fashions are useful resource intensive, requiring substantial computational energy and huge datasets. Given its scalability and extensibility, Kubernetes is uniquely suited to operate as an environment friendly platform for AI and LLM mannequin pretraining, fine-tuning, deployment, and immediate engineering.” They additional elaborate that “this integration with Kubernetes not solely simplifies the adoption of cutting-edge AI applied sciences but in addition ensures a seamless and environment friendly operational move. Kubernetes, with its sturdy scalability and administration capabilities, stands as an excellent platform for generative AI tasks, aligning DevOps and MLOps practices in a cohesive ecosystem.”
This sentiment is already shared by a large swath of the business. In keeping with the CNCF survey above, as of 2025, 66% of organizations run generative AI workloads on Kubernetes. These organizations embrace OpenAI, which makes use of Kubernetes for its AI/LLM software experimenting and testing; Tesla, which makes use of KServe to handle production-grade LLM inference; and Adobe, which makes use of Kubernetes to energy its suite of generative artistic fashions. Different corporations taking this method embrace Uber, Intuit, and Google. With extra corporations adopting this follow for his or her generative AI and LLMs operations, it’d be prudent for any group to leverage Kubernetes for their very own GenAI and LLM workflows.
Agentic AI
Practically coinciding with the rise of GenAI has been the regular development of agentic AI. In contrast to GenAI, agentic AI goes past answering easy prompts and producing textual content in its capacity to function autonomously to carry out complicated, multistep actions, make the most of instruments, and make unbiased choices. With its capacity to help each conventional ML processes and GenAI and LLM operations, it ought to come as no shock that Kubernetes has a job within the agentic AI ecosystem as properly.
In keeping with Ronald Petty, principal guide at RX-M, “Kubernetes has been leveraged to host machine studying pipelines, together with AI mannequin coaching and inference. As inference choices have grow to be plentiful and inexpensive, on and off-premise, we now have seen the rise of brokers. Coupling cloud native applied sciences and well-liked protocols, we now see brokers shifting from advert hoc demos to complicated fleets of brokers on techniques like Kubernetes.” So what are some examples of the mixing between these two applied sciences?
One notable providing is Kagent, an OS programming framework that runs AI brokers in Kubernetes and “helps engineers construct highly effective inner platforms by tackling cloud native duties equivalent to configuration, troubleshooting, complicated deployment situations, observability pipelines and dashboards, and safely enabling community safety.” Working alongside related traces is K8sGPT, an AI-powered software that leverages clever insights and automatic troubleshooting to research Kubernetes clusters for configuration issues and safety points, in addition to generates options to issues found in evaluation.
A newer entry within the area is Sympozium, a Kubernetes-native coordination layer for multi-agent AI techniques that “solves the identical downside Kubernetes solved for containers, however for brokers that must share context, hand off duties, and keep shared situational consciousness.” One other newer providing is Agent Sandbox, which lets you run AI brokers as remoted, stateful workloads with a local API on Kubernetes.
The basics
Whereas it’s vital to pay attention to the most recent developments and tendencies affecting your area, that shouldn’t come on the expense of foundational information and abilities. As basketball nice Michael Jordan as soon as stated, “Get the basics down and the extent of every thing you do will rise.” Some of the basic abilities for working with Kubernetes is networking, and frustratingly sufficient, it’s one of many tougher ones to grasp. As Cisco senior employees engineer Nico Vibert observes, “Platform engineers are usually comfy with Linux networking however much less so with protocols like BGP and IPv6; community directors know these protocols properly however discover Kubernetes abstractions unfamiliar. Each personas wrestle to navigate the handfuls of networking instruments seemingly required to fulfill connectivity and safety necessities.” But as organizations transfer mission-critical workloads, AI coaching pipelines, and controlled monetary companies onto Kubernetes, the engineers who can design, safe, and troubleshoot the community layer have grow to be a few of the most sought-after professionals within the business.
In recognition of each the significance and tough nature of the Kubernetes networking talent, the CNCF just lately introduced a brand new certification centered on the Kubernetes community engineer position. The certification is designed to validate hands-on networking experience throughout all the aforementioned layers, filling a spot that the Kubernetes neighborhood has lengthy acknowledged.
For organizations that use Kubernetes to develop and ship purposes, leaders and decision-makers have to be conscious that using Kubernetes together with the most recent AI instruments is now not a luxurious however a needed follow that can enable their corporations to thrive. An identical onus needs to be positioned on the fundamentals. When hiring your subsequent DevOps, community, or web site reliability engineer, make sure that their capacity to design, safe, and troubleshoot the Kubernetes community layer is second to none.
If you wish to dive deeper, try Roland Huß and Daniele Zonca’s Generative AI on Kubernetes, Jonathan Johnson’s GPU Kubernetes Homelab dwell course, Alex Corvin, Taneem Ibrahim, and Kyle Stratis’s Scalable Kubernetes Infrastructure for AI Platforms, Ashok Srirama and Sukirti Gupta’s Kubernetes for Generative AI Options, and Yogesh Raheja’s K8sGPT Necessities on-demand course. They’re all on O’Reilly. If you happen to’re not a member, you may get began with a free trial.
