CNCF Warns Kubernetes Alone Is Not Enough to Secure LLM Workloads

CNCF Warns Kubernetes Alone Is Not Enough to Secure LLM Workloads - InfoQ

[BT](http://www.infoq.com/int/bt/ "bt")

InfoQ Software Architects' Newsletter

A monthly overview of things you need to know as an architect or aspiring architect.

View an example

Enter your e-mail address

Select your country - [x] I consent to InfoQ.com handling my data as explained in this Privacy Notice.

We protect your privacy.

Close

QCon San Francisco (Nov 16-20): What's next in AI? What's next in software? Learn from the teams already doing it.Register Now

Close

Toggle Navigation

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

English edition

• English edition
• Chinese edition
• Japanese edition
• French edition

[Write for InfoQ](http://www.infoq.com/write-for-infoq/ "Write for InfoQ")

Search

RegisterSign in

Unlock the full InfoQ experience

Unlock the full InfoQ experience by logging in! Stay updated with your favorite authors and topics, engage with content, and download exclusive resources.

Log In

or

Don't have an InfoQ account?

Register

• Stay updated on topics and peers that matter to youReceive instant alerts on the latest insights and trends.
• Quickly access free resources for continuous learningMinibooks, videos with transcripts, and training materials.
• Save articles and read at anytimeBookmark articles to read whenever youre ready.

Logo - Back to homepage

NewsArticlesPresentationsPodcastsGuides

Topics

[Development](http://www.infoq.com/development/ "Development")

• [Java](http://www.infoq.com/java/ "Java")
• [Kotlin](http://www.infoq.com/kotlin/ "Kotlin")
• [.Net](http://www.infoq.com/dotnet/ ".Net")
• [C#](http://www.infoq.com/c_sharp/ "C#")
• [Swift](http://www.infoq.com/swift/ "Swift")
• [Go](http://www.infoq.com/golang/ "Go")
• [Rust](http://www.infoq.com/rust/ "Rust")
• [JavaScript](http://www.infoq.com/javascript/ "JavaScript")

Featured in Development

• #### From VR to Flat Screens: Bridging the Input and Immersion Gap

Dany Lepage discusses the architectural journey of porting a hit VR title to seven non-VR platforms. He explains how his team solved the challenges of cross-progression, diverse input paradigms, and maintaining release velocity across Steam, iOS, and PlayStation. Beyond the tech, he shares candid lessons on the "product fit" gap when translating immersive social presence to 2D screens.


All in developmentFollow Topic

[Architecture & Design](http://www.infoq.com/architecture-design/ "Architecture & Design")

• [Architecture](http://www.infoq.com/architecture/ "Architecture")
• [Enterprise Architecture](http://www.infoq.com/enterprise-architecture/ "Enterprise Architecture")
• [Scalability/Performance](http://www.infoq.com/performance-scalability/ "Scalability/Performance")
• [Design](http://www.infoq.com/design/ "Design")
• [Case Studies](http://www.infoq.com/Case_Study/ "Case Studies")
• [Microservices](http://www.infoq.com/microservices/ "Microservices")
• [Service Mesh](http://www.infoq.com/servicemesh/ "Service Mesh")
• [Patterns](http://www.infoq.com/DesignPattern/ "Patterns")
• [Security](http://www.infoq.com/Security/ "Security")

Featured in Architecture & Design

• #### Platform Engineering: Lessons from the Rise and Fall of eBay Velocity

Randy Shoup discusses the "Velocity Initiative," a transformation that doubled engineering productivity and modernized eBay’s DORA metrics. He shares the technical playbook used to scale 4,500 services while explaining why even elite engineering execution can’t save a company hampered by waterfall planning, risk aversion, and a "pathological" culture of fear.


All in architecture-designFollow Topic

[AI Infrastructure](http://www.infoq.com/ai-ml-data-eng/ "AI Infrastructure")

• [Big Data](http://www.infoq.com/bigdata/ "Big Data")
• [Machine Learning](http://www.infoq.com/machinelearning/ "Machine Learning")
• [NoSQL](http://www.infoq.com/nosql/ "NoSQL")
• [Database](http://www.infoq.com/database/ "Database")
• [Data Analytics](http://www.infoq.com/data-analytics/ "Data Analytics")
• [Streaming](http://www.infoq.com/streaming/ "Streaming")

Featured in AI, ML & Data Engineering

• #### Lakehouse Tower of Babel: Handling Identifier Resolution Rules Across Database Engines

Lakehouse architectures enable multiple engines to operate on shared data using open table formats such as Apache Iceberg. However, differences in SQL identifier resolution and catalog naming rules create interoperability failures. This article examines these behaviors and explains why enforcing consistent naming conventions and cross-engine validation is critical.


All in ai-ml-data-engFollow Topic

[Culture & Methods](http://www.infoq.com/culture-methods/ "Culture & Methods")

• [Agile](http://www.infoq.com/agile/ "Agile")
• [Diversity](http://www.infoq.com/diversity/ "Diversity")
• [Leadership](http://www.infoq.com/leadership/ "Leadership")
• [Lean/Kanban](http://www.infoq.com/lean/ "Lean/Kanban")
• [Personal Growth](http://www.infoq.com/personal-growth/ "Personal Growth")
• [Scrum](http://www.infoq.com/scrum/ "Scrum")
• [Sociocracy](http://www.infoq.com/sociocracy/ "Sociocracy")
• [Software Craftmanship](http://www.infoq.com/software_craftsmanship/ "Software Craftmanship")
• [Team Collaboration](http://www.infoq.com/team-collaboration/ "Team Collaboration")
• [Testing](http://www.infoq.com/testing/ "Testing")
• [UX](http://www.infoq.com/ux/ "UX")

Featured in Culture & Methods

• #### Empower Your Developers: How Open Source Dependencies Risk Management Can Unlock Innovation

Celine Pypaert discusses the ubiquitous nature of open-source software and shares a blueprint for securing modern applications. She explains how to prioritize high-risk vulnerabilities using exploitability data, the role of Software Bill of Materials (SBOM), and the importance of bridging the gap between DevOps and Security through clear accountability and automated governance.


All in culture-methodsFollow Topic

DevOps

• [Infrastructure](http://www.infoq.com/infrastructure/ "Infrastructure")
• [Continuous Delivery](http://www.infoq.com/continuous_delivery/ "Continuous Delivery")
• [Automation](http://www.infoq.com/automation/ "Automation")
• [Containers](http://www.infoq.com/containers/ "Containers")
• [Cloud](http://www.infoq.com/cloud-computing/ "Cloud")
• [Observability](http://www.infoq.com/observability/ "Observability")

Featured in DevOps

• #### Beyond One-Click: Designing an Enterprise-Grade Observability Extension for Docker

Docker Extensions boost developer speed but create a "visibility gap" by isolating telemetry. To meet enterprise needs, extensions must act as bridges to centralized platforms. This article details how to use OpenTelemetry, policy-as-code, and encryption to build secure pipelines. Learn to balance developer productivity with the governance required for scalable, compliant observability.


All in devopsFollow Topic

[Events](https://events.infoq.com/ "Events")

Helpful links

• [About InfoQ](http://www.infoq.com/about-infoq "About InfoQ")
• [InfoQ Editors](http://www.infoq.com/infoq-editors "InfoQ Editors")
• [Write for InfoQ](http://www.infoq.com/write-for-infoq "Write for InfoQ")
• [About C4Media](https://c4media.com/ "About C4Media")
• [Diversity](https://c4media.com/diversity "Diversity")

Choose your language

• [En](http://www.infoq.com/news/2026/04/kubernetes-secure-workloads/# "InfoQ English")
• 中文
• 日本
• Fr


[InfoQ Homepage](http://www.infoq.com/ "InfoQ Homepage")[News](http://www.infoq.com/news "News")CNCF Warns Kubernetes Alone Is Not Enough to Secure LLM Workloads

[DevOps](http://www.infoq.com/Devops/ "DevOps")

Shipping Faster, Breaking More: Rethinking Delivery Systems in the Age of AI (Webinar May 28th)

CNCF Warns Kubernetes Alone Is Not Enough to Secure LLM Workloads

Apr 17, 2026 3 min read

by

• Craig Risi

Follow Software Architect | Game Designer| Writer | Speaker

#### Write for InfoQ

Feed your curiosity.Help 550k+ global

senior developers

each month stay ahead.Get in touch

Log in to listen to this article

Loading audio

Your browser does not support the audio element.

0:00 0:00

Normal 1.25x 1.5x

Like

• Reading list

A new blog from the Cloud Native Computing Foundation highlights a critical gap in how organizations are deploying large language models (LLMs) on Kubernetes: while Kubernetes excels at orchestrating and isolating workloads, it does not inherently understand or control the behavior of AI systems, creating a fundamentally different and more complex threat model.

The article argues that LLMs introduce a new class of risk because they operate on untrusted input and can dynamically decide actions, unlike traditional applications. In a typical deployment, such as exposing an LLM via an API or chat interface, Kubernetes ensures that pods are running and resources are stable, but it has no visibility into whether prompts are malicious, whether sensitive data is being exposed, or whether the model is interacting with internal systems in unsafe ways. This creates a scenario where infrastructure appears healthy while underlying risks go undetected.

The CNCF emphasizes that LLM-based systems must be treated as programmable, decision-making entities, not just compute workloads. By placing an LLM in front of internal tools, logs, APIs, or credentials, organizations are effectively introducing a new layer of abstraction that can be influenced through prompt input. This opens the door to risks such as prompt injection, unintended data exposure, and misuse of connected tools, threats that traditional Kubernetes security controls were not designed to handle.

This shift reflects a broader evolution in cloud-native systems, where Kubernetes is increasingly used to run AI and generative workloads. As adoption grows, the platform is being stretched beyond its original purpose of managing stateless microservices into orchestrating data-intensive, agent-driven, and inference-heavy systems. However, the security model has not fully caught up with these new use cases.

While Kubernetes provides strong primitives for scheduling, isolation, and resource management, it lacks built-in mechanisms to enforce application-level or semantic controls over AI systems. For example, it cannot determine whether a prompt should be executed, whether a response leaks sensitive information, or whether an LLM should have access to certain tools or APIs.

This limitation highlights the need for additional layers of control beyond infrastructure. Traditional Kubernetes security practices, such as RBAC, network policies, and container isolation, remain necessary but are insufficient on their own. Instead, organizations must consider AI-specific controls, including prompt validation, output filtering, tool access restrictions, and policy enforcement at the application layer.

The blog points to an emerging need for AI-aware platform engineering, where security is embedded across both infrastructure and application layers. This includes integrating frameworks such as the OWASP Top 10 for LLMs, applying policy-as-code, and introducing guardrails that govern how models interact with data and external systems.

Industry discussions increasingly frame this as a shift from traditional threat models to behavioral and context-aware security models, where the focus is not just on protecting infrastructure, but on controlling how intelligent systems behave within it. As LLMs evolve into autonomous or agentic systems capable of executing actions, these concerns become even more critical.

The CNCF's analysis serves as a warning for organizations rapidly adopting AI on Kubernetes: operational health does not equal security. A system can be fully compliant with Kubernetes best practices while still exposing significant risks through its AI layer.

Major technology and security vendors are converging on similar principles. Industry guidanceincreasingly recommends a multi-layered security model, combining runtime monitoring, human-in-the-loop controls, and strict policy enforcement around what AI systems are allowed to do. A consistent theme is that LLMs should never be treated as authoritative decision-makers: instead, they must operate within bounded contexts with explicit guardrails, continuous validation, and auditability.

As LLM adoption accelerates, the industry is being pushed to rethink long-standing assumptions about trust boundaries, workload isolation, and application behavior. The result is a new security paradigm, one where Kubernetes remains a foundational layer, but must be complemented by AI-specific governance, observability, and control mechanisms to ensure safe and reliable deployment of intelligent systems.

About the Author

#### Craig Risi

Craig Risi is a man of many talents but has no sense of how to use them. He could be out changing the world but prefers to make software instead. He possesses a passion for software design, but more importantly software quality and designing systems in a technically diverse and constantly evolving tech world. Craig is also the writer of the book, Quality By Design: Designing Quality Software Systems, and writes regular articles on his blog sites and various other tech sites around the world. When not playing with software, he can often be found writing, designing board games, or running long distances for no apparent reason.

Show more Show less

#### This content is in the DevOps topic

Follow Topic

##### Related Topics:

• DevOps### DevOps

Followers: 5044

Follow Topic

• AI, ML & Data Engineering### AI, ML & Data Engineering

Followers: 5862

Follow Topic

• Artificial Intelligence### Artificial Intelligence

Followers: 211

Follow Topic

• Kubernetes### Kubernetes

Followers: 490

Follow Topic

• Cloud Security### Cloud Security

Followers: 77

Follow Topic

• Cloud Native Computing Foundation### Cloud Native Computing Foundation

Followers: 50

Follow Topic

• Large language models### Large language models

Followers: 137

Follow Topic

* #### Popular in DevOps

• ##### GitHub Copilot CLI Reaches General Availability

• ##### Duolingo's Kubernetes Leap

• ##### New Rowhammer Attacks on NVIDIA GPUs Enable Full System Takeover

• ##### OpenTelemetry Declarative Configuration Reaches Stability Milestone

• ##### Airbnb Migrates High-Volume Metrics Pipeline to OpenTelemetry

* #### Related Sponsors

• ##### The Case for Real-Time Threat Monitoring and Analysis in Modern Mobile App Security

• ##### The Rise of Client-Side Risk and the Trust Gap

• ##### Before it Breaks: AI-Driven Azure Incident Response

• ##### Stateful agents, stateful infra: The transport gap AI teams are patching by hand

• ##### Does your AI stack need a session layer? A maturity framework for teams building AI agents

* #### Related Sponsor


• May 12, 2026, 1:30 PM EDT

##### Designing Data Layers for Agentic AI: Patterns for State, Memory, and Coordination at Scale

Presented by: Karthik Ranganathan - Co-CEO & Co-Founder at YugabyteDB, and Aditi Gupta - Snr. GenAI/ML Specialist Solutions Architect

SPONSORED BY YUGABYTEDB Save your seat

Related Content

• ##### Kubernetes Autoscaling Demands New Observability Focus beyond Vendor Tooling

Mar 31, 2026

• ##### KubeVirt v1.8 Brings Multi-Hypervisor Support and Confidential Computing to Kubernetes

Mar 31, 2026

• ##### Kubescape 4.0 Brings Runtime Security and AI Agent Scanning to Kubernetes

Mar 29, 2026

• ##### AWS Load Balancer Controller Reaches GA with Kubernetes Gateway API Support

Mar 25, 2026

• Icon##### Duolingo's Kubernetes Leap

Apr 06, 2026 

• ##### Proactive Autoscaling for Edge Applications in Kubernetes

Feb 17, 2026 

• ##### Anthropic Introduces Agent-Based Code Review for Claude Code

Apr 17, 2026

• ##### Cursor 3 Introduces Agent-First Interface, Moving Beyond the IDE Model

Apr 16, 2026

• ##### CNCF and Kusari Partner to Strengthen Software Supply Chain Security across Cloud-Native Projects

Apr 10, 2026

Related Sponsors

• #### Designing Data Layers for Agentic AI: Patterns for State, Memory, and Coordination at Scale (Live Webinar May 12, 2026) - Save Your Seat

AI agents create new architectural challenges: shared memory, cross-agent state, and auditability. This session explores data layer patterns—conversation state, knowledge persistence, coordination—and tradeoffs in consistency, latency, and cost at scale using AWS and YugabyteDB.

• Sponsored by


Related Content

• ##### Google Opens Gemma 4 Under Apache 2.0 with Multimodal and Agentic Capabilities

Apr 16, 2026

• ##### Cloudflare Launches Code Mode MCP Server to Optimize Token Usage for AI Agents

Apr 16, 2026

• ##### Google’s TurboQuant Compression May Support Faster Inference, Same Accuracy on Less Capable Hardware

Apr 15, 2026

• ##### New Rowhammer Attacks on NVIDIA GPUs Enable Full System Takeover

Apr 14, 2026

• ##### Anthropic Paper Examines Behavioral Impact of Emotion-Like Mechanisms in LLMs

Apr 14, 2026

• ##### Google Released Gemma 4 with a Focus on Local-First, On-Device AI Inference

Apr 13, 2026

**The InfoQ** Newsletter

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

Enter your e-mail address

Select your country - [x] I consent to InfoQ.com handling my data as explained in this Privacy Notice.

We protect your privacy.

• Development

• ##### [From VR to Flat Screens: Bridging the Input and Immersion Gap](http://www.infoq.com/presentations/game-vr-flat-screens/ "From VR to Flat Screens: Bridging the Input and Immersion Gap")

• ##### [Cursor 3 Introduces Agent-First Interface, Moving Beyond the IDE Model](http://www.infoq.com/news/2026/04/cursor-3-agent-first-interface/ "Cursor 3 Introduces Agent-First Interface, Moving Beyond the IDE Model")

• ##### [Claude Code Used to Find Remotely Exploitable Linux Kernel Vulnerability Hidden for 23 Years](http://www.infoq.com/news/2026/04/claude-code-linux-vulnerability/ "Claude Code Used to Find Remotely Exploitable Linux Kernel Vulnerability Hidden for 23 Years")

• Architecture & Design

• ##### [Cloudflare Launches Code Mode MCP Server to Optimize Token Usage for AI Agents](http://www.infoq.com/news/2026/04/cloudflare-code-mode-mcp-server/ "Cloudflare Launches Code Mode MCP Server to Optimize Token Usage for AI Agents")

• ##### [Zendesk Says AI Makes Code Abundant, Shifting the Bottleneck to “Absorption Capacity”](http://www.infoq.com/news/2026/04/zendesk-absorption-capacity/ "Zendesk Says AI Makes Code Abundant, Shifting the Bottleneck to “Absorption Capacity”")

• ##### [Platform Engineering: Lessons from the Rise and Fall of eBay Velocity](http://www.infoq.com/presentations/platform-engineering-lessons/ "Platform Engineering: Lessons from the Rise and Fall of eBay Velocity")

• Culture & Methods

• ##### [Platform as a Product: Delivering Value While Balancing Competing Priorities](http://www.infoq.com/news/2026/04/platform-product-deliver-value/ "Platform as a Product: Delivering Value While Balancing Competing Priorities")

• ##### [Empower Your Developers: How Open Source Dependencies Risk Management Can Unlock Innovation](http://www.infoq.com/presentations/open-source-dependencies/ "Empower Your Developers: How Open Source Dependencies Risk Management Can Unlock Innovation")

• ##### [Tiger Teams, Evals and Agents: The New AI Engineering Playbook](http://www.infoq.com/podcasts/tiger-teams-evals-agents/ "Tiger Teams, Evals and Agents: The New AI Engineering Playbook")

• AI, ML & Data Engineering

• ##### [Anthropic Introduces Agent-Based Code Review for Claude Code](http://www.infoq.com/news/2026/04/claude-code-review/ "Anthropic Introduces Agent-Based Code Review for Claude Code")

• ##### [Lakehouse Tower of Babel: Handling Identifier Resolution Rules Across Database Engines](http://www.infoq.com/articles/lakehouse-sql-identifier-rules/ "Lakehouse Tower of Babel: Handling Identifier Resolution Rules Across Database Engines")

• ##### [Google Opens Gemma 4 Under Apache 2.0 with Multimodal and Agentic Capabilities](http://www.infoq.com/news/2026/04/google-gemm4/ "Google Opens Gemma 4 Under Apache 2.0 with Multimodal and Agentic Capabilities")

• DevOps

• ##### [CNCF Warns Kubernetes Alone Is Not Enough to Secure LLM Workloads](http://www.infoq.com/news/2026/04/kubernetes-secure-workloads/ "CNCF Warns Kubernetes Alone Is Not Enough to Secure LLM Workloads")

• ##### [OpenTelemetry Declarative Configuration Reaches Stability Milestone](http://www.infoq.com/news/2026/04/opentelemetry-declarative-config/ "OpenTelemetry Declarative Configuration Reaches Stability Milestone")

• ##### [New Rowhammer Attacks on NVIDIA GPUs Enable Full System Takeover](http://www.infoq.com/news/2026/04/rowhammer-attacks-nvidia/ "New Rowhammer Attacks on NVIDIA GPUs Enable Full System Takeover")

**The InfoQ** Newsletter

A round-up of last week’s content on InfoQ sent out every Tuesday. Join a community of over 250,000 senior developers. View an example

• Get a quick overview of content published on a variety of innovator and early adopter technologies
• Learn what you don’t know that you don’t know
• Stay up to date with the latest information from the topics you are interested in

Enter your e-mail address

Select your country - [x] I consent to InfoQ.com handling my data as explained in this Privacy Notice.

We protect your privacy.

**May 7 | June 10, 2026 | Online** Architecture decisions are hard to validate while shipping. Join a **5-week online cohort** for **senior engineers, architects, and team leads** to pressure-test real decisions, apply practical frameworks, and work through challenges with a confidential peer group. Facilitated by Luca Mezzalira, Principal Architect at AWS, this cohort helps you: * Pressure-test real decisions. * Apply frameworks to real problems. * Publish on InfoQ.com and earn your certification. **RESERVE YOUR PLACE**

#### Events

• ##### Online InfoQ Architect Certification

May 7, 2026

• ##### QCon AI Boston

June 1-2, 2026

• ##### Online InfoQ Architect Certification

June 10, 2026

• ##### QCon San Francisco

November 16-20, 2026

#### Follow us on

Youtube 232K FollowersLinkedin 26K FollowersRSS 19K ReadersX 57.1k FollowersFacebook 21K LikesBluesky New

#### Stay in the know

The InfoQ PodcastEngineering Culture PodcastThe Software Architects' Newsletter

General Feedback [[email protected]](mailto:[email protected]) Advertising [[email protected]](mailto:[email protected]) Editorial [[email protected]](mailto:[email protected]) Marketing [[email protected]](mailto:[email protected])

InfoQ.com and all content copyright © 2006-2026 C4Media Inc.

Privacy Notice, Terms And Conditions, Cookie Policy

Close

[BT](http://www.infoq.com/int/bt/ "bt")