VibeSec Reckoning

Vibe coding enables non-technical users (or as we call them, citizen builders) to build applications with AI that they simply could not have built before. When our AI applications team in Global Marketing at Thoughtworks was asked to scale a vibe coded prototype built by one of our citizen builders in global marketing, we discovered serious cracks that prevent vibe coded applications from going into production safely.

Speed without guardrails is a risk no team can afford to ignore. What follows is the story of what we found, what it means for teams building with AI, and the steps we are taking to ensure every workflow, prototype, and app we ship is one we can stand behind.

What We Learned the Hard Way

The AI applications team within Global Marketing was tasked with scaling a video assembly prototype built with Gemini, Replit AI, and Claude AI to create on-brand videos for use across our 10,000 employees. The team encountered two moments that halted work entirely. In both cases, the AI suggested paths with significant security implications. In both cases, it took a human asking the right questions to identify the issues.

Security Risk #1

#### Public Storage Access

The AI recommended making the storage bucket public or setting cloud file storage to "anyone with the link." When challenged, it justified this by stating that every company does it. Only a firm rejection prompted a secure alternative.

This could have exposed sensitive unreleased brand assets and audience data to the public internet.

Security Risk #2

#### Excessive Token Permissions

A service account was assigned the Access Token Creator role, granting it the ability to create short-lived tokens and access databases and other resources far beyond what the task required. The team identified this issue before running the code.

This would have allowed a compromised service account to move laterally through an entire cloud workspace.

The key insight here is that AI tools often recommend the path of least resistance. That path is not always the secure one. Human judgment remains essential, but it should not be the sole control. The goal is to provide agents with technical security rules as context from the initial prompt and validate their output through deterministic checks in the development workflow to ensure that insecure code, permissions, secrets, or infrastructure remain undetected.

The Numbers Behind the Risk

| Metric | Value | Source | | --- | --- | --- | | AI-generated code with confirmed vulnerabilities | 25% | AppSec Santa, 2026 | | Increase in attacks exploiting application vulnerabilities, year on year | 44% | SQ Magazine AI Coding Security Statistics, 2026 | | Codebases with high or critical severity vulnerabilities | 78% | Black Duck OSSRA 2026 | | Organizations with no sensitive data policies for AI | 50% | AppSec Santa, 2026 | | Mean vulnerabilities per codebase increase year on year | +107% | Black Duck OSSRA 2026 | | Enterprise breaches now caused by AI-generated code | 1 in 5 | Aikido Security, 2026 | | New CVEs from AI-generated code in March 2026 alone | 35 | Georgia Tech Vibe Security Radar, SSLab, March 2026 | | AI systems with prompt injection exposure in 2026 audits | 73% | SQ Magazine AI Coding Security Statistics, 2026 | | Share of all new enterprise software that is AI-generated | 42% | Sonar Developer Survey, 2026 | | Security teams say keeping up with AI-generated code volume is getting harder | 62% | ProjectDiscovery AI Coding Impact Report, April 2026 |

The Real Problem: Prompts Are Not Enough

After sharing these incidents with engineering and security colleagues, a clear message emerged: telling an AI agent to be safe is not the same as enforcing that it is safe. Prompts can be overridden, misunderstood, or ignored. The moment a user pushes back on a restriction or phrases a request differently, the constraint can evaporate.

"It is not sufficient to merely tell the LLM the desired behavior of your output artifacts. If you absolutely do not want something to be true, it must be codified in non-negotiable rules somewhere in your development lifecycle." - Engineering Leadership

Think of it this way: prompting for test-driven development is not the same as enforcing code coverage thresholds in your build tool. One is a suggestion. The other is a gate. Birgitta Böckeler’s work on harness engineering makes this concrete by outlining a mental model for building trust in coding agents. Instead of relying solely on prompts, developers wrap the agent in an outer “harness” structured along two axes:

• Guides (feedforward controls) anticipate unwanted behavior and steer the model before it acts, while Sensors (feedback controls) observe the code after the agent acts to flag errors.
• Computational controls are deterministic, fast, and CPU-run (like linters or test suites). Inferential controls rely on semantic analysis and AI-driven judgment (like specific system prompt constraints).

Why business functions need to pay attention

Business functions like our marketing team, who are building with AI, are not exempt from the security obligations that apply to engineers building applications. Building security into software is a fundamental requirement for protecting customer and employee data. Even lightweight internal prototypes must comply with enterprise security standards. Without the right guardrails, AI-assisted development can expose sensitive data long before an application reaches production.

Client trust

#### Compliance is contractual

Adhering to standards like ISO 27001 ensures the protection of sensitive data. All applications, regardless of how quickly they are built, must meet these security benchmarks to maintain the trust of customers and employees.

Brand integrity

#### Brand assets need protection

Core work involves sensitive functional data (e.g., unreleased campaign assets, financial data, or audience insights). Over-permissioned service accounts put far more than code at risk.

Reputation

#### Business functions can set the standard

When business functions like marketing lead with security discipline, they signal responsible AI adoption to the wider organisation and to clients.

Short-term habits

You do not need to be a security expert to start building responsibly. These three habits can get you started.

#### Feed your technical security rules into every session

Add your organisation's security guidelines as “Rules” in Claude, Cursor, or Replit to begin with (later on invest in a shared sensible default layer across all tools). The AI agent uses them as guidance, making secure patterns more likely from the start. They still need to be backed by deterministic checks that fail unsafe code, exposed secrets, broad permissions, vulnerable dependencies, or insecure infrastructure before anything is deployed.

#### Question every permission the AI suggests

If a tool recommends making something public or assigning a broad service account role, stop and ask why. The path of least resistance and the secure path are rarely the same thing.

#### Try the red team prompt

Ask your AI to roleplay as a bad actor and pen test what it just built. This technique consistently surfaces vulnerabilities that forward-looking prompts miss, particularly around permissions and data exposure.

Medium-term solutions

Reading about risk is one thing. Doing something about it is another. These incidents prompted two practical initiatives. The principles behind them are replicable by any team building with AI, regardless of technical background.

#### A security context file

We compiled our technical security rules into a structured context file loaded into every AI coding session before any code is written. It covers zero trust enforcement, secrets management, harness engineering, and supply chain integrity. The key distinction from a casual prompt is operational discipline: the file is versioned, loaded by default, reviewed, and paired with automated checks. It acts as a comprehensive inferential guide telling the agent what good looks like; but it must be paired with computational sensors in the pipeline to validate whether the output is acceptable.

#### A daily security intelligence feed

Currently, this automated consolidation ensures we see supply chain alerts the day they are published. In the future, as we work toward an agentic enterprise, we envision agents proactively creating story cards and identifying and patching known vulnerabilities behind the scenes for human review, significantly reducing the Software Development Lifecycle (SDLC) cycle time.

The security context file in practice

The idea behind this approach is straightforward: AI tools read context at the start of a session, so make that context your technical security rules. The file is the result of working through your organisation's security requirements and structuring them in a form the AI can act on, not just acknowledge.

What follows is the kind of coverage any such file should include. The specifics will differ by organisation, but the categories are consistent.

| 区域覆盖 | 好的样子是什么 | 为什么很重要 |
| --- | --- | --- |
| 零信任和最小权限 | 对每个服务账户和存储资源进行严格的身份验证和最小访问权限 | 设置推理指导参数以防止令牌权限风险。 |
| 密码管理 | AI 不会在代码中生成或存储 API 密钥、密码或令牌；始终路由到环境变量或密码管理器 | 在到达仓库之前阻止凭证泄露。 |
| 工程门控 | SAST 扫描、凭证扫描和基础设施验证必须在部署前通过；不能仅依赖提示指令 | 用确定性的计算传感器支持推理指令。 |
| 供应链完整性 | 只使用经过验证的库；定期审计所有依赖项以发现已知漏洞 | 减少 AI 建议不明或未经验证的包的风险。 |
| AI 责任 | 所有由 AI 生成的代码在部署前都会标记供同行评审和自动安全扫描；不允许未经批准的 AI 使用 | 用于符合性审计。 |

与提示的区别在于文件包含不可协商的规则，强制 AI 代理拒绝违反政策的请求。如果 AI 代理被要求绕过检查、禁用日志记录或将某些内容设置为公共访问，规则应引导其拒绝并解释原因。但重要的是，确定性的检查和部署门控应该即使代理未能遵循该指导原则也能捕获问题。这种拒绝正是我们在两次差点失误中所缺少的。

## 实践中的安全情报流

保持知情状态本身就是一种防御方式。工作流程监控团队主动使用的工具和语言，并每天提供最新的 CVE、平台通告和安全简报。最重要覆盖领域在组织间是一致的：他们编写的编程语言、部署的云平台、AI 编码工具本身以及整个 CVE 数据库。

目标很简单：在漏洞披露当天了解它，而不是几周后。现在，42% 的新企业软件都是由 AI 生成或辅助生成的，加速开发的工具也最有可能出现在新的 CVE 披露中。积极监控它们是拥有自己安全的一部分。

#### 更广泛的观点

这两种方法都不需要工程背景就能采用。一个是为 AI 消费而结构化的政策文档。另一个是自动化搜索。它们共同点是认识到被动的安全意识在 AI 以高速生成代码时是不够的。

## 长期组织变革

#### 从提示到流水线

将工程门控集成到标准原型模板中。从概率性提示转移到明确的反馈循环。如果计算传感器（如自动安全扫描器）触发，代理循环必须结构性地迫使模型自我纠正，直到通过为止。

#### 将安全规则输入应用程序构建器（Cursor、Claude 等）

将组织的技术安全规则编译成一个结构化的上下文 Markdown 文件，并将其加载为“规则”，模型必须遵守这些规则。它在修复成本最低的地方捕获最常见的错误，即在提交代码之前。

#### 让安全路径成为易走路径

给构建者一个默认安全的起点。预配置认证模式、私有存储默认值、密钥处理和依赖扫描的模板减少了因截止压力而取捷径的可能性。

#### 定义跨功能的起始工程门控

由业务部门、工程和安全共同构建的共享起始工程门控为每个构建者提供了一个从第一天开始的安全基础，而不是每个团队独立重新发现同样的错误。

## 结论：超越原型的扩展

这段旅程始于我们被邀请支持另一支团队为全球营销黑客马拉松构建视频组装平台。随着我们帮助扩展解决方案，很明显，“氛围编码”没有企业级护栏可能会引入组织无法忽视的风险。

通过将我们的技术安全规则直接嵌入代理工作流中，我们将早期差点失误转变为一个安全且生产就绪的平台，在黑客马拉松期间成功部署给 150 名用户。

这种转变——从依靠人类来发现问题，到将技术安全规则、自动化检查和人工责任纳入工作流——已成为我们在代理时代快速前进的同时保持工程严谨性的蓝图。

* * *

## 思想工具有关进一步阅读

这些挑战并不局限于我们的团队。思想工具有关同事深入研究这些问题并发布研究成果。以下是与我们描述最相关的部分。

思想工具有关 CTO Rachel Laycock 的文章《为什么工程严谨性不会消失》[https://www.thoughtworks.com/about-us/news/2026/combat-ai-cognitive-debt-radar-v34]，讨论了当 AI 生成代码时，工程严谨性如何被重新定位。涵盖了将编码代理束之以笼子、零信任架构以及为什么安全护栏现在是不可或缺的。

思想工具有关 [从业者和企业领导者聚会](https://www.thoughtworks.com/en-us/about-us/events/the-future-of-software-development)，发现安全在 AI 采用过程中一直被优先级降低。授予代理广泛的工具访问权，特别是电子邮件，被视为立即且具体的风险。

Birgitta Boeckeler, a Distinguished Engineer at Thoughtworks, discusses placing deterministic controls within the agentic loop. The definitive guide to building a harness that enforces security rules without paralyzing the model.

How to embed security guardrails and never-allow rules as mandatory pre-commit hooks in any AI-assisted development workflow.

Thoughtworks' annual technology outlook warns that without rigorous engineering oversight, generative tools risk compounding technical debt and introducing security vulnerabilities. Success requires a co-construction model where humans ensure AI-generated code maintains architectural integrity.

Acknowledgments

Big thanks to the team for the engaging discussions that shaped this article, particularly Julie Woods-Moss for championing AI prototyping risks, and Natalie Drucker for her significant role in shaping the content and title.

Thanks to Martin Fowler for reshaping our perspective on short-term versus long-term habits and harness engineering, and to Kief Morris for anchoring the core argument on LLMs bypassing prompts.

Appreciations to Jim Gumbley for bringing in the critical governance and ISO27001 focus and reviewing the article. Honorable mentions: Craig Stanley for advocating the strategic rollout, Anderson Maeda for the concept of a centralized skills repository, Simone Thompson for the bad-actor simulation methodology, Premanand Chandrasekaran for defining the technical boundaries of LLM enforcement, Matteo Vaccari for the nuances on balancing strict rules with model flexibility, and Bilal Jaffery.

GenAI (Gemini and Claude) was used to streamline the research process, pull in insights, and polish the language for maximum clarity and readability.