为什么你的AI演示会在生产中崩溃

Towards Data Science

Towards Data Science2026年5月18日

为什么你的AI演示会在生产中崩溃

8.5内容质量

TL;DR · AI 摘要

本文探讨了AI演示可能因隐私问题和跟踪机制而失效的原因，强调了如何优化这些方面以提高其可信度。

核心要点

AI演示应优先考虑用户隐私和数据安全以避免合规问题。
有效的cookie管理是确保用户体验和网站功能的关键。
定期审查和调整session跟踪实践有助于解决潜在的问题。

结构提纲

按章节快速跳转。

§引言
文章讨论了AI演示可能在生产环境中失败的原因，包括隐私问题和跟踪机制。
·核心机制
文章详细解释了如何通过优化隐私设置、合理使用cookies以及管理session来提升AI演示的可信度和用户体验。

思维导图

用一张图看清主题之间的关系。

查看大纲文本（无障碍 / 无 JS 友好）

AI演示与隐私保护
- 隐私设置
  - Cookie BCTempID
  - Duration: 10分钟
  - Description: No description available.
- Session管理
  - Necessary cookies
  - Duration: 7天
  - Description: Amazon Web Services set this cookie for load balancing.

金句 / Highlights

值得收藏与分享的关键句。

AI演示应优先考虑用户隐私和数据安全以避免合规问题。
— 第2段
⬇︎ 下载 PNG 𝕏 分享到 X
有效的cookie管理是确保用户体验和网站功能的关键。
— 第2段
⬇︎ 下载 PNG 𝕏 分享到 X
定期审查和调整session跟踪实践有助于解决潜在的问题。
— 第2段
⬇︎ 下载 PNG 𝕏 分享到 X

#privacy#cookies#session management

打开原文

Why Your AI Demo Will Die in Production | Towards Data Science

Image 1: Revisit consent button

We value your privacy

We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies.

Customise Reject All Accept All

Customise Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ...Show more

Necessary Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Cookie BCTempID
Duration 10 minutes
Description No description available.

Cookie __cf_bm
Duration 1 hour
Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management.

Cookie AWSALBCORS
Duration 7 days
Description Amazon Web Services set this cookie for load balancing.

Cookie _cfuvid
Duration session
Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services

Cookie li_gc
Duration 6 months
Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.

Cookie __hssrc
Duration session
Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.

Cookie __hssc
Duration 1 hour
Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.

Cookie wpEmojiSettingsSupports
Duration session
Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.

Cookie BCSessionID
Duration 1 year 1 month 4 days
Description Blueconic sets this cookie as a unique identifier for the BlueConic profile.

Cookie _octo
Duration 1 year
Description No description available.

Cookie logged_in
Duration 1 year
Description No description available.

Cookie __Secure-YEC
Duration past
Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video

Cookie __eoi
Duration 6 months
Description Description is currently not available.

Cookie AWSALBTGCORS
Duration 7 days
Description No description available.

Cookie login-status-p
Duration past
Description Description is currently not available.

Cookie AWSALBTG
Duration 7 days
Description No description available.

Cookie csrf_token
Duration session
Description No description available.

Cookie token_v2
Duration 1 day
Description Description is currently not available.

Cookie D
Duration 1 year
Description Description is currently not available.

Cookie PHPSESSID
Duration session
Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.

Cookie VISITOR_PRIVACY_METADATA
Duration 6 months
Description YouTube sets this cookie to store the user's cookie consent state for the current domain.

Cookie cookietest
Duration session
Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.

Cookie __Host-airtable-session
Duration 1 year
Description This cookie is used to enable us to integrate the services of Airtable.

Cookie __Host-airtable-session.sig
Duration 1 year
Description This cookie is used to enable us to integrate the services of Airtable.

Cookie m
Duration 1 year 1 month 4 days
Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.

Cookie BIGipServer*
Duration session
Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user.

Cookie __cfruid
Duration session
Description Cloudflare sets this cookie to identify trusted web traffic.

Cookie _GRECAPTCHA
Duration 6 months
Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.

Cookie __Secure-YNID
Duration 6 months
Description Google cookie used to protect user security and prevent fraud, especially during the login process.

Cookie cookieyes-consent
Duration 1 year
Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.

Functional

[x]

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Cookie lidc
Duration 1 day
Description LinkedIn sets the lidc cookie to facilitate data center selection.

Cookie brw
Duration 1 year
Description No description available.

Cookie brwConsent
Duration 5 minutes
Description Description is currently not available.

Cookie WMF-Uniq
Duration 1 year
Description Description is currently not available.

Cookie loom_anon_comment
Duration 1 year
Description No description available.

Cookie loom_referral_video
Duration session
Description Description is currently not available.

Cookie VISITOR_INFO1_LIVE
Duration 6 months
Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.

Cookie yt-remote-connected-devices
Duration Never Expires
Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos.

Cookie ytidb::LAST_RESULT_ENTRY_KEY
Duration Never Expires
Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.

Cookie yt-remote-device-id
Duration Never Expires
Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos.

Cookie yt-remote-session-name
Duration session
Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.

Cookie yt-remote-fast-check-period
Duration session
Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.

Cookie yt-remote-session-app
Duration session
Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.

Cookie yt-remote-cast-available
Duration session
Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.

Cookie yt-remote-cast-installed
Duration session
Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.

Cookie cp_session
Duration 3 months
Description Codepen sets this cookie for Help systems found in the website.

Cookie loid
Duration 1 year 1 month 4 days
Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.

Analytics

[x]

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Cookie __hstc
Duration 6 months
Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).

Cookie hubspotutk
Duration 6 months
Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.

Cookie _ga
Duration 1 year 1 month 4 days
Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.

Cookie _ga_*
Duration 1 year 1 month 4 days
Description Google Analytics sets this cookie to store and count page views.

Cookie __Host-psifi.analyticsTrace
Duration 6 hours
Description Description is currently not available.

Cookie __Host-psifi.analyticsTraceV2
Duration 6 hours
Description Description is currently not available.

Cookie _gh_sess
Duration session
Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.

Cookie YSC
Duration session
Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.

Cookie ajs_anonymous_id
Duration 1 year
Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.

Cookie vuid
Duration 1 year 1 month 4 days
Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website.

Performance

[x]

Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Cookie AWSALB
Duration 7 days
Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.

Cookie acq
Duration past
Description Description is currently not available.

Cookie acq.sig
Duration past
Description Description is currently not available.

Cookie ptc
Duration 2 years
Description No description available.

[x]

Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Cookie muc_ads
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website.

Cookie guest_id_marketing
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to identify and track the website visitor.

Cookie guest_id_ads
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to identify and track the website visitor.

Cookie personalization_id
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.

Cookie guest_id
Duration 1 year 1 month 4 days
Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.

Cookie bcookie
Duration 1 year
Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.

Cookie __Secure-ROLLOUT_TOKEN
Duration 6 months
Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.

Cookie yt.innertube::nextId
Duration Never Expires
Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.

Cookie yt.innertube::requests
Duration Never Expires
Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.

Cookie session_tracker
Duration session
Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.

Cookie edgebucket
Duration session
Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.

Cookie did
Duration 1 year
Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.

Uncategorised

Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet.

No cookies to display.

Reject All Save My Preferences Accept All

Skip to content

Image 3: Towards Data Science

Publish AI, ML & data-science insights to a global community of data professionals.

Sign in

Submit an Article

* *

Write For TDS

Image 4: Towards Data Science

Toggle Mobile Navigation

Toggle Search

Search

Artificial Intelligence

Why Your AI Demo Will Die in Production

95% of enterprise AI pilots fail to launch. Why?

Ari Joury, PhD

May 18, 2026

7 min read

Share

Image 5: Photograph of a winding desert road through reddish hills, muted copper and terracotta palette, lonely cinematic mood

If you thought that the journey from pilot to production was a smooth high road, you’re mistaken. Image generated with Leonardo AI

If you have spent any time in enterprise AI over the last two years, you know the pattern. A small team builds a proof-of-concept using a state-of-the-art Large Language Model (LLM). The demo is spectacular. The executive sponsor is thrilled. The budget is approved.

And then, six months later, the project is… abandoned?

The statistics are grim. According to recent industry analyses, roughly 95% of embedded or task-specific generative AI pilots never make it into production. The failure rate is staggering, but the reasons behind it are rarely discussed with engineering rigor.

When a project fails, the post-mortem usually blames the model (“it hallucinated too much”) or the data (“we didn’t have the right context”). But having transitioned from theoretical particle physics to founding an enterprise AI company, I have seen that the root causes are almost never purely algorithmic.

The failure is structural. It is the result of accumulating what I call Production Debt.

When you build a demo, you are optimizing for a “happy path.” You’re just trying to show that your idea can even be built in practice.

When you build for production, you are building a complex, probabilistic system that must survive in a deterministic, unforgiving enterprise environment. The gap between those two states, pilot and production, is defined by five specific types of debt.

If you want your agentic system to survive, you must pay them down.

1. Technical Debt: The Fragility of Prompts

In a demo, a hardcoded prompt is sufficient. In production, it is a liability.

Technical debt in agentic systems usually manifests as brittle orchestration. You treat the LLM like a deterministic function, assuming that a specific input will always yield a specific structural output. When the model inevitably deviates—perhaps by wrapping a requested JSON object in markdown backticks—the downstream pipeline shatters. As noted in recent discussions on agentic AI challenges, ensuring reliability and predictability is paramount.

This fragility is compounded when teams attempt to chain multiple LLM calls together without robust error handling. A failure in step one cascades through the entire system, leading to unpredictable and often catastrophic outcomes. The solution is not to write a “better prompt,” but to build a system that anticipates and gracefully handles failure. The shift from passive LLMs to agentic AI systems requires a fundamental change in how we approach software architecture.

The Fix: Move from prompt engineering to systems engineering. Implement strict data contracts using libraries like Pydantic. Enforce input validation before the prompt is ever sent, and use structured output constraints (like OpenAI’s JSON mode or function calling) to guarantee the shape of the response. If the output fails validation, the system must fail fast and trigger a retry loop, rather than passing malformed data downstream.

2. Operational Debt: The Ownership Vacuum

Who owns the AI agent when it goes down at 2 AM?

In many organizations, the data science team builds the model, but they do not know how to maintain infrastructure. The DevOps team knows infrastructure, but they do not understand how to debug a probabilistic failure in an LLM chain. This ownership vacuum is Operational Debt. The complexity of orchestration explodes fast when moving to production.

This vacuum becomes glaringly obvious during the first major incident. When an upstream API changes its rate limits, or a new model version subtly alters its response formatting, the system breaks. Without clear ownership, the resolution time stretches from minutes to days, eroding trust in the entire AI initiative.

Furthermore, the lack of ownership often leads to a lack of proper monitoring. Teams might track basic metrics like API uptime, but they fail to monitor the specific health indicators of an LLM system, such as token usage spikes or context window saturation.

The Fix: Treat AI agents as tier-one microservices. This means establishing a clear RACI matrix before launch. It requires building monitoring dashboards that track not just latency and error rates, but token consumption and context window saturation. It demands documented runbooks and an on-call rotation. If you cannot answer the question “Who gets paged when the agent hallucinates?”, you are not ready for production.

3. Evaluation Debt: The “Vibe Check” Fallacy

How do you know if your new model is better than the old one? If your answer involves reading a few outputs and deciding it “feels better,” you are drowning in Evaluation Debt.

Vibes-based assessment is the silent killer of AI projects. Without objective, quantifiable metrics, you cannot safely iterate on your system. You might fix a bug in one edge case while silently degrading performance across ten others.

This is particularly dangerous in agentic systems, where the output is not just text, but a sequence of actions. A “vibe check” cannot tell you if the agent is making the optimal sequence of API calls, or if it is taking unnecessary steps that inflate costs and latency. As agentic AI handles complex tasks, the need for rigorous evaluation becomes even more critical.

The Fix: Build automated test suites and golden datasets. You must define decision-grade metrics that go beyond simple accuracy. Measure reliability (does the same input consistently produce a good output?), latency (is it fast enough for the workflow?), and cost (is the token usage sustainable?). Every code change or prompt update must be run against this automated scorecard before deployment.

4. Integration Debt: The Vacuum Chamber

An AI agent that generates perfect insights is useless if it cannot deliver those insights to the systems where work actually happens.

Integration Debt occurs when an AI system is built in a vacuum, without a deep understanding of the downstream APIs, legacy databases, and user interfaces it must interact with. The AI might generate a perfectly valid date format, but if the legacy CRM expects a different format, the integration fails.

This debt is often the result of siloed development teams. The AI team builds the agent, and the engineering team is expected to “wire it up.” But without co-designing the interfaces, the resulting integration is brittle and prone to failure.

Moreover, integration debt often manifests as a failure to handle state. Agentic systems frequently need to maintain context across multiple interactions, but if the integration layer is stateless, the agent will constantly lose track of what it is doing.

The Fix: API mocking and schema alignment must happen on day one. Do not build the AI logic and then try to wire it up later. Define the API contracts first, build integration tests, and ensure the agent’s output is strictly typed to match the expectations of the receiving system.

5. Governance Debt: The Compliance Wall

This is the debt that kills projects the day before launch.

You have built a brilliant agent that automates customer support. But you didn’t loop in the legal or compliance teams. Suddenly, questions arise about data privacy, PII redaction, and audit trails. Because the system was not designed with governance in mind, retrofitting it is impossible, and the project is shelved.

In regulated industries like finance and healthcare, governance is not an optional feature; it is a prerequisite for deployment. Failing to account for it early in the development lifecycle is a guaranteed path to failure.

Furthermore, governance debt often includes a lack of explainability. If an agent makes a decision that negatively impacts a customer, you must be able to explain why that decision was made. If your system is a black box, you cannot meet this requirement.

The Fix: Governance cannot be an afterthought, especially in regulated industries. You must design for auditability from the ground up. This often means implementing Human-in-the-Loop (HITL) approvals for high-risk actions, building immutable audit logs of every decision the agent makes, and ensuring that data retention policies are strictly enforced at the orchestration layer.

The Path Forward

The transition from a successful demo to a reliable production system is not about finding a better foundation model. It is about acknowledging that AI systems are dynamic, probabilistic entities that require rigorous engineering discipline to tame.

By systematically identifying and paying down these five debts, you can move your projects out of the lab and into the enterprise.

If this piece showed you one thing, then that it’s not easy to go to production. If you want to be among the 5% of pilots that actually make it, you now know what to do: Start paying down the debts you might have not even known you had.