The demo worked. The operating system around it did not.
The team proved that the model could produce an impressive result. Leadership saw the demo, approved more budget and expected the capability to become part of the business.
Then the project slowed down.
The prototype depends on clean sample data. The real workflow has exceptions the demo never encountered. Nobody has decided which outputs require human approval. The tool works in isolation but not inside the systems employees already use. Cost rises when usage increases. Users try it once and return to the old process.
The pilot did not necessarily fail. It reached the point where a technical possibility must become an operating capability.
That requires a different kind of work.
What production actually means
Production does not simply mean the application is hosted or available to users.
An AI capability is in production when it can repeatedly perform a defined job inside a real operating environment with acceptable quality, cost, speed and risk.
That usually requires:
- A clear business outcome
- A defined user and workflow
- Access to reliable data and systems
- Evaluation against realistic cases
- Monitoring after release
- Human review where consequences require judgment
- Permission, security and privacy controls
- A recovery path when the system fails
- Named ownership for ongoing performance
- Evidence that intended users actually rely on it
Google Cloud’s production guidance treats deployment, monitoring, continuous evaluation and operational reliability as connected parts of the AI system. The model cannot be separated from the environment in which it is expected to work.
Why pilots stall before production
The proof of concept answered the wrong question
The pilot proved that AI could generate the output. It did not prove that the output solved the user’s complete problem.
The workflow was simplified for the demo
Real work contains incomplete information, conflicting priorities, exceptions, approvals and recovery. These are often removed to make the pilot move quickly.
Nobody owns the complete deployment
Engineering owns the build. Product owns the feature. Operations owns the process. Security owns risk. The users own adoption.
Everyone contributes. Nobody is accountable for the complete outcome.
Evaluation is anecdotal
The team reviews a handful of strong outputs but has not defined acceptable performance, failure categories or the cases that matter most.
Production economics were ignored
A prototype may be affordable at low volume and unsustainable when every task triggers several model calls, large context windows or expensive retrieval.
Users were introduced too late
The team built what leaders imagined rather than what users could adopt inside their actual work.
Start with the production outcome
Do not begin by asking what remains on the technical backlog.
Ask what must become true for the business.
Examples include:
- Reduce the time required to complete a recurring workflow.
- Increase the number of customer requests a team can resolve without lowering quality.
- Help sales representatives prepare accurate account research before a meeting.
- Turn internal knowledge into reliable first-draft responses with human approval.
- Detect and route operational exceptions earlier.
Define:
- The user
- The job they are trying to complete
- The current baseline
- The expected improvement
- The acceptable error or risk
- The evidence of success
- The owner of the final outcome
The production architecture should follow that definition.
Map the real workflow
Sit with the people doing the work.
Document:
- What triggers the workflow?
- What information enters?
- Which systems are involved?
- Which decisions require judgment?
- What exceptions occur?
- Who approves consequential actions?
- What happens when information is missing?
- What is the record of the completed work?
- How does the user recover from a bad output?
The strongest production design may change the workflow rather than insert AI into every existing step.
Build the missing production layer
Data and context
Determine which information the system needs, where it comes from, who owns it and how current it must be. Define what the model is allowed to access and what must remain excluded.
Integrations
Place the capability where the work already happens. That may require connecting the AI system to the CRM, support platform, knowledge base, product or internal operating tools.
Evaluation
Build a representative set of real cases, including difficult and high-consequence examples. Score the system against criteria tied to the job, not general impressions.
Human control
Decide what the system can draft, recommend, approve or execute. Make human review visible and efficient rather than an informal safety net.
Observability
Track quality, errors, latency, cost, user behavior and changes in the production environment. NIST’s AI Risk Management Framework organizes this work through the functions Govern, Map, Measure and Manage.
Recovery
Define what happens when the model, integration or source data fails. A dependable workflow needs a fallback that users understand.
Control cost before usage expands
AI economics are part of product design.
Measure cost per completed workflow, not only cost per token or model call.
Review:
- Model choice by task
- Prompt and context size
- Retrieval volume
- Number of calls required
- Caching opportunities
- Batch processing
- Repeated or unnecessary generation
- Human time created or removed
- Infrastructure and observability costs
The least expensive model is not always the least expensive system. Poor outputs create rework, review and customer risk. The objective is reliable business value at an acceptable total cost.
Release narrowly, then expand
Begin with a defined user group and workflow.
During the first release:
- Observe real use
- Capture failure cases
- Compare results with the prior process
- Track whether users complete the workflow
- Measure cost and latency
- Interview users who stop using it
- Update the evaluation set with production evidence
Expansion should follow demonstrated reliability and adoption, not the planned launch date.
Which operator does the work require?
Forward-Deployed Engineer
Best when a specific pilot must be embedded with users, connected to real data and adapted inside the production environment.
Applied AI Engineer
Best when the company needs reusable architecture, evaluation infrastructure, monitoring and technical reliability across multiple use cases.
AI Product Lead
Best when the business outcome, user, product direction or success criteria remain unclear.
The title should follow the actual gap. Hiring another engineer will not solve unclear ownership or workflow design.
What a useful mandate sounds like
“Productionize the prototype” leaves the operating result undefined.
A stronger mandate is:
Turn the customer-support pilot into a dependable production workflow that reduces resolution time while preserving accuracy, approval and traceability. Own the integrations, evaluation, user rollout and capability transfer required to make it operational.
The mandate should state the outcome, users, systems, authority, resources, risks and evidence of adoption.
What the company should retain
- A working production workflow
- Documented architecture and integrations
- A representative evaluation set
- Monitoring for quality, cost and reliability
- Defined human review and recovery
- Real adoption evidence
- Clear ongoing ownership
- Internal capability to operate and improve the system
The project succeeds when people can depend on the capability to complete real work, not when the prototype reaches a server.