AI production issue detection is the practice of catching LLM application failures that traditional monitoring cannot see: outputs that are wrong, ungrounded, off-goal, or unsafe while every infrastructure metric — latency, error rate, uptime — stays green. It requires classifying failures by mode and matching each mode to a detection signal.
The core problem is that LLM failures are semantic, not operational. McKinsey's State of AI research has repeatedly found that inaccuracy is the generative AI risk organizations most often experience in practice — ahead of security or privacy incidents — yet most production monitoring stacks are built to catch the latter and are blind to the former. An agent that calls the wrong tool with plausible arguments returns a 200. A chatbot that contradicts itself on turn 9 throws no exception. A model update that quietly degrades one use case moves no error-rate chart.
This guide gives you two things: a practical taxonomy of six recurring production failure modes, and a five-layer detection framework that maps each mode to the signal that catches it. Both are tool-agnostic — the framework describes capabilities, and you can implement it on whatever observability stack you run.
Why does traditional monitoring miss AI failures?
Traditional monitoring detects failures that announce themselves — exceptions, timeouts, resource exhaustion. LLM failures rarely announce themselves: the model always produces something, and the something is often fluent, well-formatted, and wrong. Detection therefore has to evaluate output content and behavior, not just execution status.
Three properties of LLM systems break the classical monitoring model:
- Failures return successfully. A hallucinated answer, a wrong tool call, and an ignored escalation request can all complete without errors. Status-code monitoring cannot detect these semantic failures by itself.
- Failures are distributional, not binary. Quality degrades a use case at a time — refund conversations get worse while everything else holds. Aggregate metrics hide localized regressions until users complain.
- The system changes under you. Model provider updates, prompt edits, index refreshes, and shifting user behavior all move output quality without any deploy on your side. There is no stable baseline unless you build one.
The consequence: issue detection for AI applications is an evaluation problem wearing a monitoring costume. The rest of this guide treats it that way.
The failure taxonomy: six modes
Production LLM failures cluster into six recurring modes: tool misuse, context loss, goal drift, silent quality degradation, retrieval failure, and policy violation. Each mode has a distinct symptom, lives at a distinct level of the trace hierarchy, and requires a distinct detection signal — which is why one generic "quality score" catches almost none of them.
1. Tool misuse
The agent calls the wrong tool, calls the right tool with wrong or fabricated arguments, ignores a tool it should have used, or loops on the same tool repeatedly. Tool misuse is the signature failure of agentic systems, and it is invisible unless tool spans capture both the arguments the model generated and the results returned. A refund agent that calls issue_refund with an amount the user never stated has failed even though the API call succeeded.
2. Context loss
The application drops information it previously held: a chatbot forgets the order number given on turn 2, an agent re-asks for data the user already provided, a long-running task contradicts its own earlier output. Context loss is a cross-turn failure — each individual response looks fine in isolation, which is precisely why request-level monitoring cannot see it. Detection requires evaluating the full thread, not the trace.
3. Goal drift
The system stops serving the user's actual objective while continuing to produce competent-looking output. A research agent asked for competitor pricing that returns a general market overview has drifted; so has a support bot that turns a cancellation request into a retention pitch. Goal drift is the hardest mode to catch with static rules because every individual step can be locally reasonable — only comparing the outcome against the original intent reveals it.
4. Silent quality degradation
Output quality declines gradually with no triggering incident — a model version update, an accumulating prompt patch, seasonal shift in user inputs, or slow decay of a retrieval index. This is the mode that erodes trust in AI products over months rather than breaking them in a day. Detection requires trending evaluation scores over time against a baseline, segmented by use case and version, because the aggregate almost always looks stable while a segment sinks.
5. Retrieval failure
The generation step is fine, but the evidence feeding it is wrong: the retriever returns irrelevant chunks, stale documents, or misses the relevant passage entirely. Retrieval failures masquerade as hallucinations — the model faithfully summarizes bad context — which sends teams to tune prompts when they should be fixing the index. Only span-level capture of queries and returned chunks separates the two.
6. Policy and safety violation
The output leaks PII, produces off-brand or harmful content, gets manipulated by prompt injection, or takes an action outside its authorization. These are lower-frequency, higher-severity failures, and they are the subject of dedicated frameworks — the OWASP Top 10 for LLM Applications catalogs the adversarial variants, and the NIST AI Risk Management Framework frames the governance side. Detection differs from the other modes: violations warrant per-trace screening, not sampling, because a single incident carries the cost.
The diagnostic table
Use this table as the working artifact: when an issue surfaces, identify the symptom, and the table tells you which mode you are dealing with, where in the trace hierarchy to look, and which signal should have caught it.
Failure mode | Typical symptom | Trace level | Primary detection signal |
|---|---|---|---|
Tool misuse | Wrong action taken; task "completed" with wrong side effects | Span (tool) | Tool-correctness evaluation on tool spans; argument validation against user intent |
Context loss | Repeated questions; contradictions across turns; forgotten facts | Thread | Knowledge-retention and turn-relevancy metrics on closed threads |
Goal drift | Output is competent but does not answer what was asked | Trace + thread | Task-completion / intent-fulfillment evaluation comparing outcome to the original request |
Silent quality degradation | Slow rise in complaints; no incident to point to | Population over time | Score trending vs. rolling baseline, segmented by use case and prompt/model version |
Retrieval failure | Confident answers that are wrong; "hallucinations" that trace to bad context | Span (retrieval) | Contextual relevancy on retrieval spans; faithfulness on generation spans to localize |
Policy / safety violation | PII in output; injected instructions followed; unauthorized action | Trace (every) | Per-trace screening: PII detection, injection heuristics, action allow-lists |
Two patterns in the table are worth calling out. First, the trace-level column is why instrumentation depth determines detection ceiling: without tool and retrieval spans, modes 1 and 5 are undetectable; without thread grouping, modes 2 and 3 are. Second, no single evaluation metric covers more than one row — a detection strategy is necessarily a portfolio.
The detection framework: five layers
The framework has five layers, ordered by adoption sequence: instrumentation, signal classification, online evaluation, drift detection, and alerting with human review. Each layer catches failures the previous one cannot, and each depends on the ones below it — teams that skip layers get blind spots exactly where the skipped layer's failure modes live.
Layer 1: Instrumentation — make failures observable
Every request produces a trace; every model call, retrieval, and tool execution produces a span with its semantic payload; every conversation carries a thread ID. This layer detects nothing by itself, but it determines what can be detected: the taxonomy above maps failure modes to trace levels, and a mode whose level you do not capture is a mode you cannot catch. If you are starting from zero, the data model is covered in What Is LLM Tracing?.
Layer 2: Signals — classify before you score
Before committing to quantitative metrics, run lightweight qualitative classification over traces and threads: user sentiment, user intent, anomaly flags, error-pattern clusters. Signals answer "what is happening out there?" without requiring you to have already decided what to measure — and the patterns they surface are the best input for choosing which evaluation metrics to invest in. For user-facing applications, sentiment and escalation-request signals routinely surface failure clusters weeks before any metric is configured to catch them.
Layer 3: Online evaluation — score what matters
Run evaluation metrics on production traffic at chosen trigger moments: after response generation first, after tool execution second, on individual spans third, on closed threads last (the sequencing rationale is covered in setting up trigger moments for online evals). This is the layer that catches tool misuse, retrieval failure, and goal drift per the diagnostic table. Metrics must be validated against human judgment before you trust them — an unvalidated metric in production is a noise generator with an alert budget.
Layer 4: Drift detection — compare against a baseline
Trend evaluation scores and signal rates over time, segmented by use case, prompt version, and model version, and compare against a rolling baseline rather than absolute thresholds. This is the only layer that catches silent quality degradation: the mode has no per-trace signature, only a population-level trend. Segmentation is non-negotiable — a regression confined to one use case disappears into any aggregate chart.
Layer 5: Alerting and human review — close the loop
Route regressions to the channels your team already treats as incidents (Slack, PagerDuty, Teams), with the alert linking directly to the failing traces or threads. Then make review productive: confirmed failures get labeled by a human and converted into evaluation dataset entries, so every production incident permanently hardens your regression suite. Detection without this loop plateaus — the same failure modes recur because nothing accumulates.
Implementation order and success criteria
Adopt the layers in order, and hold each to a concrete acceptance test before moving on:
- Instrumentation — acceptance: for any user-reported bad output, you can locate its trace and walk the span tree in under two minutes.
- Signals — acceptance: you can name your top three production failure patterns from signal data, not from support tickets.
- Online evaluation — acceptance: at least one metric is validated against human labels, with false-positive and false-negative rates documented and acceptable for the use case, and running on live traffic.
- Drift detection — acceptance: a deliberately degraded canary (or a known past regression) is visible in the segmented trend view.
- Alerting + review loop — acceptance: the last real regression produced an alert, a labeled failure, and a new dataset entry — without an engineer doing manual export.
Implementation time depends on application complexity, traffic, and review requirements, but the layer order should not change. The common failure pattern is inverting it: configuring alerts on unvalidated metrics over uninstrumented applications, which produces alert fatigue and weak detection.
Platform choice matters less than the framework, but it does gate how much of it you can implement — the capabilities to check for are span-level payload capture, thread grouping, evaluation on production traffic, segmented drift views, and trace-to-dataset conversion. Confident AI implements all five layers natively; whatever stack you choose, evaluate it against the layers, not the feature list.
Frequently Asked Questions
What is AI production issue detection?
AI production issue detection is the practice of catching semantic failures in deployed LLM applications — wrong outputs, misused tools, lost context, off-goal behavior, quality drift, and policy violations — that return successfully and therefore evade traditional error monitoring. It combines trace instrumentation, qualitative signals, online evaluation, drift detection, and alerting into a layered detection system.
What are the most common LLM failure modes in production?
Six recurring modes form a useful working taxonomy: tool misuse (wrong tool or arguments), context loss (forgotten or contradicted information across turns), goal drift (competent output that misses the user's objective), silent quality degradation (gradual decline with no triggering incident), retrieval failure (bad evidence feeding a fine generator), and policy violations (PII leakage, prompt injection, unauthorized actions).
Why don't standard APM tools detect AI failures?
Because LLM failures are semantic, not operational. A hallucinated answer, a wrong tool call, and a contradicted promise all return successful status codes with normal latency. APM tools monitor execution health and deliberately discard the payloads — prompts, completions, retrieved context, tool arguments — that detection of semantic failures requires.
How do I detect silent quality degradation in an LLM application?
Trend evaluation scores over time against a rolling baseline, segmented by use case, prompt version, and model version. Silent degradation has no reliable per-trace signature — each individual output may look acceptable — so population-level comparison is what reveals it. Calibrate relative-regression alerts from historical variance and incident tolerance rather than copying one percentage across use cases.
How do I tell retrieval failures apart from hallucinations?
Capture retrieval spans with the query and returned chunks, then run contextual relevancy on the retrieval span and faithfulness on the generation span. If retrieval returned bad context and the model summarized it faithfully, fix the index or the query strategy. If retrieval was fine and the output contradicts it, fix the prompt or the model. Without span-level capture, the two are indistinguishable and teams routinely fix the wrong one.
Should evaluation metrics run on every production trace?
Not uniformly. Policy and safety screening (PII, injection) should run per-trace because single incidents carry the cost. Quality metrics can run on sampled traffic or at specific trigger moments — after response generation, after tool execution — and expand as validated. Signals (lightweight classification) are cheap enough to run broadly and are the right default before metrics are validated.
Read next
- What Is LLM Tracing? Traces, Spans, and Threads Explained — the instrumentation layer this framework builds on.
- Setting Up Trigger Moments for Online Evals — the sequencing playbook for layer 3.
- Setting Up Multi-Turn Agent Observability — detecting the cross-turn failure modes (context loss, goal drift) on threads.