When the error body is HTML — classifying provider failures you can't parse

Every AI provider documents a tidy JSON error envelope. Anthropic nests a typed error object; OpenAI and Mistral nest type and code; Google sends a numeric code beside a gRPC-style status string. Build your error handling against those shapes and it works beautifully, right up until the day the response is not from the provider at all.

Ours came from a timeout. A long upstream call died at the infrastructure in front of the provider, and what arrived was Cloudflare's 524 page — HTML, doctype and all — where a JSON envelope should have been. The proxy's job at that moment is to log what failed so the dashboard can say something useful, and every assumption in that path — parse the body, read error.type, store it — was written for a JSON world. An HTML page is not a malformed version of the contract. It is a different sender: the error channel is exactly where intermediaries speak, because edges and load balancers answer instead of the origin precisely when things go wrong. The error path sees non-provider bodies at the worst possible frequency — rarely enough that nothing forces you to handle them, reliably enough that they will eventually carry your worst outage.

Rule one: store the finding, never the evidence#

The redesign starts from a decision about what an error record is. Ours stores exactly two facts per failed call: the HTTP status, and the provider's machine-readable error code. Never the error message.

The message is withheld because it carries request content, not out of vague caution. Anthropic's invalid_request_error quotes the offending field and often its value — "prompt is too long: 215000 tokens > 200000" is a fact about the customer's prompt. OpenAI's echoes the parameter. Sanitising messages would mean maintaining a denylist against a response format we do not control and that changes without notice, and a privacy control that cannot be tested against its input space is not a control. So the message never reaches storage at all: the code is the finding, the message is evidence, and we store findings. If your telemetry stores provider error messages verbatim today, it stores fragments of your users' prompts today.

Rule two: the code is untrusted input wearing a helpful face#

The code itself arrives in a response body from over the network, and the dashboard renders it — which makes it injection surface unless proven otherwise. So it runs a gauntlet: lowercase; a shape gate (^[a-z_]{1,64}$ — bounded length, closed character class); then an allowlist of codes we have actually seen and named, spanning all four envelope grammars, including their quirks — Google's code field is the numeric HTTP status, so the useful value is status; Mistral sometimes flattens the fields onto the body root, where the literal string "error" must be excluded because that is Anthropic's root discriminator, not a code. Anything that fails any stage is stored as one sentinel: unrecognized.

The HTML page never gets near any of this, and needs no special case: it is not an object, so code extraction yields null, which normalises to unrecognized beside its 524. That is the correct record — "the upstream failed with a 524 and no provider-shaped error" is the finding, and it is visibly different from overloaded_error in a way that tells you which layer to page. New codes get discovered safely by the same funnel: an unknown-but-shape-valid code logs to the console (safe to print — the shape gate already ran) and a human adds it to the allowlist after seeing it in the wild. The database only ever contains strings we have vouched for.

The definition underneath: what counts as "failed"#

One more decision turned out to be load-bearing: a single shared predicate for "did the provider serve the caller", used by the status column, the error classifier and the detection rules alike, so they cannot drift into three definitions of failure. It is 2xx — plus exactly 304. A 304 is the provider answering a conditional request with "your copy is current": the caller got what they asked for, and logging it as a failure had rendered a working integration as a failing agent. The exemption is deliberately not all of 3xx: a 301 arriving at the proxy was never followed to a served response, so the caller got no answer, and not-served is the honest reading — the same redirects-are-not-success lesson our own webhook paid for. Edge cases in a failure predicate are product decisions, and each one deserves a sentence saying why.

The result on the dashboard: a failed call shows 521 · unrecognized or 429 · rate_limit_error — enumerable, chartable, safe to render — and the pipeline that produces it cannot be surprised by HTML, because it never trusted the body in the first place. The timeout that started all this got its own treatment upstream: stream long calls, or cut before the edge does.