Traditional software calls an API, and if the API fails, the user sees an error and retries. Annoying, survivable. Agents are different in a way that took me embarrassingly long to internalize: they chain calls. A research agent might run a search, scrape four pages, parse a PDF, and embed the results before it produces anything. Every link in that chain is a place to die, and the failure math compounds.
The arithmetic is simple and brutal. If each tool call succeeds 99% of the time, a 20-call chain completes about 82% of the time. At 95% per call, the 20-call chain completes about 36% of the time. Nobody ships a feature that works one time in three, but that is what “my scraper is usually fine” turns into once an agent is doing the calling.
The three ways tool calls actually fail
Running failover across roughly 30 providers gives me a decent taxonomy of what goes wrong. It clusters into three families.
Rate limits and 429s. Every provider has them, they are often per-key rather than per-account, and agents are spectacularly good at hitting them because agents burst. A human does one search; an agent fans out eight. A 429 is not really an error, it is the provider telling you to go away for a while, and the correct response is usually to go somewhere else instead.
Blocks and soft failures. Scraping is the poster child: the request returns 200 and the body is a cookie wall, a bot challenge, or an empty JS shell. Your HTTP client calls that success. Your agent then confidently summarizes an empty page. In my experience these soft failures are more damaging than hard errors precisely because nothing throws, which is why our router treats an empty scrape body as a failure and moves on to the next provider.
Outages and brownouts. Full outages are rare and obvious. Brownouts, where a provider’s error rate quietly climbs to 20 or 40% for half an hour, are common and sneaky. Status pages lag reality. If your reliability strategy is “check the status page”, you have a reliability strategy with a fifteen minute detection delay.
Retry is not failover
The reflexive fix is retry with exponential backoff, and for transient blips it works. But retrying the same provider assumes the problem is transient and random. Most of the failures above are neither. A 429 means retries make it worse. A block on your scraper’s IP range will still be there in eight seconds. A brownout lasts longer than any sane backoff window.
Failover means the retry goes to a different provider. That only works if a different provider exists in your stack, which is the real argument for multi-vendor tooling: not price leverage, survival. Our router tries up to three providers per request, and the response includes the full routing.attempted chain, so you can see exactly which providers failed and which one finally answered. That transparency matters when you are debugging why a task took nine seconds instead of two.
Two refinements make failover production-grade rather than decorative:
Circuit breakers. If a provider is failing 30% of its requests over the last five minutes, it should not be first in anyone’s chain. A circuit breaker skips it until it recovers, which protects both your latency (no waiting for a doomed attempt) and the provider (no pile-on). That threshold and window are exactly what our router uses, and I will write up the pattern properly in circuit breakers for AI agents.
Wall-clock budgets. Failover without a time limit converts outages into hangs. Each request needs a total budget across all attempts: ours is 20 seconds for search and scales by category up to 330 seconds for video generation. When the budget is spent, fail loudly. An agent that waits forever is worse than an agent that errors, because at least the error is visible.
Failure domains: the part everyone skips
Here is the subtle failure of naive failover: falling back to a provider that fails for the same reason as your first choice. Two Google-SERP resellers share Google as an upstream; when the upstream has a problem, your “redundancy” fails as one unit. Real failover pairs providers from independent failure domains, which is why Brave’s independent index earns a place in a search chain despite costing more than a SERP reseller ($6.00 per 1k queries routed versus Serper’s $1.20 in our catalog). I go deeper on that specific pairing in Serper vs Brave.
The same logic applies everywhere: pair a scraper that renders JavaScript with one that does not, pair a managed transcription service with a hosted open-weights model. Diversity of implementation is what you are buying, not just a second logo.
Build it or buy it, but have it
You can absolutely build this yourself: two SDKs, a try/except, a health counter, a deadline. Plenty of teams do, and for one category with two providers it is a reasonable weekend project. The maintenance is the part that grows: every new category doubles the surface, and the health stats only get good with traffic volume. That tradeoff, wrapper versus router, deserves its own honest post. In the meantime, whichever way you go, the principle stands: no unattended agent should have a single point of vendor failure.
If you want to see what multi-provider chains look like per category, start with the search API comparison.