US East · Ashburn
2026-05-0499.43% over 90 daystoday
- 414ms
- 1611ms
- 11ms
Three tiers, three regions
Health is observed, not declared. Every square below is one UTC day of measured availability for one upstream in one region — and every dip on it has an entry on the incident timeline.
Degraded performance90-day window2026-05-04 → 2026-08-01
Degraded performance
Paid routes are serving normally. One or more cells are below their baseline and traffic is being weighted away from them.
7 / 9
1 degraded · 1 down
99.083%
Averaged across all 9 provider × region cells
2
Anything not yet marked resolved on the timeline below
414ms
Surplus · P50, with ~11ms of that ours
The board
cell by cell.
A tier is not a single machine and a region is not a single rack, so the honest unit is the pair. Today’s square decides the badge: below 99% reads as degraded, below 92% as an outage.
100% — clean day≥ 99.5% — routine noise≥ 99% — minor degradation≥ 95% — degraded< 95% — major outage
Surplus
Frontier models at list price. Each call settles in USDG before the stream opens.
99.450% mean over 90 daysOperational
Operational
EU West · Frankfurt
2026-05-0499.57% over 90 daystoday
- 483ms
- 1332ms
- 13ms
AP Southeast · Singapore
2026-05-0499.35% over 90 daystoday
- 591ms
- 1711ms
- 10ms
AntSeed
Redundant capacity for the same catalog — the fallback when an upstream degrades.
99.437% mean over 90 daysDegraded performance
US East · Ashburn
2026-05-0499.35% over 90 daystoday
- 476ms
- 1586ms
- 10ms
EU West · Frankfurt
2026-05-0499.33% over 90 daystoday
- 562ms
- 2010ms
- 13ms
AP Southeast · Singapore
2026-05-0499.63% over 90 daystoday
- 668ms
- 2519ms
- 14ms
Dolphin
Open-weight models served at no charge. Rate-limited, never metered.
98.363% mean over 90 daysMajor outage
US East · Ashburn
2026-05-0498.80% over 90 daystoday
- 979ms
- 2575ms
- 8ms
EU West · Frankfurt
2026-05-0498.79% over 90 daystoday
- 950ms
- 2889ms
- 8ms
AP Southeast · Singapore
2026-05-0497.50% over 90 daystoday
- 1397ms
- 5404ms
- 11ms
Latency, compared
across every cell.
LLM providers run at 99–99.5% uptime — 6 to 14× worse than cloud infrastructure. Two providers with automatic failover cut expected downtime from about 41 hours a year to about 12 minutes.
TianPan.co, LLM API Resilience in Production · 2026-03-11
bar = P50 · marker = P99 · scale to 5404ms
Surplus
US East · Ashburn
414ms / 1611ms
AntSeed
US East · Ashburn
476ms / 1586ms
Surplus
EU West · Frankfurt
483ms / 1332ms
AntSeed
EU West · Frankfurt
562ms / 2010ms
Surplus
AP Southeast · Singapore
591ms / 1711ms
AntSeed
AP Southeast · Singapore
668ms / 2519ms
Dolphin
EU West · Frankfurt
950ms / 2889ms
Dolphin
US East · Ashburn
979ms / 2575ms
Dolphin
AP Southeast · Singapore
1397ms / 5404ms
These are end-to-end request times, upstream inference included. The router’s own share is the overhead figure on each card — around 11ms at the median. A latency routing policy reads this table continuously rather than from a static config, which is why a slow region stops receiving traffic before it starts returning errors.
Incident history
written out in full.
Open entries are expanded by default; resolved ones collapse to a single line. Each note says what broke, who it hit, what we changed, and — where money was involved — whether it was refunded.
Two of six AntSeed workers in Singapore are draining after a host migration, and queue depth on frontier text routes is running 3–4× normal. Requests are completing, but TTFT on this region is roughly double its baseline. Cost-optimized routing has been temporarily weighted away from AP-Southeast; latency-optimized policies are unaffected. Replacement capacity is provisioning and we are monitoring recovery.
- 2026-07-31 · 16:48 UTC
- Still open
- −2.4 pts / day
- AntSeed · AP Southeast · Singapore
Free-tier demand in Singapore has exceeded the capacity we reserve for it, and Dolphin is returning 429 on a large share of requests to that region. Paid routes on Surplus and AntSeed are unaffected. Callers pinning the free tier should either allow fallback to a paid tier or retry against US-East. Additional open-weight capacity is being brought online; we will post again when the shed rate drops below 5%.
- 2026-07-30 · 21:36 UTC
- Still open
- −34 pts / day
- Dolphin · AP Southeast · Singapore