Correction (2026-10-12): This post originally described a 75-platform cohort with figures that do not match the published ARS 2026 dataset. It now reports the dataset's actual numbers: 100 platforms, mean ARS 50.7, median 52, highest score 79 (Runway), no platform at 80 or above, and the real category means and benchmark pass rates. We removed the llms.txt score differential (the dataset cannot measure llms.txt adoption) and the documentation-ownership, code-sample, authentication-failure-share and team-size findings (no supporting data). We also clarified that the benchmark is a simulated, deterministic traversal, not live agent runs. Full list with evidence: ARS 2026 errata (also in the research repository).
We spent months pointing a simulated AI agent at software.
Not to test the AI — to test the software.
Our benchmark methodology is simple in principle and laborious in practice: take a standardized set of agent journeys and run them, as a deterministic simulated traversal, against the documentation, API surfaces, and SDK ecosystems of 100 AI and developer infrastructure platforms. Score each platform on 10 benchmarks. Compile the results.
The findings are the State of Agent Readiness 2026 Report.
This is the summary. For the full dataset and methodology, the complete report lives at glintbase.dev/research.
The central finding: almost every platform in our 2026 cohort is easy for an agent to find — and hard for an agent to actually use. 95% pass the cold-start discoverability benchmark, but no benchmark past the machine entrypoint is passed by more than 29% of platforms, and not one of the 100 platforms passes rate-limit transparency.
What the ARS 2026 Benchmark Measures
The Agent Readiness Score (ARS) is a 0–100 composite. In the 2026 edition, each platform is run through 10 benchmarks. Each benchmark is passed or failed, and each records how many hops the traversal needed.
| Benchmark | What It Checks | Cohort Pass Rate |
|---|---|---|
| B-01 Cold Start Discoverability | Can an agent find the documentation entry point from nothing? | 95% |
| B-02 Machine Entrypoint Quality | Is there a usable machine-facing entrypoint? | 94% |
| B-03 Auth Path Completeness | Can the full authentication path be followed? | 21% |
| B-04 Quickstart Completeness | Does the quickstart contain everything needed to finish it? | 18% |
| B-05 SDK Discoverability | Can the SDKs be located and identified? | 29% |
| B-06 API Reference Navigability | Can the API reference be navigated to a specific operation? | 18% |
| B-07 Core Concept Disambiguation | Are the core concepts defined unambiguously? | 16% |
| B-08 Error Surface Completeness | Are errors documented well enough to recover from? | 11% |
| B-09 Rate Limit Transparency | Are rate limits documented where an agent can find them? | 0% |
| B-10 Versioning & Change Signal | Are versions and changes signalled clearly? | 22% |
Scoring is performed by automated traversal, not human review. It is a simulated, deterministic traversal run by our Pathfinder scanner: 1,000 scans, 100 platforms × 10 benchmarks. The scanner does not call an LLM. It records pass/fail and hop counts, and its token figures are modelled from hop counts rather than measured from live agent runs.
The Headline Numbers
Cohort: 100 AI and developer infrastructure platforms across 10 categories (Foundation Models, AI Infrastructure, Vector Databases, Observability & Evals, Orchestration & Frameworks, Dev Tools & Media APIs, Data & Annotation, Agent & Automation, Security & Guardrails, and Gateways & Edge).
Median ARS: 52 out of 100
Mean ARS: 50.7 out of 100
Highest Score: 79 (Runway, Dev Tools & Media APIs)
Lowest Score: 14 (AutoGen, Orchestration & Frameworks)
Platforms scoring 80 or above: 0
Platforms scoring 70–79: 5 (Runway 79, Letta 73, Exa 72, Cerebrium 71, OpenRouter 71)
Platforms scoring below 40: 23 (23% of cohort)
| ARS Tier | 80 and above | 70–79 | 50–69 | 30–49 | Below 30 |
|---|---|---|---|---|---|
| Platforms | 0 | 5 | 49 | 38 | 8 |
Nearly half of the cohort — 46 of 100 platforms — scores below 50, and not one platform reaches 80. The median platform passes only 2 of the 10 benchmarks. In practice, an agent can find the docs, then hits friction at auth, quickstart, SDK and reference depth.
Category Breakdown
Performance varied significantly across software categories.
| Category | N | Avg ARS | Top Score | Bottom Score |
|---|---|---|---|---|
| J Gateways & Edge | 2 | 67.0 | 71 (OpenRouter) | 63 (Cloudflare AI) |
| F Dev Tools & Media APIs | 14 | 56.3 | 79 (Runway) | 31 (E2B) |
| D Observability & Evals | 10 | 53.0 | 68 (Helicone) | 30 (WhyLabs) |
| E Orchestration & Frameworks | 10 | 52.9 | 73 (Letta) | 14 (AutoGen) |
| C Vector Databases | 10 | 52.4 | 72 (Exa) | 38 (Jina AI) |
| H Agent & Automation | 8 | 52.4 | 60 (AgentOps) | 41 (Fixie AI) |
| B AI Infrastructure | 12 | 51.8 | 71 (Cerebrium) | 37 (Lambda Labs) |
| I Security & Guardrails | 4 | 45.8 | 58 (Lakera) | 37 (Protect AI) |
| A Foundation Models | 24 | 44.7 | 66 (Cohere) | 23 (Z.ai) |
| G Data & Annotation | 6 | 44.2 | 61 (Labelbox) | 24 (Cleanlab) |
Gateways & Edge has the highest mean, but with only two platforms that is not a meaningful lead. Among the larger categories, Dev Tools & Media APIs (14 platforms) leads at 56.3.
Foundation Models — the largest category, at 24 platforms — is second-lowest at 44.7, ahead of only Data & Annotation. The companies building the models agents run on are, on average, among the harder platforms for those agents to integrate with.
The Five Findings That Surprised Us
Finding 1: Discovery is nearly solved. Everything after it is not.
95% of platforms pass B-01 (cold start discoverability) and 94% pass B-02 (machine entrypoint quality). After that, pass rates collapse: 29% for SDK discoverability, 22% for versioning, 21% for auth, 18% for quickstart and for API reference, 16% for concept disambiguation, and 11% for error surfaces.
Finding 2: Not one platform documents rate limits where an agent can find them.
B-09 (rate limit transparency) has a 0% pass rate across all 100 platforms. For an agent, an undocumented rate limit is indistinguishable from a random failure.
Finding 3: Foundation model providers are among the least agent-ready platforms.
Category A (Foundation Models, 24 platforms) averages ARS 44.7, second-lowest of the 10 categories. OpenAI scores 35 (rank 85 of 100) and Anthropic 57. Cohere is the category's best at 66.
Finding 4: Authentication is the first wall.
Only 21% of platforms pass B-03 (auth path completeness). Common ways documentation leaves the auth path incomplete:
- OAuth flow described without clear redirect URI examples
- API key scoping not explained
- Token expiry and refresh not covered
- Sandbox vs. production credential differences not distinguished
Authentication is the gate that determines whether an agent can access any other part of your product, so its quality has outsized leverage.
Finding 5: The top of the table is not the usual names.
The five highest scores belong to Runway (79), Letta (73), Exa (72), Cerebrium (71) and OpenRouter (71). The lowest is AutoGen (Microsoft) at 14. Brand size and market position did not line up with agent readiness in this cohort.
The published dataset includes each platform's ARS, the pass/fail status and hop count for each of the 10 benchmarks, and modelled token figures for all 100 platforms. Token figures are a fixed function of hop count, not measured LLM consumption.
What High-Scoring Platforms Have in Common
We looked at the five platforms that scored 70 or above to identify the structural patterns they share. These are qualitative observations from reading their documentation, not measured benchmark results.
They treat documentation as a release artifact. Documentation updates ship with code changes. Stale examples are caught in CI before they reach production. The documentation is versioned alongside the SDK.
They publish structured machine-readable context. Machine-facing indexes and specifications give agents a reliable starting point. (The dataset's surface flags for llms.txt, OpenAPI and MCP are set for every platform, so they cannot be used to separate high and low scorers.)
They front-load prerequisites. Every guide begins with a clear prerequisites section: what you need installed, what credentials you need, what version of the SDK is expected. This pattern removes a common source of agent failure.
They maintain a single source of truth per concept. High-scoring platforms resist the temptation to explain the same concept in multiple places. Repetition creates inconsistency. Inconsistency creates ambiguity. Ambiguity is a major driver of agent hallucination.
They expose a flat, navigable documentation structure. Deep nesting — documentation buried four or five levels into a navigation tree — increases discovery overhead. Top platforms keep their most important content within a few navigational steps of the homepage.
What Low-Scoring Platforms Have in Common
The bottom of the table shows a recognizable pattern too (again, qualitative observations rather than measured results).
Documentation is treated as a separate product from the software. It is maintained on a different schedule, by a different team, with different quality gates. The result is inevitable drift between documentation and implementation.
Examples are aspirational, not tested. Code samples are written to illustrate concepts, not to be executed. They reference the API as it was designed, not necessarily as it currently behaves.
Machine-readable context is thin or stale. An outdated OpenAPI spec, no semantic metadata, an index that points nowhere useful. Agents must discover the product structure through expensive HTML traversal.
Error guidance is missing. Error code documentation is sparse or absent — only 11% of the cohort passes B-08. When an agent encounters a failure, it has no reliable way to diagnose the cause from documentation alone.
The Trajectory
The 2026 benchmark establishes the baseline.
We intend to re-run this benchmark annually. Our expectation — a forecast, not a measured result — is that the median ARS will climb as documentation practices adapt to the agentic era.
The gap between the top and bottom of the table will also narrow, but not because the bottom improves quickly. It will narrow because platforms that do not improve will lose developer adoption to platforms that do.
Agent-mediated discovery is becoming the dominant mode through which developers evaluate new software integrations. If your platform is hard for agents to navigate, it will increasingly be invisible to developers.
Read the complete State of Agent Readiness 2026 Report → glintbase.dev/research
