Engineering Changelog

Every scored request declares GlintbaseScanner and links to the bot page, which lists what a scan requests. Scans are user-initiated or weekly re-checks of leaderboard companies, so robots.txt is read as evidence but rules aimed at the scanner's own name are not an opt-out; sites can opt out of re-checks by email.

102 leaderboard sites scanned as both identities, back to back, from Vercel: no site blocked the honest identity while serving Chrome (one site challenged Chrome only), average score 49.1 with Chrome and 50.0 with the honest identity, 83 sites identical. Two sites served a JavaScript shell or an error page to the browser string; the scanner now sees what declared agents receive.

The .md twin probe derives its URL from the page path on the same host. A scan of https://example.com previously requested https://example.com.md, an unrelated host.

@glintbase/scanner-core and @glintbase/mcp move to 3.5.1.

A failed anti-crawler-challenge-free (a bot challenge or HTTP 403 on the homepage) or content-no-js (under 300 characters of prose without JavaScript and relies on scripts to render (an app root, a noscript notice, or far more script than text)) caps the score at 47, the top of grade D. Scorecards report gates[] and uncappedScore. The cap value is a policy choice.

When the homepage is blocked, the checks that read its content are skipped as not observed instead of scoring the refusal page.

llms-txt-exists and llms-txt-formatting are now bonus checks (formatting drops from 2 to 1 point), so a missing llms.txt costs nothing. Soft-404 bodies are rejected and reachability needs an href to llms.txt on the homepage.

The live probe speaks MCP 2026-07-28 (server/discover) and falls back to initialize for older servers, then runs tools/list and the error canary in the server's era.

New bonus checks: markdown-alternate-link, mcp-protocol-modern, mcp-server-card, mcp-tools-menu-size, mcp-tools-schema-dialect and mcp-auth-scope-challenge. New reported checks worth 0 points: poppy-discovery and robots-edge-consistency. The registry has 137 checks.

OpenAPI specs linked from Link headers, the API catalog and the homepage are tried before standard paths, and YAML specs are parsed. Server Cards are found where the final SEP-2127 puts them.

Every check result says whether its evidence is live, content, presence or declared, and scorecards report pillars.runtimeMeasured, a usability score built only from live and content evidence that does not change the overall score.

MCP 2026-07-28 deprecates Logging, Sampling and Roots (SEP-2577), so mcp-logging-notifications, mcp-sampling-support and mcp-roots-support no longer earn points.

glintbase.dev/api/mcp and scan.glintbase.dev/api/mcp answer server/discover (2026-07-28) and still serve legacy initialize clients. Each publishes a Server Card at /api/mcp/server-card, listed in /.well-known/ai-catalog.json. Packages @glintbase/scanner-core, @glintbase/mcp and @glintbase/cli move to 3.5.0, on MCP TypeScript SDK 1.32.1.

Passes on an active server whose namespace is the site's domain (com.stripe/…, which the registry grants only after domain verification), whose remote endpoint is hosted on the domain, or whose namespace is the brand's GitHub owner. Third-party servers about a brand do not count.

Passes on skills published from the site's own domain or from the brand's GitHub owner. If either registry does not answer in time, the check is skipped rather than failed, so an outage never costs a site points.

Scorecard pages chart every completed scan of a company, grouped by score version. Lines never join scores from different versions.

The scanner homepage shows only real, current-version scans (example scores were removed), and the surface inventory re-checks every redirect hop against the private-network policy.

MCP checks now come from a live handshake, tools/list, real tool schemas and readOnlyHint / destructiveHint annotations, declared capabilities, and an unknown-tool error canary. The scanner only calls tools the server itself marks read-only.

API usability checks read the published spec (operationIds, request and error schemas, pagination, rate-limit headers, problem+json, OpenAPI 3.1) instead of passing because a spec file exists.

Every scan reports how many checks it measured and how much of each layer it could verify. Layers resting on less than half of their scored checks are flagged as low evidence.

@glintbase/scanner-core packages the ARS 3.2 engine. @glintbase/cli 3.3.0 scores with it (audit, ci, check, fix and its MCP tools), adds evidence coverage to its output and a new `glintbase mcp lint <url>` command, and refuses private and localhost URLs; local folder audits (`glintbase audit .`) are unchanged.

Legacy SSE transport and auth.md stage checks are now bonus-only, a duplicate error-signalling check was removed, and letter grades and colour bands share one set of boundaries.

Benchmark datasets, reproduction runners, figures, and 30 full autonomous agent protocol logs (both machine-readable transcript.jsonl and formatted step-by-step transcript_readable.md). Correction (2026-10-12): the original release did not redact all credentials. Modal tokens appeared in 16 transcript files and one transcript contained an unrelated private repository listing; both were redacted on 2026-10-12. Redaction does not remove the values from git history; key rotation and a history purge are being handled separately.

Defined the Integration Tax metric: a weighted ratio of median tokens, steps and wall time to the cohort minimum, times a pass-rate penalty (published: Modal 1.03×, E2B 1.17×, boat.dev 1.21×, Vercel 1.37×, Daytona 1.43×). Correction (2026-10-12): with corrected pass rates the published penalty rule gives E2B 1.17×, Vercel 1.37×, Modal 1.37×, Daytona 1.43× and boat.dev 1.61×. The inputs were self-reported by the agent and the prompts differed by provider, so the ranking is indicative, not a controlled comparison.

Cataloged runtime traps for autonomous agents, including workspace preview proxy auth walls, synchronous vs. background server detaches, and generator return types on sandbox lifecycle listings. Correction (2026-10-12): none of these three traps (including the Daytona HTTP 401 proxy wall) is captured in the published transcripts, and neither are the boat.dev HTTP 402 payment gate, the Modal malformed-token error or the Vercel human stop. The only captured trap is boat.dev's PEP 668 pip error.

Shipped post-trial verification routines that list each provider's sandboxes after every run. Correction (2026-10-12): the original verifier deleted any leftover sandboxes and still passed, so it could not confirm zero leaks; Daytona Trial 4 had one leftover sandbox that was silently removed. Leak status is now reported as not verifiable, and all 30 verifiers have been fixed to fail when sandboxes remain.

The Sandboxes Edition 1 results were corrected after publication: pass rate 28/30 (93.3%), not 30/30, because Modal and boat.dev ran no tests in Trial 6; human stops 0 for every provider; front-door ARS Modal 36, E2B 75, boat.dev 82, Vercel 69, Daytona 48. Full list with evidence: https://glintbase.dev/research/sandboxes-edition-1/reports/ERRATA.md and https://github.com/Glintbase/research/blob/main/benchmarks/sandboxes-edition-1/ERRATA.md.

Deployed interactive research explorer at /research/sandboxes-edition-1 featuring master leaderboard matrices, provider comparison heatmaps, and downloadable canonical CSV and JSON datasets.

Strict single remote endpoint standard supporting standard JSON-RPC 2.0 and Server-Sent Events (SSE) with sliding-window IP rate limiting (60 req/min), permanent HTTP 308 alias redirection, and optional Bearer auth.

Simulation tools emit live inline Journey Tree SVG image blocks (`type: 'image'`) natively inside Claude and Cursor chats, accompanied by a zlib-deflated base64url state hash (`#data=`) linking directly to the web visual replay cockpit.

Zero-configuration tunnel spawning a secure, public HTTPS tunnel for local MCP servers in milliseconds, enabling remote team agents and container workers to interact with offline environments.

Codified 119 discrete checks across 4 operational layers (Discovery, Access, Usability, Payments). Replaced rigid denominators with archetype-based baselines (85 for devtools/docs, 100 for ecommerce) and unpenalized bonus upside.

Dual-engine multi-turn execution harness with persona emulators for claude-code (200k), cursor (128k), and perplexity (zero-JS). Measures Dollar Tax burn ($3/M in, $15/M out) and Schema Friction Index.

AST-driven auto-remediation engine synthesizing llms.txt, auth.md, robots.txt, and Anti-SPA 404 canaries with 1-click Git PR branching (`--branch`) and zero-drift GitHub Actions quality gates.

Integrated @e2b/code-interpreter to spin up ephemeral, secure Linux sandboxes that execute code examples discovered in docs to detect broken parameters and 500 errors.

In-memory ArsSandbox automatically mounts suggested fixes to simulate agent re-runs, empirically calculating exact token savings and latency reductions prior to disk writes.

Powered by Three.js and WebGL, visualizing inter-page cross-references, API endpoint dependencies, and orphaned documentation clusters.

Measures ambiguity in tool parameters, type mismatches between OpenAPI descriptions and schemas, and unit omissions that trigger agent hallucination loops.

Exposes structured platform state, page navigation, and callable tools directly via window.modelContext for visiting AI browser agents.

Active HTTP probes send randomized canary requests to detect SPAs that return 200 OK HTML shells for missing routes—the #1 cause of agent context pollution.

Automated server-side generation of high-resolution summary cards showing score band, layer breakdown, and verification badges.

Ranks hundreds of public software products by Agent Readiness Score, driving transparency in machine operability.

Extracted structured content from __NEXT_DATA__, RSC flight chunks, and JSON-LD from JavaScript-heavy documentation shells.

Simulated whether language models could complete multi-step setup tasks using only documentation text.