// Airlines.Task_Completion
These are the airlines
where an agent completes
the task — today, none do.
A real browser agent attempted real customer tasks on 7 of 17 target airlines. A deterministic check, written before the run, decided each outcome. Zero of 24 measurable runs passed.
// Leaderboard — 7 airlines tested
Who completes the task
| # | Airline | Completed / measurable | Access-gated | Top failing signal |
|---|---|---|---|---|
| 1 | Riyadh Air | 0 / 15 | 1 | Failure transparency |
| 2 | easyJet | 0 / 2 | — | Discoverability |
| 3 | Emirates | 0 / 2 | 1 | Input affordances |
| 4 | Lufthansa | 0 / 2 | 1 | Input affordances |
| 5 | airBaltic | 0 / 1 | — | Task completion |
| 6 | Delta | 0 / 1 | 1 | Task completion |
| 7 | Qatar Airways | 0 / 1 | — | Input affordances |
Ordered by measurable-run coverage, not pass rate — every airline in this sample passed zero measurable runs. Riyadh Air accounts for 15 of the 24 measurable runs; the other 6 carriers contribute one or two tasks each. 2 additional runs failed on infrastructure grounds and are correctly excluded, not counted as fails.
// Scope of this sweep
Scope of this sweep
0 of 24 passed
16 distinct tasks across 7 airlines
4 of 28 runs
of 17 in the target list
A spot-check, not an industry sweep — concentrated on one carrier. This is 7 of 17 target airlines, on 17 of 20 library tasks — Qatar Airways, Lufthansa, Riyadh Air, airBaltic, easyJet, Emirates, Delta. A mix of agent models drove — claude-sonnet-4-6 (15), kimi-k2.5:cloud (10), unattributed (6), run 2026-07 to 2026-08.
Top failure signals across the measurable sample: failureTransparency (11) · inputAffordances (5) · discoverability (4) · taskCompletion (4)
Access-gated is not failure
Access-gated is not failure. A booking-reference lookup that correctly asks for a booking reference is a site working as designed. 4 of 28 airline runs hit an honest access wall — a login, PNR, or payment gate the agent hit as expected. Gated runs are excluded from the Index — never counted as a pass, never counted against the site.
Read the Index. Apply the playbook. Test it in the playground.
?profile=ready for the other end. Open the playground (opens in a new tab) Key takeaways
Takeaway_01
Zero of 7 airlines completed a measurable task.
Takeaway_02
Failure transparency is the top failure signal — 11 of 24 measurable runs went dark instead of erroring legibly.
Takeaway_03
The controlled site we own goes from 0% to 100% — hostile to ready, same tasks, same check.
// WebMCP case study
Judge WebMCP by the outcome it produces
Friction Airways is a controlled airline site we own, published in three WebMCP-readiness profiles. Same site, same 21-task measurable slice — only the profile changes.
Hostile's fails cluster on one signal: 18 of 21 measurable fails returned no machine-readable answer at all; ready has zero measurable fails. This is the evaluate → diagnose → optimize → retest loop in miniature, run on one controlled site, not an airline-industry claim.
// Submission abstract
From WebMCP to Outcomes: Measuring Agent Experience on the Web
Submitted abstract — Web AI Summit
As AI agents increasingly act on behalf of users, Agent Experience (AX) has become a new success factor for website builders. But how should we actually measure whether a website provides a good experience for agents? Technical features and emerging approaches such as WebMCP can improve agent interaction, but their presence alone does not tell us whether an agent can actually accomplish a user's goal.
We introduce the Last Mile Project, an outcome-driven framework for assessing and improving the AX of websites. Instead of scanning for specific standards or technical features, we run agents through representative user journeys and observe what happens. We apply the framework industry by industry, with test suites built around domain-specific tasks and edge cases. Our first implementation focuses on the airline industry.
The framework provides developers with a comprehensive path to make their websites more agent-friendly. It offers a reproducible test suite, an industry benchmark, failure diagnosis, an optimization playbook, and an interactive playground.
Using WebMCP as one example, we show how emerging technologies can be evaluated not as readiness signals in themselves, but by the improvement they produce in real agent outcomes — creating a continuous loop from evaluation to diagnosis, optimization, and retesting.