Skip to content
← AI in practice / October–November 2026

Agent-friendly websites

llms.txt vs APIs vs MCP: An Agent-Ready Website Guide

An implementation guide to agent-friendly websites, with a short llms.txt template, API contract, failure tests and a practical decision framework.

AI Superpower · Published · 8 min read

AI Superpower guide: A small, dependable interface. Illustrated example, not a measured result.
Original explanatory graphic. The worked scenario is fictional; it is not a client result or performance claim.
In this guide
  1. The useful starting point
  2. What to build—and what each layer does
  3. Choose the lightest useful interface
  4. Worked example: a service lookup contract
  5. Test the contract beyond the happy path
  6. The six-step implementation
  7. A short llms.txt you can adapt
  8. If you need an API, start with a read
  9. A better audit prompt
  10. Test what happened, not what the assistant says happened
  11. Avoid the shortcuts that create new problems
  12. Questions worth answering
  13. Optional hosting for a working prototype
  14. Put one improvement to work
  15. Sources and editorial notes
  16. Continue with the right practical resource
  17. Sources and further reading

The useful starting point

The Route Doctor newsletter example describes a short llms.txt file and two public endpoints, with a successful Claude test reported by its author. That is an instructive pattern, not evidence that every agent will discover the file or that the approach improves rankings. The linked project and checker could not be independently verified during this review, so this guide does not repeat their commands as tested installation instructions.

There is a useful idea inside the excitement about agent-ready websites: stop making software infer every capability from a visually complex page. Give it a short map, a clear contract and a result it can check. The mistake is treating those ingredients as a universal certification or a shortcut to search rankings.

The practical answer

The Route Doctor newsletter example describes a short llms.txt file and two public endpoints, with a successful Claude test reported by its author. That is an instructive pattern, not evidence that every agent will discover the file or that the approach improves rankings. The linked project and checker could not be independently verified during this review, so this guide does not repeat their commands as tested installation instructions.

What to build—and what each layer does

LayerUseful jobWhat it does not establish
Readable, accessible pagesExplain the offer and make the next action understandable.That an agent will interpret every detail correctly.
A short llms.txtProvide a concise map for clients that use it.Permission, universal adoption or a search ranking boost.
Documented read-only dataReturn structured facts for a defined task.That those facts are fresh or safe without maintenance.
Optional MCP integrationExpose capabilities to compatible clients.Automatic discovery, authorization or reliable outcomes.

Choose the layer that solves the observed problem. llms.txt proposal and format describes the file convention; OpenAPI specification describes HTTP API contracts; Model Context Protocol architecture explains MCP’s client/server architecture. These are different responsibilities.

Choose the lightest useful interface

If the task is reading information, a well-structured page may suffice. If the task is retrieving fresh, structured data, a documented read-only API can reduce ambiguity. OpenAPI describes the HTTP contract; it does not run the service. MCP connects compatible clients to exposed capabilities and introduces its own implementation and maintenance work. Add it when a real client workflow benefits. None of these automatically grants the agent permissions.

Worked example: a service lookup contract

Use a fixed public service catalog for the first prototype. GET /api/v1/services returns identifiers, names, scope, pricing basis, source URLs and a checked date. A query for an unsupported identifier returns an explicit error, not an empty success message or invented match. Unknown routes return 404; unsupported methods return 405. Keep the example free of customer data and administrative actions.

Test the contract beyond the happy path

Validate both status and content type. A 200 response containing your application’s HTML shell is not a valid JSON API response. Check malformed input, no matching result, rate limiting, stale data and upstream failure. If a request is retried, ensure it does not create a duplicate action. For real deployments, add appropriate authorization, abuse controls, operational logs and ownership; a documentation file is not a security boundary.

The six-step implementation

Six steps: Define the task: Choose one safe useful action.; Write the contract: Inputs, outputs and errors.; Implement the read: Keep private data out.; Publish the map: Link real docs and endpoints.; Test failure paths: Verify status and response.; Observe real use: Maintain what clients need.
Use the sequence as a working checklist. Each step produces something you can inspect.
  1. Define the task. Choose one safe useful action.
  2. Write the contract. Inputs, outputs and errors.
  3. Implement the read. Keep private data out.
  4. Publish the map. Link real docs and endpoints.
  5. Test failure paths. Verify status and response.
  6. Observe real use. Maintain what clients need.

A short llms.txt you can adapt

This is an illustrative template. Replace every example.com URL with an existing public page on your own domain. Check the content, not just the link. If your site has no public API, leave API instructions out. Do not publish placeholder endpoints, private URLs, access tokens or customer information.

# Service Finder Demo

> Discover a documented read-only service, call it with valid input, and explain an error without inventing a result.

## Useful pages
- [Main information](https://example.com/docs/): The current offering and relevant details.
- [Conditions and process](https://example.com/api-reference/): Eligibility, limitations and how the process works.
- [Next step](https://example.com/status/): The official route for a person to continue.

## Boundaries
This example does not support automated purchases or bookings.
Use the linked pages for current facts; ask when information is missing.

Serve the file at your site root when your hosting setup supports it. In a typical static build, that means placing it in the directory copied to the public root. Verify the deployed URL: a local file alone is not a published file. If your platform already generates one, inspect and maintain that version instead of creating conflicting instructions.

If you need an API, start with a read

The following is a proposed contract, not an endpoint installed on this site. Choose fields that answer the task, document their types and meaning, and return only information you intend to make public. Include a source URL and an update signal where useful. A checked date records a review; it does not guarantee the underlying data is current.

GET https://example.com/api/v1/services
Accept: application/json

{
  "example": true,
  "checked_at": "2026-11-11",
  "items": [{
    "id": "intro",
    "name": "Introductory service",
    "pricing_basis": "Request a current quote",
    "source_url": "https://example.com/services/",
    "available_actions": ["read_details"]
  }]
}

Define no-result and error behavior as carefully as success. Keep private data behind proper access controls. For future actions that change records or spend money, design authentication, authorization, explicit confirmation and duplicate-action protection separately. OWASP API Security Project provides a useful security review starting point.

A better audit prompt

Use an assistant with browsing or request capabilities. If it cannot access the site, it should report the limitation; a fluent response is not proof of inspection.

Review this public website: [URL]. Task: Discover a documented read-only service, call it with valid input, and explain an error without inventing a result.

Use browsing or request tools if available; if unavailable, say what you cannot verify. Inspect the relevant public pages and /llms.txt. Record the URLs you actually fetched and the facts you found. Do not infer success from a 200 response alone. Separate observed facts, missing information and suggested changes. Do not sign in, submit forms, change data, add to a cart or purchase.

Draft a concise llms.txt using only verified public URLs. Include an API example only if a documented public endpoint exists and its read operation is appropriate to test. Do not invent endpoints. Rank the three most useful fixes by customer impact and effort. Explain how to retest each one.

Test what happened, not what the assistant says happened

TestEvidence to keepA passing result
Retrieve the mapFinal URL, status, content type and response body.The actual text file appears; no login page or application shell.
Follow its linksThe final destination and the relevant source facts.Useful pages resolve and match the stated purpose.
Try the customer taskExact prompt, tool capability, answer and cited URLs.Required facts and next step are correct.
Ask an unsupported questionThe answer and what the source actually says.The assistant names the limit rather than inventing a capability.
Repeat after a changeSource version, date and all attempted runs.Any improvement is reproducible within the tested setup.

Measure for this site’s use case: Contract-valid responses, factual task completion, error recovery, latency and maintenance burden. Tool adoption and rankings require separate evidence.

Run a baseline before editing. Use fresh sessions and keep the prompt and available tools comparable. Include failed attempts in your denominator. Record whether you explicitly told the agent about llms.txt; that is different from observing spontaneous discovery. A scanner is a diagnostic aid, not a certification.

Avoid the shortcuts that create new problems

  • Do not replace readable pages with machine-only content. People still need to evaluate the offer.
  • Do not treat robots.txt or llms.txt as authentication. Protect private routes with real access controls.
  • Do not tell every crawler it may access everything merely to improve a score.
  • Do not duplicate prices and policies into several files without an update process.
  • Do not claim a successful booking, lead or sale from a generated answer. Verify the actual system result.

For interface implementation, consult Google web.dev: agent-friendly website UX and W3C: labeling form controls. Preserve descriptive controls, meaningful form labels and visible state changes. Agent testing complements human usability and accessibility testing.

Questions worth answering

Will llms.txt improve Google rankings?

No ranking benefit is established by adding this file. Google’s current guidance says it does not use llms.txt for Search visibility. Treat the file as an optional usability aid for clients that consume it; keep ordinary SEO work and task testing separate.

Do I need MCP or Markdown copies of every page?

Not for every project. Start with the smallest interface the task requires. Add Markdown resources or a compatible tool integration when there is a clear consumer and maintenance plan. Check platform support rather than assuming all assistants behave the same way.

Can I use this without coding?

Yes. You can clarify public pages, check links and run the task exercise with your existing website editor and assistant. Publishing a root file depends on the host. A public API requires additional implementation; ask for help if that is outside your skills.

How often should I review it?

Review whenever an offering, policy, route or API contract changes. Assign an owner. Frequently changing prices or stock need stronger freshness controls than a static guide.

Search guidance: Google Search: optimizing for generative AI. Documentation reviewed November 11, 2026. Provider behavior can change; recheck the linked sources when implementing.

Optional hosting for a working prototype

Your existing host may be sufficient. If a custom read-only service needs deployment, you can evaluate Railway through our referral link. Compare current usage charges and operating requirements before choosing it. Referral disclosure: we may receive a benefit for qualifying activity. No paid host is required to complete the planning and local test exercises.

Put one improvement to work

Start with this task: Discover a documented read-only service, call it with valid input, and explain an error without inventing a result. Save the baseline, make one justified improvement and keep the retest evidence. Use the result to choose the next useful change.

Plan a useful tool trial →

Sources and editorial notes

This guide combines primary-source documentation with original implementation advice and a fictional worked example. It is not a report of measured client results, guaranteed traffic or earnings. The examples use reserved example.com URLs and must be adapted and verified before deployment.

Sources and further reading

Sources checked November 11, 2026. Tool features and plan availability can change; review the linked publisher documentation before connecting an account.