servers VS llms-txt

Most agents don't speak raw HTTP — they speak MCP. So the real distribution surface is a tiny MCP server that exposes each endpoint as a tool and does the paying under the hood. The agent calls web_search(query); the server hits the paid endpoint, handles the 402 with its operator's wallet, returns JSON. One line to install:

The Model Context Protocol (MCP) is an open standard for connecting AI hosts (Claude, ChatGPT, Cursor, VS Code, and others) to external context and actions through a structured protocol instead of ad-hoc plugins.

Model Context Protocol specification

That changed when I built mpesa-mcp — a Model Context Protocol server that wraps the Daraja API and makes it directly available to Claude, GPT-4, and Gemini.

If you haven't run into it yet, MCP (Model Context Protocol) is an open standard for exposing tools to AI agents. two-go ships an MCP server that runs over stdio with no dependencies, no URL, no account, and no API key. It's all local.

MCP server surface. The CLI is already agent-friendly, so I'm double minded about whether a Model Context Protocol server would make integration smoother. I'm personally not a fan of MCP and prefer skills and CLIs.

As noted in the Model Context Protocol (MCP) Introduction, this protocol allows Claude to interact with systems like GitHub and local databases through a standardized interface.

Keep the project's living context: current task state, decisions, conventions, known footguns as plain Markdown committed to the repo, and expose it to the agent over MCP.

The MCP ecosystem doubles every few months. Bookmark the official MCP servers repository for the canonical list.

I was building a module that reads every product in a Magento catalog to generate an llms.txt file. On a store with 70,000 products, the standard Magento collection was fast at page 1 and unusably slow around page 40. The code did not change — only the page number.

llms.txt is a proposed convention: a single Markdown file at your domain root that tells an assistant, in plain prose, what lives here and where the canonical version is. Think of it as a robots.txt written for comprehension instead of permission.

Putting it in .well-known/ was immediately raised as an issue from the beginning; it’s issue #2 in fact:

https://github.com/AnswerDotAI/llms-txt/issues/2

It’s been completely ignored ever since.

The llms.txt file

4. No llms.txt. This newer standard gives AI crawlers a structured summary of your site. Most sites don't have one — add it in 10 minutes:

In September 2024, Jeremy Howard of Answer.AI proposed a standard for a /llms.txt file at the root of any docs site, structured to be cheap for a language model to ingest. Within months, Mintlify rolled out automatic support across every site they host. Mintlify built the llms-full.txt variant with Anthropic (a single concatenated dump of all docs), because parsing fragmented HTML was killing Anthropic's indexing pipelines. Cloudflare, Vercel, and a thousand others followed in months.

llms.txt is an emerging convention: a Markdown file at a known location that tells AI agents how to use your content. I used it to build sf-llms-context — a small, opinionated Salesforce knowledge base aimed squarely at coding agents.

Optional llms.txt: Some teams publish llms.txt to point systems at preferred docs. Optional for most architectures; Lighthouse's Agentic Browsing category includes a discoverability check if you publish one (see the scoring post linked above).

The page is two things stacked. The surface artifact is an llms.txt file — a young convention that mirrors robots.txt but addresses language models instead of search crawlers. The substance, written into that file, is a four-step routing guide off the human-facing site and onto bulk endpoints that the project would rather see crawlers use. The post resurfaced on Hacker News this week with 750 upvotes and 413 comments, three months after publication.