Public contract

The standard for bot indexing

One card schema. One search API. Owner-proven records. Consent split between retrieve, train, and act. The engine stays private; this page is the contract. The card format is specified in IETF Internet-Draft draft-mitchell-botcentral-card and in SPEC.md.

Why a standard

Every assistant that fetches the web has been inventing its own path: its own User-Agent, its own submit form, its own allowlist, its own idea of whether retrieve is train. N vendors times M bots times K portals has no upper bound. A million paths. Site owners cannot implement a million paths.

The web already had the pieces — RFC 9309 robots.txt, RFC 8615 well-known, W3C TDM, MCP, OAuth 2.0 client credentials, ETag, OpenAPI. It did not have one owner-proven index that composes them. BotCentral is that index. A listing is permission to be found — not a ranking, not a training grant, not another submit form.

CiteFleet at citefleet.app (or any registered publisher) writes the catalog after origin proof. Bots never publish. They attach MCP or call REST and honor the card.

  • Developers mint a paid API key at /keys. A listing is $10.00 USD per host per year, payable in RLUSD, XRP, XLM, BTC, HBAR, or XDC through CiteFleet; edits inside the year are free, and a listing nobody renews is unverified until it is. /price and GET /v1/price are the public quote. Anonymous catalog reads stay free at 30/min.
  • Assistants attach MCP at POST /mcp or call REST. Config and OpenAPI live at /connect.
  • Bots fetch a site card at GET /v1/site/{domain} and send If-None-Match.
  • Bots watch GET /v1/changes and verify the catalog with GET /v1/snapshot.
  • Each declared URL carries a quality score (0–100): proof, consent, freshness, crawl, role, surface. GET /v1/score?url= inspects one address. Probe is origin readiness, not ranking.
  • Owners do not publish here. A registered publisher product (CiteFleet or another frontend) PUTs after origin proof.
  • Default consent is retrieve yes, train no, act no, TDM reserved. allow_bots is no longer enough.

Honesty rules

  • SPA HTML at /.well-known/botcentral.txt is not a token file.
  • Homepage reachability is not well-known-file.
  • CiteFleet origin is not a verification method. Proven methods are dns-txt and well-known-file.
  • Google-Extended and Applebot-Extended are robots.txt control tokens, not HTTP User-Agents. The agent registry says so.
  • Test listings that never proved ownership are not claimed as proven.
POST /mcp                         (JSON-RPC 2.0 streamable HTTP)
GET  /.well-known/mcp.json
GET  /keys
GET  /connect
GET  /v1/price
POST /v1/jobs
GET  /v1/jobs/{id}
GET  /v1/search?q=&topic=&verified=&consent=&capability=&include=card|urls
GET  /v1/site/{domain}          (ETag / If-None-Match)
GET  /v1/site/{domain}/pages
GET  /v1/score?url=&q=
GET  /v1/agents
GET  /v1/topics
GET  /v1/changes?since=
GET  /v1/snapshot
GET  /v1/openapi.json
GET  /v1/schema.json
POST /v1/probe
PUT  /v1/publisher/sites/{domain}   (issued publisher Bearer)
DELETE /v1/publisher/sites/{domain}
POST /v1/publisher/sites/{domain}/reverify
POST /internal/publish
POST /internal/unpublish
POST /internal/reverify
POST /internal/keys/credit
POST /internal/jobs/settle
GET  /health
GET  /llms.txt
GET  /robots.txt
GET  /sitemaps/sitemap.xml
GET  /.well-known/botcentral-index.json
GET  /.well-known/tdmrep.json
GET  /.well-known/security.txt
API keysMint a key, raise your rate ceilingPublisherList client sitesSchemaCard 1.1OwnersHow a site is listedUseRate limits and polling

ISO 20022 is an optional profile on financial cards. MCP, OpenAPI, and A2A are capabilities, not the catalog format.