---
title: "Docs for agents"
description: "Publish discoverable documentation, Agent Skills, browser-agent search, and alternate Markdown for AI tools."
---

> Documentation Index
> Fetch the complete documentation index at: https://nimbus-docs.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Docs for agents

Nimbus serves documentation through endpoints that AI tools can read without parsing the HTML site.

## Release-matched guide

Every framework package includes this guide at `node_modules/@cloudflare/nimbus-docs/dist/docs-for-agents.md`. For a preview release, use that installed copy: it matches the package you are testing even when the public website describes an older stable release. The starter's `AGENT.md` points coding agents to it.

## What ships

- **Alternate Markdown/MDX versions for every discoverable page** — `/<slug>/index.md` is clean Markdown with components converted to text; `/<slug>/index.mdx` keeps imports, JSX and directives, and the `.md` links to it. API reference pages have only the `.md`, and hidden versions have neither.
- **A site `llms.txt` index** — names top-level pages and section indexes of the current docs. Each section index links its discoverable pages to their alternate Markdown versions. Older docs versions aren't listed; each has its own index at `/<version>/llms.txt`.
- **`llms-full.txt`** — the current site's pages in one Markdown document, for tools that fetch everything in one request. Over 25 MiB, it becomes an index of one file per section, so no file passes the limit. See [Add llms-full.txt](/ai/publish-markdown#3-add-llms-fulltxt).
- **Per-section indexes at `/<section>/llms.txt`** — each top-level group and collection (e.g. a `blog` or a docs version) gets its own index, and so does each folder below it (`/reference/functions/llms.txt`), linked from its parent.
- **Version labels** — on versioned sites, every alternate Markdown version's frontmatter carries a `version:` key (resolved from your `versions` manifest) so agents can pin a version. Unversioned sites emit nothing.
- **JSON-LD in every page head** — structured data so search and agent tools recognize what they're looking at.
- **`/robots.txt` and a sitemap** — wired automatically when `site` is set.
- **A remote MCP server**, when `agents: { mcp: true }` is set on a site with a server adapter — `/mcp` serves `list_pages` and `get_page` over the published Markdown pages, with a Server Card at `/mcp/server-card`. See [Remote MCP server](/ai/remote-mcp).

```text title="/llms.txt"
# Nimbus

A way to build documentation sites on top of Astro.

## Pages
- [Quickstart](https://nimbus-docs.com/quickstart/index.md)
- [Why Nimbus](https://nimbus-docs.com/why-nimbus/index.md)
```

## Agent discovery

Every site serves `/.well-known/ard.json` and `/.well-known/ai-catalog.json`, with the same content. They list `/llms.txt`, the homepage Markdown at `/index.md`, and the [remote MCP server](/ai/remote-mcp)'s card when the site runs one. A site on a domain another site owns can turn them off ([Shared domains](#shared-domains)).

The homepage advertises these files and each visible API reference in HTTP `Link` headers. Request-rendered homepages send them directly. Static builds write `_headers` rules, which Cloudflare and some other hosts read; on other hosts, set the same headers in the host's config.

Your own `_headers` wins. If your rule already sets a header for a path, exactly or through a wildcard such as `/*`, Nimbus leaves that header out, because hosts join repeated values and browsers refuse `Access-Control-Allow-Origin: *, *`.

`Link` is the exception, since joined `Link` values are valid. If your rule covers only part of a generated wildcard, the build warns and names the paths to cover.

`/index.md` is the root content entry when there is one, otherwise the `llms.txt` content. A root entry with `noindex: true` is served but left out of discovery.

If your site already has a file or route at either `.well-known` path, the build fails so you can move your entries first, unless discovery is off.

Discovery files belong at the origin root. With a `base`, the Cloudflare adapter puts them there for you; on other hosts, deploy the emitted `.well-known` directory at the root alongside the site. Static output with the Cloudflare adapter and a `base` needs `wrangler deploy --assets ./dist/client`.

For a static host serving a site under `/docs`, the public layout is:

```text
host-root/
├── .well-known/        # emitted discovery and Skills files
├── docs/               # the site's HTML, Markdown, indexes, and Pagefind files
└── _headers            # when the host supports this format
```

Use your host's mounting or deployment configuration to produce that layout. Serve discovery JSON as `application/json` with `Access-Control-Allow-Origin: *`, and apply the same CORS header to Skills artifacts. Homepage Markdown uses `text/markdown; charset=utf-8`; Skills archives use `application/gzip`. Verify the actual deployed responses, rather than only checking the emitted file:

```sh
curl -I https://docs.example.com/.well-known/ard.json
curl -I https://docs.example.com/docs/
curl -I https://docs.example.com/docs/index.md
```

### Shared domains

When another site owns the domain, such as docs at `example.com/docs` beside a main site at `example.com`, the root `.well-known` files belong to that site. Turn Nimbus's off:

```ts title="astro.config.ts"
const nimbusConfig = defineNimbusConfig({
  // …
  agents: { discovery: false },
});
```

Nimbus then publishes no `ard.json`, `ai-catalog.json` or `api-catalog`, and no homepage `Link` or `_headers` rules for them. Your own files in `public/.well-known/` are kept; under a base they're served there, such as `/docs/.well-known/ard.json`.

After switching, purge the cached homepage, whose `Link` header still lists the old files. On hosts that keep files from earlier deploys, such as an S3 sync without `--delete`, remove them too.

Everything else stays on, including `llms.txt`, Markdown versions, Markdown negotiation and the remote MCP server, which agents still find through `llms.txt`.

Agent Skills still publish. With the Cloudflare adapter, or without a base, they land at the origin root's `/.well-known/agent-skills/`, so leave out the `skills/` folder unless this site owns that path. A site serving its own `api-catalog` sets its `Content-Type` in its own `_headers`, since the file has no extension.

### Content Signals

The starter's `src/pages/robots.txt.ts` declares `Content-Signal: ai-train=yes, search=yes, ai-input=yes`. Change the values to match your policy; if you set one to `no`, add the policy text from [contentsignals.org](https://contentsignals.org/) above it. Existing sites keep their `robots.txt`; `nimbus-docs migrate` describes the change. Content Signals and `robots.txt` are requests to crawlers, not access control. Private docs need authentication.

## API catalog and published specs

Sites with API collections serve an [RFC 9727](https://www.rfc-editor.org/rfc/rfc9727) catalog at `/.well-known/api-catalog`, unless `agents.discovery` is `false`. It has one entry per visible collection and version, linking the reference page, its Markdown and the published spec. A version selected by query is listed at its query URL, such as `/pets/?version=v1`, and has no Markdown link when it isn't the default.

Each spec is published as one self-contained JSON file at its mount path, such as `/pets/openapi.json`, with referenced files embedded under `x-ext`. The discovery manifest lists every visible spec. The homepage `Link` header lists only each family's default reference and spec, plus the catalog.

Set `publishSpec: false` on a collection, or on one version, to keep its spec unpublished; the catalog still links its docs. A spec that can't be bundled is skipped with a warning. Hidden versions publish their file but aren't listed anywhere.

The catalog file has no extension, so a host that ignores `_headers` needs its own rule for its media type and its `Link` header on HEAD.

## Markdown negotiation

On server output, a request-rendered page answers `Accept: text/markdown` with the same Markdown as its `index.md`, at the same URL. Both forms send `Vary: Accept`. Markdown wins only when it outranks every HTML and wildcard type in the header, so a browser's `*/*` and `text/plain` still get HTML. Once a collection renders on request, the homepage negotiates too.

Cloudflare reads the published Markdown through its assets binding. Other adapters fetch it from the origin in Nimbus's `site`, never from the request's Host, so deploy the static Markdown with the server and set `site` to the deployment's origin, previews included.

API collections rendered on request need Cloudflare. On other adapters, render API collections at build and prose collections on request.

For example, a Node deployment uses the official [Astro Node adapter](https://docs.astro.build/en/guides/integrations-guide/node/):

```ts title="astro.config.ts"
import node from "@astrojs/node";
import { defineConfig } from "astro/config";
import nimbus, { defineConfig as defineNimbusConfig } from "@cloudflare/nimbus-docs";

export default defineConfig({
  output: "server",
  adapter: node({ mode: "standalone" }),
  integrations: [nimbus(defineNimbusConfig({
    site: "https://docs.example.com",
    title: "Example",
    rendering: { collections: { docs: "request" } },
  }))],
});
```

Use the same Nimbus rendering policy with the official Netlify or Vercel adapter, following that adapter's deployment instructions. Static output, discovery assets, Skills, and browser search do not require a server adapter.

```sh
curl -i -H 'Accept: text/markdown, text/html;q=0.9' https://docs.example.com/guide/
curl -I -H 'Accept: text/markdown' https://docs.example.com/guide/
```

Prerendered pages and static sites return HTML: the host serves the file before any middleware runs. For those, Cloudflare's [Markdown for Agents](https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/) negotiates at the platform level and converts the HTML. Assets, API endpoints, and unknown routes pass through unchanged.

## Agent Skills

Add `skills/<name>/SKILL.md` folders at the project root, in the [Agent Skills](https://agentskills.io/specification) format, and Nimbus publishes them at `/.well-known/agent-skills/`. The index follows the discovery RFC v0.2.0 and gives each entry a SHA-256 digest.

A single-file skill is published as its `SKILL.md`; a multi-file skill as a `.tar.gz` with `SKILL.md` at the root. The discovery manifest lists the index, unless `agents.discovery` is `false`.

A skill whose frontmatter breaks the spec is skipped with a build warning that names the folder and the rule. Dotfiles are left out of archives and symlinks are skipped. Without a `skills/` folder nothing is published.

For example, create `skills/getting-started/SKILL.md`:

```md
---
name: getting-started
description: Help a user install and configure the example project.
---

# Getting started

Read the installation guide before changing project configuration.
```

Run the production build, then inspect `/.well-known/agent-skills/index.json` and fetch each advertised artifact. Removing a skill folder removes its publication on the next build. Existing public files or routes at the generated discovery and Skills paths cause a build error with move/remove guidance.

## Browser-agent search

The starter registers a `search_documentation` tool with `document.modelContext` ([WebMCP](https://webmachinelearning.github.io/webmcp/), an experimental browser API). It searches the site's Pagefind index and returns section titles, anchored URLs, and plain-text excerpts, with the same visibility rules as site search. Nothing loads until an agent calls the tool, and browsers without the API do nothing.

Sites with `search: false` or a custom provider register no tool. The index comes from a production build, so in development the tool returns `index_unavailable`. Pagefind 1.5.2 or newer is required. Existing sites copy `WebMCP.astro` from the starter and render it inside `<head>` in `BaseLayout.astro`.

The browser tool is independent of the deployment provider. It uses the experimental `Document.modelContext` API, not `Navigator.modelContext`, and is not a remote MCP server or a browser polyfill. A browser without that API keeps ordinary search and registers no agent tool.

An agent invokes the registered tool with JSON such as:

```json
{ "query": "installation", "limit": 5 }
```

| Field | Rule |
|---|---|
| `query` | Required. Non-blank, at most 500 characters. |
| `limit` | An integer from 1 to 20. Default 5. |
| `version` | Versioned sites only: one of the visible versions. Without it, deprecated versions are left out. |

Drafts, hidden versions and `noindex` pages aren't searched, unless a page sets `searchable: true`. Each result has a `title`, `heading`, anchored `url` and a plain-text `excerpt` of at most 1,000 characters.

### Search failures and recovery

| Code | Meaning and action |
|---|---|
| `invalid_input` | Correct the query, limit, or visible version before retrying. |
| `index_unavailable` | Search assets could not load, or Pagefind is incompatible. Check the connection and reload. Site owners should verify that a production build's complete Pagefind output is deployed; upgrade to Pagefind 1.5.2 or newer when instructed. |
| `search_failed` | Search execution, result loading, or the lazy search-code import failed. Retry an execution failure after assets recover; if code still cannot load, reload the page. |



## How pages are indexed

Every page collection is indexed: those made with `docsCollection()`, `componentsCollection()` or a `withNimbusMarkdown()` loader, and API collections. Nimbus reads each page's `title` and `description` when present. Plain data collections aren't indexed, and hidden versions are left out.

Alternate Markdown/MDX versions and `llms.txt` indexes use public agent-endpoint assets. Request-time authorization cannot make private material safe here. Keep private material outside public content collections. Nimbus omits excluded entries and fails the build when it cannot determine whether content is public.

To keep a page **out** of the `llms.txt` indexes, set `draft: true` (excluded entirely) or `noindex: true`. A page that should never be agent-readable belongs outside the content collection altogether.

### Custom content loaders

Nimbus's collection factories prepare Markdown automatically. If a custom loader retains Markdown bodies for alternate Markdown versions or `llms-full.txt`, wrap it with `withNimbusMarkdown()` from `@cloudflare/nimbus-docs/content`. See [Other collections](/writing/collections#add-a-collection) for a complete example.

## Route setup

New projects include these routes. To add or update them in an existing project, follow [Markdown and llms.txt endpoints](/ai/publish-markdown).

## AGENT.md

The scaffolder writes an `AGENT.md` at the project root — the same context a coding agent reads when working in your repo. It's a real file you maintain; keep it in sync with your project's conventions.

## Why it matters

The cost of being a dead end on the agent web is real and growing. Because these files and metadata are defaults, a Nimbus site is legible to agents the day it ships — see [Metadata and SEO](/publish/metadata-and-seo) for the human-search side.

Source: https://nimbus-docs.com/ai/docs-for-agents/index.mdx