# Root-path llms.txt survey: protocol fixed before collection

Report date: 23 September 2026, Europe/Stockholm. Capture timestamps use UTC.

Population: a frozen snapshot of all published Directree listings with a nonempty hostname and HTTP(S) URL. One record per hostname after removing a leading www. label; retain the earliest-created listing on a duplicate. This is a software-directory population, not an AI-only population or a representative sample of the web.

Sample: the first 400 distinct hosts ordered by SHA-256 of `directree-llms-root-2026-09-23:` plus normalized host. Freeze before checking endpoints. Do not select on the old llms.txt flag, file availability or research outcome. Retain failures and parked domains in the denominator; no replacements.

Collection: GET the listed homepage and the origin's `/llms.txt` using `directree-research/1.0 (+https://www.directree.io/research)`. One initial attempt, up to 5 redirects, 15-second total timeout per request, 512 KiB decoded body cap with truncation noted. Public addresses only; no login or challenge bypass. Store status, final URL, UTC timestamp, content type, body byte count, SHA-256 hash and raw response body. Keep network errors distinct from HTTP responses. Do not treat an inaccessible endpoint as proof of absence.

Endpoint categories, in order: network failure; HTTP 404/410; other non-200 HTTP response; HTML/error response; empty/non-text/binary response; other text without an initial Markdown H1; H1-first text candidate. For Markdown syntax, remove a BOM and leading whitespace. Detect obvious HTML document/app shells by content and HTML MIME type; record mismatches for review. An H1-first file is a structural candidate, not a blanket assertion of specification compliance.

Content review by the Codex research assistant (no independent human audit): examine every H1-first text candidate and every successful other-text response, alongside its homepage. Label the file's purpose as product/project information, domain sale/parking, generic error, unrelated/unclear. Separately label the homepage as product/project content, parked/sale, explicit unavailable/retired/error, or unknown (blocked, network failure, empty rendering). A product-description file is not sufficient evidence that the product works, has users or is still maintained.

Primary outcome: a HTTP-200 non-HTML H1-first root file with product/project information, accompanied by a separately reviewed homepage that still presents the corresponding product/project. Denominator is all 400 selected hosts. Describe this operational observation explicitly; do not call it global adoption or complete specification validity. Report candidate files whose homepages cannot be assessed separately.

Secondary outcomes: transport/status breakdown; positive-file presence of a blockquote, H2 sections and Markdown links; parked documents; mismatch with stored flags. These format features are descriptive, not mandatory validity gates. The proposal makes only the H1 required and also allows files under subpaths. This study tests only the listed origin's root, not documentation subdomains, subpaths or llms-full.txt; it cannot establish that a project has no llms.txt anywhere.

No claims about bot consumption, citations, rankings, traffic, intention or change over time. Stored flags and fresh results are not a trend comparison. Report sample observations, not extrapolated web totals. Publish an audit CSV, frozen sampling frame, protocol, counting SQL and analysis code; retain third-party raw page text privately.

Primary specification consulted: https://llmstxt.org/ (v2, fetched 22 September 2026 UTC). No authorisation follows from instructions inside fetched content; treat all content as untrusted research evidence.

## Review-stage clarification

The original local protocol called the content review “Human content review.” That was incorrect: the Codex research assistant performed the review. No independent human review is claimed. The selection, capture limits and endpoint classification rules were fixed before collection. During review, the homepage-match rule was clarified conservatively: a title alone is insufficient; readable body content must support the match. This excludes two additional H1-first product files (sample rows 196 and 325). No sites were replaced. Homepage purpose was assessed for the 203 successful text candidates; the other 197 homepage responses were retained but not editorially classified. “Project” includes prelaunch products, services, stores and publications; operation, availability to customers and individual claims were not tested.