AI Search (GEO) Field Note Updated 2026-11-16 3 min read ChatGPT · Claude · Perplexity · llms-full.txt

Optimizing llms-full.txt Token Budgets, Chunk Boundaries & Retrieval Precision

Dumping 20,000 lines of unformatted text into llms-full.txt triggers retrieval truncation in lightweight AI fetches. Structuring deterministic headers, summary tables, and crisp section boundaries maximizes citation accuracy.

Dmitry K., Technical SEO Lead · SEO Space

Indexation benchmark targets

Index file
/llms.txt <= 4 KB curated directory
Section density
Self-contained H2 answer blocks
Fact placement
Canonical pricing + SLA near top

Why Bloated llms-full.txt Files Lose Key Facts to Truncation

Many AI crawler agents impose strict byte and token caps when ingesting plain-text manifest files over HTTP. If critical service scopes, pricing tiers, or flagship guides sit at line 15,000 behind verbose boilerplate, retrieval pipelines truncate before reading your highest-margin pages.

Separating a compact /llms.txt routing manifest from a tightly structured /llms-full.txt—where pricing, core services, and exact methodology facts lead first—ensures accurate ingestion under tight token limits.

  • Place canonical company facts, pricing tiers, geographic scope, and direct URLs in the top 1,500 tokens of llms-full.txt.
  • Format every guide summary with explicit H2 headings, bullet takeaways, and zero UI navigation strings.
  • Auto-generate both manifests from the exact same data build step that produces your HTML pages.

Clean Chunk Template for /llms-full.txt

Structure each block so isolated vector retrieval captures both the claim and its authoritative source URL.

Balance fast routing discovery with deep technical completeness.

Manifest Path Primary Consumer Optimal Size & Focus
/llms.txt Initial crawler triage & agent routing 2–5 KB: high-signal URL index + brief descriptions
/llms-full.txt Deep RAG ingestion & grounded answers Structured Markdown facts, pricing, and technical docs
Page .md / HTML Per-URL deep inspection Semantic headings + inline tables matching HTML 1:1
## Service Tier: Monthly SEO + AEO + GEO Retainer
- Canonical URL: https://seo-space.online/#pricing
- Scope: Technical crawl hygiene, Schema @graph engineering, AI search citations
- Starting Fee: Transparent fixed monthly scope (USD / EUR / GBP)
- Direct Request: https://seo-space.online/#contact

Need this indexation setup audited on your domain?

Send your website URL. We check raw HTML parity, log crawl hits, canonical discipline, and sitemap architecture manually within 24 hours.