Crawl & Logs Field Note Updated 2026-11-16 3 min read HTTP/3 · Brotli · Googlebot Crawl Speed

Brotli Compression, HTTP/3 & TLS Session Resumption for Faster Crawling

Smaller compressed HTML transfers and zero-round-trip TLS handshakes raise Googlebot fetch concurrency while cutting TTFB across global regions.

Dmitry K., Technical SEO Lead · SEO Space

Indexation benchmark targets

Compression
Brotli static level 11 (br)
Transport
HTTP/3 QUIC + HTTP/2 h2

Why Payload Size & Handshake Latency Gate Crawler Throughput

Googlebot schedules crawl rate dynamically around average server response time and error rates. When each HTML response carries heavy unminified JSON-LD or slow TLS handshakes, crawl queues throttle automatically.

Pre-compressing static assets with Brotli-11 reduces HTML and SVG sizes by 18–26% compared to gzip, letting crawlers download more pages per network slot.

gzip off;
brotli on;
brotli_static on;
brotli_comp_level 6;
brotli_types text/html text/plain text/css application/javascript application/json image/svg+xml;

Deploying 103 Early Hints Without Starving Origin Threads

Sending HTTP `103 Early Hints` for hero fonts and LCP images allows browsers to warm connections before dynamic edge SSR or database assembly completes.

Production HTTP stack benchmarks for high-volume domains.

Layer Target Configuration SEO / CWV Impact
HTML Encoding Static Brotli (.br) + gzip fallback 18–26% smaller crawl & mobile payload
TLS Negotiation TLS 1.3 + OCSP Stapling Sub-40ms handshake overhead
Asset Discovery 103 Early Hints for WOFF2/LCP 200–400ms LCP reduction on mobile
  • Verify OCSP stapling and TLS session tickets are active across all edge POPs to save 60–120ms per fresh session.
  • Avoid dynamically compressing static files at Brotli level 11 on request CPU—pre-build `.br` files at CI time.
  • Keep `Vary: Accept-Encoding` explicit so CDNs never serve Brotli bytes to legacy gzip-only clients.

Need this indexation setup audited on your domain?

Send your website URL. We check raw HTML parity, log crawl hits, canonical discipline, and sitemap architecture manually within 24 hours.