HTTP Headers for SEO: The Ones That Actually Affect Rankings

Most HTTP headers are invisible plumbing that nobody should care about. A handful directly affect how Google crawls your site, whether your pages get indexed, and how fast they load — which means they affect rankings.

This is a short list of the headers that actually matter for SEO, what each one does, and how to see what your server is sending right now.

How to See Your Current Headers

Before you change anything, check what you're already sending. In a terminal:

curl -I https://yourdomain.com/some-page

The -I flag requests headers only. You'll get back something like:

HTTP/2 200
content-type: text/html; charset=UTF-8
cache-control: public, max-age=0, must-revalidate
x-robots-tag: index, follow
strict-transport-security: max-age=63072000

If you don't have a terminal handy, paste the URL into our HTTP Header Checker and it'll show you every header the server returned, plus flag missing SEO-critical ones.

Now the ones worth knowing.

X-Robots-Tag — The Most Underrated SEO Header

This is the HTTP-header version of a meta robots tag. The difference: you can apply it to non-HTML files (PDFs, images, JSON feeds) where you can't add a <meta> tag.

Example header:

X-Robots-Tag: noindex, nofollow

When to use it:

• PDFs you don't want indexed. Lead magnets, gated whitepapers. Add X-Robots-Tag: noindex to the response for .pdf files.
• Image URLs. If you're serving originals from a CDN and only want the optimized versions indexed.
• Development or staging subdomains. Set X-Robots-Tag: noindex, nofollow at the server level on staging.yourdomain.com and you don't have to remember to remove meta tags later.
• API endpoints and JSON feeds that somehow end up in Google.

Watch out: if you set X-Robots-Tag: noindex on a URL that's also canonicalized to a different URL, Google's behavior is undefined. Pick one signal and be consistent. For deeper canonical strategy, our canonical tags guide covers the full tradeoff.

Cache-Control — Speed Equals Rankings

Google uses Core Web Vitals as a ranking factor, and caching is one of the cheapest wins for Largest Contentful Paint and Time to First Byte. The Cache-Control header tells browsers and CDNs how long to keep a file before re-requesting it.

Common patterns:

• Static assets (CSS, JS, images with hashed filenames): Cache-Control: public, max-age=31536000, immutable — cache for a year.
• HTML pages: Cache-Control: public, max-age=0, must-revalidate — always re-check, but let CDNs serve if the content hasn't changed.
• API responses: short max-age values, usually 60 to 300 seconds.

If your HTML has Cache-Control: no-store, every page load hits your origin server cold. That's almost always a mistake unless the content is user-specific.

For a full breakdown of how these numbers translate into ranking signals, see our Core Web Vitals guide.

Content-Type — Don't Let Your HTML Get Misinterpreted

Content-Type: text/html; charset=UTF-8

Googlebot uses this to decide how to parse the response. If your server returns HTML with Content-Type: text/plain, Google may skip the page entirely — it won't render the DOM, won't follow links, won't index content.

Real-world bug: some frameworks serve a JSON error page with Content-Type: text/html. Google tries to render HTML, finds garbage, and the page either doesn't index or indexes with nothing useful on it.

Every HTML page should declare UTF-8 in the charset. If you skip it, browsers guess, and non-ASCII characters can render as question marks in search snippets.

Link Header — HTTP-Level Canonicals

You can set the canonical URL in the HTTP headers instead of (or in addition to) a <link rel="canonical"> tag in the HTML:

Link: <https://yourdomain.com/canonical-url>; rel="canonical"

Why bother? Same reason as X-Robots-Tag: non-HTML files. PDFs, images, and any resource where you can't embed HTML can still signal a canonical via this header.

For HTML pages, the <link> tag is easier to maintain. Pick one mechanism and stick with it — conflicting signals between the header and the HTML tag force Google to guess.

Strict-Transport-Security (HSTS)

Strict-Transport-Security: max-age=31536000; includeSubDomains

HSTS tells browsers to only access your site over HTTPS for the next year. HTTPS is a confirmed ranking factor, and HSTS eliminates the window where a user hits HTTP and gets redirected (which adds latency and leaks a click to an insecure hop).

Enabling HSTS is a one-line nginx or Cloudflare change. The includeSubDomains directive extends it to every subdomain, which also protects assets loaded from cdn.yourdomain.com.

Pair this with an SSL Certificate Checker run once a quarter to confirm your cert hasn't silently expired — an expired cert breaks HSTS hard, and browsers refuse to fall back to HTTP.

Content-Encoding — Compress or Pay the Speed Penalty

Content-Encoding: gzip

or

Content-Encoding: br

Brotli (br) compresses about 15-20% better than gzip for text. Most modern CDNs enable Brotli by default, but some origin servers don't. If your HTML is being served uncompressed over HTTPS, you're shipping 3-5x more bytes than you need to, which shows up directly in LCP scores.

Check for a content-encoding line in your curl output. No line means no compression.

Vary — The Header That Breaks Caches

Vary: Accept-Encoding

Vary tells caches "the response depends on this header, so cache separate copies for each value." It's necessary for compression (Brotli-capable browsers get one version, older ones another).

Where it goes wrong: some sites set Vary: User-Agent thinking it helps mobile/desktop splits. That effectively disables CDN caching because every User-Agent string is unique. Fix by removing it and using responsive design or the Vary: Accept header for content negotiation.

Redirect Headers — The Difference Between 301 and 302

The status line isn't technically a header, but response codes shape what Google does with a URL.

• 301 Moved Permanently: passes full ranking signal, Google updates the index to the new URL within a few crawls.
• 302 Found / Temporary: preserves the old URL in the index; Google eventually treats long-running 302s like 301s, but it's slower.
• 307 Temporary Redirect: same as 302 for SEO purposes but preserves the HTTP method.
• 308 Permanent Redirect: same as 301 but preserves the method.

If you migrate a site and use 302s instead of 301s, rankings drop for weeks while Google figures out what you meant. Always 301 permanent moves. And audit your redirect hops with a Redirect Chain Checker — each extra hop costs crawl budget and a few hundred milliseconds of page speed.

The Short Checklist

• X-Robots-Tag set correctly for PDFs, images, and staging
• Content-Type: text/html; charset=UTF-8 on every HTML page
• Cache-Control tuned: long for static assets, short for HTML
• Strict-Transport-Security enabled with max-age of at least a year
• Content-Encoding: br or gzip on all text responses
• No Vary: User-Agent unless you really know why
• 301 (not 302) for permanent moves, and no redirect chains

Run your homepage and a couple of interior pages through the HTTP Header Checker once a quarter. Most sites go years without a review, and silent regressions — a CDN config change, a framework upgrade — are where the weird indexing bugs come from.

Headers aren't glamorous. They're also where a lot of technical SEO problems actually live.