X-Robots-Tag Header
Checks X-Robots-Tag HTTP header for unintentional noindex directives.
What this check measures
We read the X-Robots-Tag header on every response. Same semantics as meta robots, but set at the server/CDN level. Accidental noindex here is common and hard to debug.
Why it matters
X-Robots-Tag: noindex on all responses makes you vanish from Google. Harder to spot than meta robots because it's not in the HTML — hidden in response headers.
How our audit detects it
Read response headers, parse X-Robots-Tag. Flag any noindex/nofollow value on indexable pages.
Typical findings
- error_outlineX-Robots-Tag: noindex on every response from a misconfigured Nginx.
- error_outlinenoarchive set on pages that should be cached by Google.
How to fix
Audit server/CDN config for X-Robots-Tag rules. Only set noindex on utility paths (admin, login, API). Verify in Search Console → Pages report.
Frequently asked questions
Meta robots or X-Robots-Tag?expand_more
Want this checked on your site?
Pantra runs the full audit (SEO, Security, GEO, Performance, Schema, Technical, Images) in 10 seconds and generates stack-specific fix prompts.
Scan my siteRelated checks
SSR / Pre-rendering
Checks if the page is server-rendered or statically pre-rendered.
TechnicalHTTP Status Codes
Checks that pages return proper status codes — 200 for live, 404 for missing, 301 for moved.
TechnicalNo Client-Side Redirects
Checks that redirects happen server-side, not via JavaScript.
TechnicalViewport Meta Tag
Checks for <meta name="viewport" content="width=device-width, initial-scale=1">.