SEO Cloaking Checker

I use a cloaking checker as a post-deploy sanity check, and I learned that the hard way. After a plugin update on a WordPress site, organic traffic dropped 40% over a week. The page itself looked fine to me. I ran a cloaking checker and it showed that Googlebot was being served a 302 redirect to a staging domain, while human visitors saw the real page. The plugin had a misconfigured bot-detection rule. That single check saved me days of guesswork.

A good checker works by fetching the URL twice: once with a Googlebot user agent and once with a standard browser. It then compares the HTTP response headers, the HTML body, and sometimes the rendered DOM. When a mismatch appears, the tool highlights the difference. For me, the most useful output is a side-by-side diff of the two responses. I want to see whether the bot got a different status code, a different canonical tag, or different content entirely.

I also use it to spot hidden text or links. Some plugins inject content that is visually hidden via CSS but present in the HTML. Google treats that as a spam policy violation. A cloaking checker that parses the HTML can flag elements with display:none or visibility:hidden that contain text or links. I have caught affiliate links hidden behind zero-opacity divs that were only visible to crawlers.

The key decision rule I follow is: if the bot gets a different HTTP status code than the user, investigate immediately. If the bot sees different content but the same status, check whether the difference is layout-only or substantive. Layout differences for mobile-first indexing are often legitimate; content differences are not.

One admission: I used to think a cloaking checker was a silver bullet. I now know it only compares what it fetches. If a page relies on JavaScript to inject content after the initial HTML load, a simple user-agent swap will miss the mismatch. That leads me to what the tool is awkward for.

A cloaking checker is awkward for sites that use JavaScript to render content, for sites with complex personalisation, and for sites that serve different content based on IP geolocation. I manage a site that displays pricing in different currencies based on the visitor's IP. When I run a cloaking checker, it flags a mismatch every time because Googlebot's IP is typically US-based while my test browser is UK-based. That's not cloaking; it's e-commerce. The tool cannot tell the difference between a legitimate business decision and a deceptive practice.

JavaScript-heavy single-page applications are another pain point. Most online cloaking checkers only fetch the raw HTML from the server. They do not execute JavaScript. If your app loads the main content via an API call after the initial render, the checker compares an empty shell against a fully rendered page. The mismatch is real, but it is not cloaking—it is a JavaScript architecture. For those sites, I use a headless browser tool like Puppeteer or Playwright to fetch the URL with a Googlebot user agent, wait for JavaScript to execute, and then compare the DOM. That is more reliable than any online checker I have tried.

Dynamic rendering is a grey area. Google's documentation allows serving pre-rendered content to bots as long as the underlying content is equivalent. But a basic checker cannot verify equivalence; it only sees a diff. I once flagged a client's site for cloaking because the dynamically rendered version had slightly different heading text. It turned out to be a caching issue in the prerender service. The tool was right about the mismatch but wrong about the intent.

Another awkward case is content behind login walls. If your site requires authentication, Googlebot will see a login page while logged-in users see the full content. A cloaking checker will call that a mismatch. But Google knows about paywalls and login gates; it does not treat them as cloaking as long as the crawler can access a representative sample. The checker cannot distinguish between a paywall and a hidden-content scheme.

My decision rule for these edge cases is: if the difference is caused by user-agent detection alone, it is suspicious. If it is caused by IP, geolocation, login state, or device type, it is probably legitimate—but verify the content is equivalent in meaning.

The best free alternative is Google Search Console's URL Inspection tool. It shows you exactly what Google crawled and what it rendered after JavaScript execution. I use it to cross-reference against what I see in my browser. If the rendered HTML differs from what I see on screen, I know there is a potential issue. The tool also flags any directives like noindex or canonical tags that might differ. It does not do a side-by-side diff with a user-facing view, so you have to do that comparison manually.

For automated bulk checks, I use Screaming Frog SEO Spider. I configure it to crawl the site as Googlebot (custom user agent) and then export the crawl data. I can compare the status codes, meta tags, and headers against a baseline crawl with a regular browser user agent. Screaming Frog also checks for JavaScript-rendered content if you enable the headless browser option. That makes it more thorough than most online checkers.

For a quick command-line sanity check, I run curl -A "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" https://example.com/page and compare the output with a standard curl without a user agent. This catches redirects and header differences instantly. It is not a full solution but it is the fastest way to rule out a common class of cloaking.

I also rely on the [301 redirect](/301-redirect/) audit as part of my [SEO audit](/seo-audit/) process. A cloaking issue often manifests as a redirect that only fires for bots. Checking redirect chains for each user agent is tedious but necessary.

Another alternative is to monitor [canonical tags](/canonical-tags/) and [robots.txt](/robots-txt/) for differences between crawler and user views. If the canonical tag changes depending on the user agent, that is a red flag. Similarly, if a page is disallowed for some user agents but not others, it could cause a mismatch.

For [JavaScript SEO](/javascript-seo/) sites, I recommend a headless browser approach. I wrote a small Node script using Puppeteer that fetches a URL with Googlebot user agent, waits for the network to idle, and then dumps the DOM. I compare that against the DOM from a headless browser with a standard user agent. That is the only way I trust for modern web apps.