How to fix Cloudflare "Just a moment..." blocking Playwright scrapes
The problem
My headless Playwright scrape of a Cloudflare-managed site returned the interstitial instead of the page: HTTP 403, <title>Just a moment...</title>, body Attention Required! | Cloudflare. page.waitForSelector('#content') timed out after 30s because the challenge never cleared.
What didn't work
- Patching
navigator.webdriverviaaddInitScript— Cloudflare also fingerprints the TLS ClientHello and theHeadlessChromeUA token, so a JS-only patch is not enough. - Rotating User-Agent strings — the JA3/JA4 hash of stock headless Chromium is the giveaway, not the header text.
- Bumping the timeout to 120s and waiting — a headless session that fails the fingerprint never passes the managed challenge, no matter how long you wait.
The fix
// scrape.js — npm i playwright-extra puppeteer-extra-plugin-stealth
import { chromium } from 'playwright-extra';
import stealth from 'puppeteer-extra-plugin-stealth';
chromium.use(stealth());
const browser = await chromium.launch({
headless: false, // headful clears the managed challenge; headless=new works on low-trust paths
args: ['--disable-blink-features=AutomationControlled', '--no-sandbox'],
});
const ctx = await browser.newContext({
userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/141.0.0.0 Safari/537.36',
viewport: { width: 1366, height: 768 },
locale: 'en-US',
});
const page = await ctx.newPage();
await page.goto('https://target.example.com', { waitUntil: 'domcontentloaded' });
await page.waitForSelector('#content', { timeout: 45_000 });
const html = await page.content();
await browser.close();
console.log(html.length > 5000 ? 'challenge cleared' : 'still blocked');
If the site is high-security tier, switch the browser to Camoufox (Firefox-based, uniform fingerprints) and/or route through a residential exit — datacenter IPs lose on reputation before the JS checks even run.
Why it works
Cloudflare scores the combination of TLS fingerprint, JS inconsistencies and IP reputation; stealth plugins remove the automation tells, a real Chrome UA plus headful kills the HeadlessChrome token, and the managed challenge then clears without interaction.