Website owner blocks browsers with inconsistent Sec-CH-UA headers, blames scrapers

What happened

A long‑running personal site and wiki is turning away some visitors after detecting inconsistent Sec‑CH‑UA‑* (User Agent Client Hints) HTTP headers. The site’s operator says certain combinations of User‑Agent and Sec‑CH‑UA values look suspicious — for example, a browser claiming to be Chrome while sending odd or missing hint fields — and are refused outright. Short story: if your headers don't line up, you might get a cold shoulder.

It has been reported that the owner’s move is driven by "abusive high‑volume crawlers" that allegedly forge User‑Agent strings to scrape pages, apparently in part to gather training data for large language models. The author also warns they’re unlikely to make exceptions for software that impersonates Chrome or other mainstream browsers; crawlers that won’t admit what they are, he calls "anti‑social."

Why it matters — and what you can do

This is a small story with a bigger context: more site operators are tightening access controls as web scraping for AI training becomes a hot issue. That’s understandable, but there’s a trade‑off. Privacy‑minded browsers or client libraries that intentionally omit or alter UA hints can be lumped in with bad actors and get blocked — collateral damage in a cat‑and‑mouse game.

If you’re legitimately blocked, the owner asks you to contact them through their university address and include your exact User‑Agent string. Want to be spared the squeeze? Don’t fake Chrome. And if you run a crawler, maybe be honest for a change — the internet is getting picky about manners.

Sources: utcc.utoronto.ca, Hacker News