Ask HN: How do systems (or people) detect when a text is written by an LLM?

A long Hacker News thread has been unpacking a question that's become painfully relevant: how can you tell whether a piece of writing came from a human or a large language model? It has been reported that participants ranged from academic researchers to curious devs, all trading tips, war stories, and healthy skepticism. The mood swung between “we can probably do this” and “don’t bet your career on it,” which felt oddly human — and a little tense.

How detection supposedly works

Commenters sketched out the usual toolkit: statistical fingerprints like token distribution, perplexity and burstiness, stylometric features (sentence length, punctuation habits), watermarking embedded by model providers, and supervised classifiers trained on model vs human corpora. It has been reported that several people pointed to the classic trick — look for unusually consistent probability patterns across tokens — while others touted vendor watermarking as the cleanest guarantee. Some even mentioned metadata and provenance systems as part of the stack. Allegedly, each approach has shown promise in controlled tests.

Why the reality is messy

But the thread quickly turned to caveats. Human editing breaks fingerprints. Fine-tuned or prompt-engineered models mimic individuals. False positives are a real fear — accuse someone wrongly and you’ve ruined trust. There’s an arms race vibe here, like spam detection before it got exhausting. And yes, legal and ethical questions popped up: do you want platforms flagging authors automatically? Do you want your writing fingerprinted? It’s not just a technical problem; it’s social, too.

Detection is possible, in spots, and useful in narrow contexts. But it is not a silver bullet. The consensus on Hacker News was pragmatic: use multiple signals, be transparent about uncertainty, and don’t treat any single detector as gospel. Who knew adopting an LLM would lead us back to the oldest rule in security — never trust a single source of truth.

Sources: Hacker News