PgQue: Zero‑Bloat Postgres Queue Goes Pure SQL

What it is

PgQue is a lean queue engine for Postgres that runs entirely in SQL and PL/pgSQL — one SQL file to install, and pg_cron to tick. It aims to resurrect the PgQ architecture in a form that works on managed Postgres platforms where custom C extensions and background daemons are verboten. It has been reported that the original PgQ was designed at Skype and, allegedly, ran messaging workloads on large self‑managed Postgres deployments for over a decade; PgQue promises the same battle‑tested pattern without the extension baggage.

How it works

The trick is simple and clever: snapshot‑based batching plus TRUNCATE‑style table rotation instead of per‑row DELETE/UPDATE and SKIP LOCKED dance. The hot path avoids dead tuples entirely, so index bloat and VACUUM pressure — the usual killers of in‑DB queues under sustained load — don’t show up. No C extension, no shared_preload_libraries, no provider approval, no restart. If you can run SQL on Postgres 14+ (RDS, Aurora, Cloud SQL, AlloyDB, Supabase, Neon, etc.), you can run PgQue.

The trade-offs

This comes with an explicit cost: delivery latency. PgQue batches via snapshots, so end‑to‑end delivery typically lands in the 1–2 second ballpark by default — about 1 s for the next tick plus consumer poll time. Per‑call latencies remain microsecond‑level. Want it faster? Tune the tick frequency, lower thresholds, or call force_tick() for demos; future builds may add logical‑decoding wakeups for sub‑second delivery. If your app needs single‑digit‑millisecond dispatch, look elsewhere. If you need stability, predictability, and zero bloat under heavy sustained load, this is exactly where PgQue shines.

Context and adoption

A quick comparison versus other in‑DB queues shows PgQue preserving snapshot batching and true zero‑bloat behavior where most popular alternatives do not. No external daemon and a pure‑SQL install make it attractive for teams on managed Postgres who don’t want another distributed system to babysit. Interested? The project is on GitHub (https://github.com/NikolayS/pgque) and surfaced on Hacker News; the repo includes docs, client libraries, benchmarks, and a quickstart to get you rolling. Clean, pragmatic, and a bit of old‑school engineering — sometimes the devil you know is the one that keeps running.

Sources: github.com/nikolays, Hacker News