DuckDB uses RDBMS to attack classic 'small changes' problem in lakehouses

The idea

DuckDB Labs has rolled out DuckLake v1.0, a production-ready lakehouse format that tries something refreshingly blunt: use a real relational database to manage lakehouse metadata. The pitch is simple and almost mischievous. Parquet hates tiny files; object stores hate lots of tiny reads. So instead of writing a new single-row Parquet file every time a table changes, DuckLake keeps those tiny updates inside a metadata RDBMS (PostgreSQL, SQLite, or DuckDB), batches them up, then flushes bigger Parquet files later. Hannes Mühleisen, DuckDB Labs co‑founder and CEO, explains it plainly: object stores aren’t built for one-row writes, databases are.

Benchmarks and reaction

It has been reported that DuckDB Labs’ own benchmark claims 926× faster queries and 105× faster ingestion compared to Apache Iceberg-based setups. Pedro Holanda, the company’s principal engineer, even wrote about seeing an apparent 1,000× difference in places — a number that, he admitted, made him worry some people would get angry. “Cheating with a better design,” he said, and some in the community have reacted with curiosity rather than outrage. The DuckLake blog and The Register quotes frame this as a pragmatic hack: leverage decades of RDBMS engineering to solve a modern lakehouse pain point.

Why it matters

Why should you care? Because the “small changes” problem is real: adding a single row can cascade into a new file, new metadata writes, and expensive object-store transfers. DuckLake’s model promises lower latency, fewer tiny files, and potentially lower costs — while remaining, DuckDB Labs says, “completely transparent to the user,” a point some have called promising and others are calling cautiously optimistic. Will the broader lakehouse ecosystem — from Databricks and Snowflake to the maintainers of Iceberg and Delta Lake — embrace a database-backed catalog as orthodox? Maybe not overnight. But when a messy old problem gets solved by leaning on what already works, people notice. Small change, big relief.

Sources: The Register