Precision over perception: Why one benchmark’s headline doesn’t tell the whole story

The claim and the pushback

It has been reported that VMware published a blog — citing a Principled Technologies study — claiming VMware Cloud Foundation 9.0 with vSphere Kubernetes Service (VKS) delivers a “5.6x pod density” advantage over Red Hat OpenShift. Red Hat pushed back hard. Their blog argues the test compared two fundamentally different architectures and that the headline number reflects topology choices, not platform efficiency. Ouch. Who wouldn’t squint at a big multiplier and want more context?

The anatomy of the test

Both platforms were run on the same four Dell PowerEdge servers. But the deployment geometry was not equal. VKS was allegedly configured with 300 virtual worker nodes across those four servers (75 VMs per host), each as a “best-effort-medium” VM. OpenShift ran as a bare-metal cluster with four worker nodes — one per physical server. The raw totals: VKS reached 42,000 pods (about 140 pods/node); OpenShift reached 7,400 pods (about 1,850 pods/node). That means OpenShift’s per-node density was more than 13× higher, while the 5.6× headline simply reflects that one side had 300 nodes and the other had four. The methodology doc also shows a YAML-deployed maxPods of 200 for VKS (despite text claiming 300), and OpenShift was set to 5,000 per node (far above the default 250), producing theoretical ceilings of ~60,000 for VKS and ~20,000 for OpenShift — apples, oranges, and a fruit salad headline.

Queueing, real workloads, and what really matters

Red Hat points to basic queuing theory: many small nodes reduce scheduling contention; few large nodes create queues. Want an analogy? Boarding 1,000 people onto four buses will make a line. Putting them into 300 minivans clears the line. Faster pod readiness in the test is therefore predictable — not necessarily a testament to better core engineering, but to topology. Crucially, the test pods were tiny and non-contentious; they didn’t mimic real production workloads that consume CPU, memory, and I/O. So the measured advantage applies mainly to empty scheduling slots, not to packed, noisy production nodes.

What was left out — and the takeaway

Red Hat also notes a missing comparison to OpenShift Virtualization, which would have been relevant if the goal was to compare virtualization-aware Kubernetes stacks. More broadly: benchmarks sell headlines, but architects need topology, maxPods, workload profile, and resource reservation details. Numbers are persuasive — when sliced right. So next time a vendor touts a 5.6× win, ask: win at what, exactly?

Sources: redhat.com, Hacker News