September 29, 2026
Day 8 — The Bug That Wasn't a Bug
Day 7 proved the basic shape: producer publishes, consumer reads, decoupled. What it didn't touch is how Kafka scales the consumer side — a topic split into partitions, a consumer group dividing those partitions among its members. Today was about watching that division happen, watching it re-divide when a consumer dies, and — mostly by accident — learning that the thing that looks most like a bug in this setup is usually just arithmetic.
Second consumer, first mystery
Two consumers, same groupId, a fresh 3-partition topic. Consumer 1
logged everything. Consumer 2 logged nothing at all. Every instinct says
broken consumer group.
It wasn't. kafkajs's default partitioner sends a keyed message to
murmur2(key) % numPartitions — deterministic, always the same
partition for the same key. I'd been testing with two keys, Bane and
Jocker. I ran both through the actual partitioner function:
Bane -> partition 0
Jocker -> partition 0
Both keys hashed to the same partition. Every message, regardless of which of the two keys it used, went to partition 0 — a roughly 1-in-9 coincidence with 2 keys and 3 partitions, not a bug. Consumer 2 had been correctly assigned partition 1; there was simply nothing produced there for it to receive. The rebalance, the assignment, the group protocol — all of it was working. The keys just weren't spread out enough to prove it.
The bug that actually was a bug
Swapping in keys that hash to different partitions surfaced a real one. The random-selection loop:
const ui = Math.floor(Math.random() * 4); // keys.length is 5
const mi = Math.floor(Math.random() * 7); // values.length is 8
* 4 against a 5-element array means index 4 — "Two-Face" — can
mathematically never come up. Same shape on * 7 against 8 values. Both
should have read .length. Nothing crashes when this happens, which is
exactly why it's the kind of bug that survives — one key in the array
just quietly never gets exercised, and everything still looks like it's
working.
Sitting right above that, a second issue: an explicit partition: 1
override for "Riddler" sat above an unconditional push that ran on
every iteration regardless. Every time the random key landed on
"Riddler", it got sent twice — once forced, once through the normal
path. In the logs that shows up as the same message appearing back to
back, which reads exactly like a duplicate-delivery bug and isn't one.
The identical-logs mystery
The strangest moment of the day: two consumer terminals showing the exact same sequence of messages, in the exact same order, down to every value. That's structurally impossible for a real concurrent split — a partition can only be owned by one group member at a time, so two consumers can't both legitimately replay its full history while both are actively part of the group.
They weren't concurrent. The first consumer had been stopped with
Ctrl+C without committing its offset — Day 7's exact lesson, showing up
again in a completely different context. When the second consumer later
became the sole owner of that partition, fromBeginning: true replayed
everything from scratch, because there was no committed offset telling
it otherwise. Identical output across two terminals turned out to be the
signature of "these ran one after another," not "partitioning is
broken."
What a correct split actually looks like
Once the key set was fixed and both consumers were genuinely running at the same time, the logs told a clean, boring, correct story — which is exactly what you want:
Consumer 1 — memberAssignment: { "todos-v2": [1] } → Penguin, Riddler
Consumer 2 — memberAssignment: { "todos-v2": [0,2] } → Jocker, Bane, Two-Face
No overlap. Killing consumer 2 triggered "the group is rebalancing" on
consumer 1's heartbeat, and its next join log showed
memberAssignment: [0,1,2] — it had absorbed both orphaned partitions
and started successfully logging messages from partitions it never used
to own. Restarting consumer 2 triggered a second rebalance, splitting
things back down.
Two more things worth seeing, not just reading about
A second, differently-named consumer group (todo-consumer-analytics)
run against the same topic at the same time as the first received
every message independently — the same primitive doing queue-style
division of work within one group, and pub/sub-style fan-out across two
different ones, depending only on whether groupId matches.
And a fourth consumer added to a group with only 3 partitions joined
successfully with "memberAssignment":{}" — not an error, not a
rejection, just an idle member sitting in the group, ready to pick up a
partition the moment one frees up. Oversubscription doesn't break
anything; it just leaves someone waiting.
What's next
Day 9 moves up a level: event-driven architecture with SNS/SQS and GCP
Pub/Sub — the managed-cloud version of today's decoupling, minus running
your own broker. Full code in the
repo — day-08-kafka-consumer-groups-partitioning/.