Storage Engines

KubeMQ ships two persistence engines: legacy (the default) and next. The engine governs only the persistence plane — the Events Store and Queues. Ephemeral patterns (Events pub/sub, Commands/Queries RPC) ride the same internal messaging core in both modes and behave identically regardless of which engine a cluster runs.

The two engines

Choosing an engine

• Default: stay on legacy. Existing deployments see zero behavior change.
• Pick next for quorum-fsynced durability (acked messages survive a node loss) or for Kafka compacted topics.
• Stay on legacy when upgrading an existing cluster — there is no migration path between engines — or when neither of the above matters for your workload.

Enabling the Kafka connector on a fresh store auto-selects next — you do not manually set store.engine: next for Kafka. See Zero-config engine selection below.

Zero-config engine selection

An explicit store.engine (Docker config.yaml) or STORE_ENGINE (env) always wins and skips every inference below. Otherwise, the default engine is legacy.

The one exception is Kafka: on a fresh store, with the Kafka connector enabled and the engine left unset, the server auto-selects next and logs a NOTICE — no manual step required.

This behavior is data-state-dependent, not an unconditional rule:

• Fresh store + Kafka enabled + engine unset → auto-selects next (NOTICE logged).
• Existing legacy store data + Kafka enabled → fails closed with a config error naming the conflicting store directory. Kafka never ran on legacy, so the server refuses to mix engines rather than silently reinterpreting existing data.
• Explicit store.engine: legacy + Kafka enabled → rejected at boot. Kafka requires the next engine; pinning legacy alongside it is a configuration error.
• Pinning STORE_ENGINE=next (or store.engine: next) skips the probe entirely and always wins — the predictable choice for IaC/GitOps that shouldn't depend on probe-time filesystem state.

Mode isolation

A cluster is born one mode and stays there. The data directory records a mode marker at first boot. Every later boot compares that marker against the configured engine — a mismatch is a fatal boot error: the server refuses to start, and it never wipes the directory. There is no in-place engine migration and no cross-format compatibility between legacy and next data.

Durability guarantees

Three distinct dimensions govern durability, and they should not be conflated:

1. Ack meaning. On legacy, a publish is acknowledged after the write lands in the file store; the store is fsynced on a timer (Broker.DiskSyncSeconds, default 5s) — so an ack can precede the fsync. On next, a publish is acknowledged only after the write is quorum-replicated across raft peers and fsynced.
2. Loss window. "Zero acked-loss" on next means an acked message is guaranteed to survive. It does not mean zero loss overall — unacked, in-flight work can still be lost on either engine if the client never receives the ack.
3. Consistency under failover. On next, per-channel sequence numbers are assigned from the committed raft apply order — they are gap-free, monotonic, and restart-stable, even across a leader election.

Native retention scope

The store-level limit and retention settings — Max channels, Max channel size, Max messages, Message retention, and Purge inactive (see Storage & Queues) — are enforced by the legacy engine only. The next engine does not consume them: a native Events Store or Queues channel on next has no age, size, or count cap and grows unbounded under a slow or absent consumer. The queue ack-wait timeout governs redelivery of in-flight messages — it is not a backlog-eviction bound.

On next, the only age-evicted channels are Kafka topic channels, via the Kafka connector's retention.ms (see Durability & Retention). Size native Events Store / Queues workloads on next by disk capacity, or use Kafka topic channels where age eviction matters.

Support & deprecation

Both engines are fully supported today. legacy is feature-frozen — it receives maintenance, security, and bug fixes only. next receives all new durability and compaction features going forward. The deprecation policy for legacy is explicitly TBD — no sunset date has been set, and both engines are supported until one is.

Compaction

Kafka log compaction is next-only. The Kafka connector's cleanup.policy accepts delete (default), compact, or compact,delete. Compaction keeps only the latest record per key and reaps tombstones after delete.retention.ms (default 24h) — it never renumbers surviving offsets. Compaction is scoped to Kafka topic channels only; native Queues and Events Store channels never compact. This is the feature that unlocks Kafka Connect (its internal topics require compaction) and Kafka Streams changelogs.

Clustered next-engine replication

The Cluster.Replication.* block configures the next engine's Dragonboat replication listener — a second membership plane a next-mode cluster runs alongside the cluster mesh. It is meaningful only when Store.Engine is next and Cluster.Enable is true. This block is Docker / config.yaml and env-only — there is no Helm/CRD path.

ReplicaID auto-derive

On a next-engine cluster with clustering enabled, Peers set, Replica ID left at 0, and POD_NAME present in the environment, the server derives ReplicaID = ordinal(POD_NAME) + 1 — the integer after the last - in the pod name, plus one (StatefulSet ordinals are 0-based, so replica ids run 1..N).

• An explicit non-zero Replica ID always wins — the derive path is skipped.
• An unparseable POD_NAME (no <name>-<ordinal> suffix) fails closed.
• The operator obligation: the host at id ordinal + 1 in Peers must be the DNS name of the pod at that ordinal — a mismatch is a DNS-time failure (the cluster boots but never forms quorum), not a config-validation error.

This lets Peers be byte-identical across every pod in a StatefulSet, with no per-pod templating required.

Kubernetes

On Kubernetes, the persistence engine is established once, at cluster creation, by the operator's engine-establishment guard — it is never changed for a live cluster. See Deployment & High Availability for the operator-side behavior.

Example

Set the persistence engine on each target. This is a single-setting snippet — see the Docker guide and the Kubernetes guide for complete, runnable configurations.

store:
  engine: next

store:
  engine: next

See Also