# Storage Engines (/docs/configure/reference/storage-engines)
KubeMQ ships **two persistence engines**: `legacy` (the default) and `next`. The engine
governs **only the persistence plane** — the Events Store and Queues. Ephemeral patterns
(Events pub/sub, Commands/Queries RPC) ride the same internal messaging core in both modes
and behave **identically** regardless of which engine a cluster runs.
## The two engines [#the-two-engines]
| | `legacy` (default) | `next` |
| ---------------------------------------------- | -------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Persistence | the legacy persistence engine (file store) | owned segment log + Dragonboat raft |
| Durability on ack | accepted into the legacy file store; fsync on a timer (`Broker.DiskSyncSeconds`, default 5s) | ack only after quorum-replication **and** fsync (zero acked-loss) |
| Cluster consensus | file-store raft over the cluster mesh | Dragonboat raft over a dedicated replication listener (optional mTLS, default off) |
| Kafka compaction | not supported (Kafka is `next`-only) | supported — unlocks Kafka Connect and Kafka Streams |
| Native store limits & retention (`store.max*`) | enforced | **not consumed** — native channels have no age/size/count cap ([details](#native-retention-scope)) |
| Status | current default; **feature-frozen** (maintenance/security/bug fixes only) | receives all new durability and compaction features |
## Choosing an engine [#choosing-an-engine]
* **Default: stay on `legacy`.** Existing deployments see zero behavior change.
* **Pick `next`** for quorum-fsynced durability (acked messages survive a node loss) or
for Kafka compacted topics.
* **Stay on `legacy`** when upgrading an existing cluster — there is no migration path
between engines — or when neither of the above matters for your workload.
Enabling the Kafka connector on a **fresh** store **auto-selects `next`** — you do not
manually set `store.engine: next` for Kafka. See
[Zero-config engine selection](#zero-config-engine-selection) below.
## Zero-config engine selection [#zero-config-engine-selection]
An explicit `store.engine` (Docker `config.yaml`) or `STORE_ENGINE` (env) always wins and
skips every inference below. Otherwise, the default engine is `legacy`.
The **one exception** is Kafka: on a **fresh** store, with the Kafka connector enabled and
the engine left unset, the server auto-selects `next` and logs a NOTICE — no manual step
required.
This behavior is **data-state-dependent**, not an unconditional rule:
* **Fresh store + Kafka enabled + engine unset** → auto-selects `next` (NOTICE logged).
* **Existing `legacy` store data + Kafka enabled** → fails closed with a config error
naming the conflicting store directory. Kafka never ran on `legacy`, so the server
refuses to mix engines rather than silently reinterpreting existing data.
* **Explicit `store.engine: legacy` + Kafka enabled** → rejected at boot. Kafka requires
the `next` engine; pinning `legacy` alongside it is a configuration error.
* **Pinning `STORE_ENGINE=next`** (or `store.engine: next`) skips the probe entirely and
always wins — the predictable choice for IaC/GitOps that shouldn't depend on
probe-time filesystem state.
## Mode isolation [#mode-isolation]
A cluster is **born one mode and stays there**. The data directory records a mode marker
at first boot. Every later boot compares that marker against the configured engine — a
**mismatch is a fatal boot error**: the server refuses to start, and it **never wipes the
directory**. There is no in-place engine migration and no cross-format compatibility
between `legacy` and `next` data.
## Durability guarantees [#durability-guarantees]
Three distinct dimensions govern durability, and they should not be conflated:
1. **Ack meaning.** On `legacy`, a publish is acknowledged after the write lands in the
file store; the store is fsynced on a timer (`Broker.DiskSyncSeconds`, default 5s) —
so an ack can precede the fsync. On `next`, a publish is acknowledged **only after**
the write is quorum-replicated across raft peers **and** fsynced.
2. **Loss window.** "Zero acked-loss" on `next` means an **acked** message is guaranteed
to survive. It does **not** mean zero loss overall — unacked, in-flight work can still
be lost on either engine if the client never receives the ack.
3. **Consistency under failover.** On `next`, per-channel sequence numbers are assigned
from the committed raft apply order — they are **gap-free, monotonic, and
restart-stable**, even across a leader election.
## Native retention scope [#native-retention-scope]
The store-level limit and retention settings — **Max channels**, **Max channel size**,
**Max messages**, **Message retention**, and **Purge inactive** (see
[Storage & Queues](/docs/configure/reference/storage-queues)) — are enforced by the
**`legacy` engine only**. The `next` engine does not consume them: a native Events Store
or Queues channel on `next` has **no age, size, or count cap** and grows unbounded under
a slow or absent consumer. The queue ack-wait timeout governs redelivery of in-flight
messages — it is not a backlog-eviction bound.
On `next`, the only age-evicted channels are **Kafka topic channels**, via the Kafka
connector's `retention.ms` (see
[Durability & Retention](/docs/connectors/kafka/concepts/durability-and-retention#retention-time-and-size)).
Size native Events Store / Queues workloads on `next` by disk capacity, or use Kafka
topic channels where age eviction matters.
## Support & deprecation [#support--deprecation]
Both engines are **fully supported today**. `legacy` is **feature-frozen** — it receives
maintenance, security, and bug fixes only. `next` receives all new durability and
compaction features going forward. The deprecation policy for `legacy` is **explicitly
TBD** — no sunset date has been set, and both engines are supported until one is.
## Compaction [#compaction]
Kafka log compaction is **`next`-only**. The Kafka connector's `cleanup.policy` accepts
`delete` (default), `compact`, or `compact,delete`. Compaction keeps only the latest
record per key and reaps tombstones after `delete.retention.ms` (default 24h) — it never
renumbers surviving offsets. Compaction is scoped to Kafka topic channels only; native
Queues and Events Store channels never compact. This is the feature that unlocks Kafka
Connect (its internal topics require compaction) and Kafka Streams changelogs.
## Clustered next-engine replication [#clustered-next-engine-replication]
The `Cluster.Replication.*` block configures the `next` engine's Dragonboat replication
listener — a second membership plane a `next`-mode cluster runs alongside the cluster
mesh. It is meaningful **only** when `Store.Engine` is `next` **and** `Cluster.Enable` is
`true`. This block is Docker / `config.yaml` and env-only — there is **no Helm/CRD path**.
| Setting | Type | Default | Valid values | Docker (config.yaml key · env var) | Helm/CRD path | Notes |
| -------------------- | ------ | ------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Replica ID | uint64 | `0` | ≥ 0 | `cluster.replication.replicaid` · `CLUSTER_REPLICATION_REPLICA_ID` | — | This node's Dragonboat replica id (1..N); must be a key in `Peers`. May be left `0` when `POD_NAME` is present — see [ReplicaID auto-derive](#replicaid-auto-derive). |
| Peers | string | `""` | `id@host:port,...` | `cluster.replication.peers` · `CLUSTER_REPLICATION_PEERS` | — | Full initial-members map — every initial member, including self. Byte-identical on every pod. |
| Join | bool | `false` | true / false | `cluster.replication.join` · `CLUSTER_REPLICATION_JOIN` | — | Boot as a later-added replica, with an empty members map. |
| RTT (ms) | uint64 | `200` | ≥ 0 | `cluster.replication.rttmillisecond` · `CLUSTER_REPLICATION_RTT_MILLISECOND` | — | Dragonboat logical-clock tick, in milliseconds. |
| Election RTT | uint64 | `10` | ≥ 0 | `cluster.replication.electionrtt` · `CLUSTER_REPLICATION_ELECTION_RTT` | — | Must be greater than `2 × HeartbeatRTT`. |
| Heartbeat RTT | uint64 | `1` | ≥ 0 | `cluster.replication.heartbeatrtt` · `CLUSTER_REPLICATION_HEARTBEAT_RTT` | — | |
| Snapshot entries | uint64 | `10000` | ≥ 0 | `cluster.replication.snapshotentries` · `CLUSTER_REPLICATION_SNAPSHOT_ENTRIES` | — | Committed entries between automatic snapshots. |
| Compaction overhead | uint64 | `2000` | ≥ 0 | `cluster.replication.compactionoverhead` · `CLUSTER_REPLICATION_COMPACTION_OVERHEAD` | — | Log entries retained past a snapshot. |
| Boot timeout (s) | int | `60` | ≥ 0 | `cluster.replication.boottimeoutseconds` · `CLUSTER_REPLICATION_BOOT_TIMEOUT_SECONDS` | — | Clustered-boot readiness-probe budget, in seconds. |
| Mutual TLS | bool | `false` | true / false | `cluster.replication.mutualtls` · `CLUSTER_REPLICATION_MUTUAL_TLS` | — | Optional mTLS trust domain. Default **off** (plaintext) — see the security callout below. |
| CA / Cert / Key file | string | `""` | file path | `cluster.replication.cafile` / `cluster.replication.certfile` / `cluster.replication.keyfile` · `CLUSTER_REPLICATION_CA_FILE` / `CLUSTER_REPLICATION_CERT_FILE` / `CLUSTER_REPLICATION_KEY_FILE` | — | A **dedicated** trust domain — not reused from `Security`. Required (fail-closed) when `Mutual TLS` is `true`. |
When **Mutual TLS** is left at its default `false`, a clustered `next` node's Dragonboat
replication listener runs an **unauthenticated, FSM-writing raft port**. This is
acceptable only on a trusted pod network, bounded by a `NetworkPolicy` — never expose this
listener beyond the cluster's own pod network.
## ReplicaID auto-derive [#replicaid-auto-derive]
On a `next`-engine cluster with clustering enabled, `Peers` set, `Replica ID` left at `0`,
and `POD_NAME` present in the environment, the server derives
`ReplicaID = ordinal(POD_NAME) + 1` — the integer after the last `-` in the pod name, plus
one (StatefulSet ordinals are 0-based, so replica ids run `1..N`).
* An **explicit non-zero** `Replica ID` always wins — the derive path is skipped.
* An **unparseable** `POD_NAME` (no `-` suffix) fails closed.
* The operator obligation: the host at id `ordinal + 1` in `Peers` must be the DNS name
of the pod at that ordinal — a mismatch is a DNS-time failure (the cluster boots but
never forms quorum), not a config-validation error.
This lets `Peers` be byte-identical across every pod in a StatefulSet, with no per-pod
templating required.
## Kubernetes [#kubernetes]
On Kubernetes, the persistence engine is established once, at cluster creation, by the
operator's engine-establishment guard — it is never changed for a live cluster. See
[Deployment & High Availability](/docs/configure/reference/deployment) for the operator-side
behavior.
## Example [#example]
Set the persistence engine on each target. This is a single-setting snippet — see the
[Docker guide](/docs/configure/docker) and the
[Kubernetes guide](/docs/configure/kubernetes) for complete, runnable configurations.
```yaml title="config.yaml"
store:
engine: next
```
```yaml title="values.yaml"
store:
engine: next
```
## See Also [#see-also]