consul-replicate copies a Consul key prefix from one datacenter to another

The problem it solves, stated as a design choice

The project description is that it provides a convenient way to replicate values from one Consul datacenter to another using the consul-replicate daemon.

The daemon integrates with Consul to perform cross-data-center K/V replication. The README states the purpose plainly: it makes it possible to manage application configuration from a central data center, with low-latency asynchronous replication to other data centers, thus avoiding the need for smart clients that would need to write to all data centers and queue writes to handle network failures.

That last clause is the reason to run a daemon instead of a library. Every client application would otherwise need to know the full list of datacenters, write to all of them, queue the writes that fail because a datacenter is unreachable, and retry later. With a replicating daemon in front, the client writes once to the central datacenter and the fan-out is somebody else's problem.

The word asynchronous is doing real work. The replication is not transactional across datacenters, and a read from a secondary datacenter may briefly disagree with a read from the source. That is the trade for not blocking the client on network failures.

The project is written in Go, licensed under MPL-2.0, and the last push to the repository was on 2026-10-19. It has no GitHub releases listed, so the released artefacts are on the releases page rather than in the tag history the README links to.

Installing it and the first command

Installation is deliberately manual, which is normal for a daemon that people distribute themselves.

Download a pre-compiled, released version from the Consul Replicate releases page. Extract the binary using `unzip` or `tar`. Move the binary into `$PATH`.

The full option list is available from the binary itself.

$ consul-replicate -h

And the CLI supports all options in the configuration file and vice-versa, which is the useful property here. Anything in the configuration file can be set as a flag and vice versa, so you can run it once from the command line to find a working invocation and then move it into configuration.

The examples in the README are the four cases you will actually need.

Replicate all keys under global from the nyc1 datacenter.

$ consul-replicate \
  -prefix "global@nyc1"

Replicate all keys under global from nyc1, renaming the key to default in the replicated stores.

$ consul-replicate \
  -prefix "global@nyc1:default"

Do it one time rather than watching for changes.

$ consul-replicate \
  -prefix "global@nyc1" \
  -once

And exclude a sub-prefix.

$ consul-replicate \
  -prefix "global@nyc1" \
  -exclude "global/private" \
  -once

The prefix syntax is the part worth internalising. The source datacenter comes after the at sign, and the target name after the colon, so `global@nyc1:default` reads as the global prefix in nyc1, replicated to a prefix called default elsewhere.

The configuration file, which is HCL with a lot of comment

Configuration files are written in the HashiCorp Configuration Language, and by proxy the configuration is also JSON compatible. That is a useful property for a tool that people tend to template.

The consul block carries the connection settings, and the comments in the example are unusually instructive.

The address defaults to 127.0.0.1:8500, which is the default bind and port for a local Consul agent. The comment explains why you should not talk to a Consul server directly and instead communicate with the local agent, and it gives the reasons: the agent is able to multiplex connections to the Consul server and reduce the number of open HTTP connections, and it provides a well-known IP address for which clients can connect.

That advice is worth taking even though the daemon runs centrally, because the same argument applies to the daemon itself.

The token option is the ACL token used when connecting to Consul, and it is not needed if you did not enable ACLs on the cluster. It is also available through the CONSUL_TOKEN environment variable, which is how you would supply it in a container or under a scheduler.

The auth block holds basic authentication information, and the SSL block handles certificates, with a separate ca_cert and ca_path, an SNI server name, and a verify flag. The comment on verify is blunt: the default is true, which checks the global CA chain, and if you are using a self-signed certificate that you have not added to the CA chain you may want to disable SSL verification, but please understand this is a potential security vulnerability.

Beyond the consul block, exclude blocks name keys to skip, kill_signal sets the signal for a graceful stop with SIGINT as the default and the empty string meaning no signal is listened for, and a log level setting that the README asks you to turn up to debug when reporting a bug.

The retry block, and the fault tolerance design behind it

The retry configuration is the most opinionated part of the file, and the comments explain the reasoning.

The block controls the retry behaviour when an error is returned from Consul. Consul Replicate is highly fault tolerant, meaning it does not exit in the face of failure. Instead, it uses exponential back-off and retry functions to wait for the cluster to become available, as is customary in distributed systems.

The knobs are enabled, which is on by default and so the block is redundant, attempts, which is the number of attempts to make before giving up, and the base sleep time. The comment gives the worked example: with a backoff of 250ms, the sleep times for five retries are 250ms, 500ms, 1s, 2s and then 4s, because each retry sleeps for an exponent of 2 longer than the base.

Setting attempts to zero implements an unlimited number of retries.

There is also max_backoff, the maximum time to sleep between attempts. Setting it to zero means there is no upper limit to the exponential sleep, and the comment shows the difference: with max_backoff set to 10s and backoff set to 1s, the sleep times are 1s, 2s, 4s, 8s, 10s, 10s.

So the default shape is a bounded exponential back-off that eventually stops trying, which is the right choice for a daemon that should not spin forever on a permanent configuration error. The unlimited option is there for the case where the source datacenter being down for a long time is normal and you would rather wait than exit.

The code, and the dependency that does the work

The repository root is flat and small, and the file names map onto the configuration surface.

There is main.go, cli.go, flags.go, config.go, config_prefix.go and config_exclude.go, plus runner.go and mapstructure.go, and each of the config files has a matching test file. There is a version/ directory, a docker/ directory, scripts/, a.release/ directory for release configuration, a.copywrite.hcl for the copyright headers, and a.go-version file pinning the toolchain.

The go.mod tells you what the project is built on. The direct dependencies are hashicorp/consul-template at v0.43.0, hashicorp/consul/api at v1.35.0, go-gatedio, go-multierror, the HashiCorp fork of HCL, mitchellh/mapstructure for decoding the configuration into structs, and pkg/errors.

consul-template being a direct dependency is the interesting one. What that library actually contributes here is not documented in the repository, so treat the delegation as an inference from the dependency list rather than a stated fact. Either way, consul-replicate is not re-implementing everything from scratch; it leans on a maintained HashiCorp library for part of the work, which is a smaller and better-tested thing to own.

The mapstructure dependency is the other half of the configuration story. The config file is HCL, which decodes into an intermediate map, and mapstructure turns that map into the Go structs, which is why the same options can be set as flags and in the file.

The Go directive is 1.26.9, and the Makefile cross-compiles for darwin, freebsd, linux, netbsd, openbsd, solaris and windows on 386, amd64 and arm, with a set of exclusions, and it builds Docker images for alpine and scratch targets.

Where consul-replicate is the wrong choice

Four cases argue against it.

The first is a need for bidirectional or conflict-aware replication. This copies in one direction from a source prefix in one datacenter to a target prefix elsewhere. If two datacenters both write to the same logical configuration and you need them to converge, this is not the tool, and Consul's own multi-datacenter features or a purpose-built replicated store are the place to look.

The second is a requirement for strong consistency reads. The replication is asynchronous and low-latency, which means a write to the source datacenter is not immediately visible in the secondary. If an application must read its own write from any datacenter, that is a design problem this daemon does not solve and does not pretend to.

The third is a cluster without ACLs or without local agents. The configuration assumes you can point the daemon at a Consul agent and present a token, and the recommended address is the local agent rather than a server. A setup without agents, or without a way to distribute tokens, does not fit the shape of the tool.

The alternative in the same direction is to write the fan-out into your application, which the README identifies as the thing this project exists to avoid. The difference in approach is that the daemon centralises the failure handling, the retry policy and the prefix mapping in one place that operations can see, while client-side fan-out pushes that complexity into every service that reads configuration. If you have two or three services, client-side fan-out with a shared library is often less machinery than running and operating a daemon. The daemon earns its place when the number of datacenters, the number of services, or the number of configuration prefixes makes the client-side approach a liability, which is precisely the case the README describes.

Editorial conclusion

consul-replicate fits a team running Consul across multiple datacenters that wants one place to write configuration and does not want every client to implement fan-out and write queuing. It does not fit a team that needs bidirectional or conflict-resolving replication, since this copies in one direction from a source prefix to a target, and it does not fit a Consul deployment without a plan for ACL tokens on both sides. Download a released binary or compile from source, write the HCL configuration with a consul block and the prefix options, and read the retry settings before you rely on the daemon, because its fault tolerance is built on exponential back-off rather than on failing loudly.

Frequently asked questions

Official sources

1. hashicorp/consul-replicate on GitHub
2. Issues
3. License: MPL-2.0
4. Project website
5. README