# Fedlearner: joining model training across institutions that cannot share data

> Fedlearner is a collaborative machine learning framework from ByteDance for joint modelling of data distributed between institutions, built on TensorFlow 1.15 with gRPC protocols and a set of worked examples. It suits a team with a legal or commercial reason to keep training data in separate places.

**bytedance/fedlearner** — A multi-party collaborative machine learning framework

- Repository: https://github.com/bytedance/fedlearner
- Stars: 899 · Forks: 174
- Language: Python
- License: Apache-2.0
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/bytedance-fedlearner

## Joint modelling without moving the data

The one-line description is that Fedlearner is a collaborative machine learning framework that enables joint modelling of data distributed between institutions. Everything else in the repository follows from that phrasing. The interesting word is joint, because the alternative to a framework like this is either pooling the data into one place or building the coordination protocol yourself, and the first option is often blocked by regulation, by data processing agreements, or simply by two competitors who do not want to hand over their records. The repository is structured as a system rather than a library, and the layout shows the parts a distributed training system needs. There is a fedlearner directory for the implementation, a protocols directory for the interface definitions, a cc directory with a build script for a custom TensorFlow operation, an internal web console, deploy and docker directories for packaging, a ci directory with a test script, and a docs directory. The example directory is the part that tells you what the framework is for, and it holds separate worked cases for data portal, fedavg, mnist, privacy, sparse_model, tree_model and wide_n_deep.

## The gRPC protocols are generated, not checked in

The transport layer is protobuf over gRPC, and the interesting detail is that the generated code is not in the repository. The Makefile has a protobuf target that runs the grpc_tools protoc compiler against the protocols directory, with the include path pointing at the TensorFlow path so that the generated stubs can reference TensorFlow's own protobuf definitions:

```
python -m grpc_tools.protoc -I protocols -I$(TF_PATH) \
	--python_out=. \
	--grpc_python_out=. \
	protocols/fedlearner/common/*.proto
```

The same target then compiles the channel protocols and, separately, the cluster and training service protos, which live under the fedlearner/fedavg tree and are compiled with the repository root as the include path rather than the protocols directory. The reason for the split is that the cluster and training service interfaces are specific to the federated averaging implementation while the common and channel interfaces are shared, which is a sign that the framework is meant to be extended with new protocols rather than used as a fixed set. The practical consequence for a new user is that the build is a prerequisite, and the Makefile shows exactly how to do it, with a protobuf target and a separate op target that builds the custom TensorFlow operation.

The op target is worth a second look because it is allowed to fail:

```
op:
	bash cc/build.sh || echo "WARN: make custom tf op failed!"
```

That echo line is a deliberate tolerance. The C++ custom operation is an optimisation, not a requirement, and the build proceeds without it with a warning.

## A dependency set that dates the project precisely

The requirements file is the most informative single artefact in the repository, because it pins the stack the framework was written against. TensorFlow is pinned to 1.15.2, which is the last release of the 1.x line, and pyspark is pinned to 3.0.2 and pandas to 1.1.5, while sqlalchemy sits at 1.2.19 and setuptools at 41.0.0. On top of that sit the pieces a distributed training system needs: grpcio-tools for the protocol compiler, etcd3 for coordination, influxdb and elasticsearch for metrics and logging, opentelemetry-api, opentelemetry-sdk and opentelemetry-exporter-otlp for tracing, cityhash and gmpy2 for hashing and arithmetic, pycryptodomex and rsa for the cryptographic parts, apsw and peewee and leveldb for storage, and scikit-learn, matplotlib and pyspark for the modelling and plotting. The Dockerfile agrees with the requirements, starting from python:3.6 and installing the GMP, MPFR and MPC development libraries plus krb5-user for Kerberos support, then running make protobuf and make op before setting the Python path and the timezone. Kerberos in the base image is a meaningful signal, since institutional deployments of this kind of system frequently sit behind corporate authentication. The honest summary is that this is a 2020 or 2021 stack, and any adoption today begins with a dependency upgrade rather than with a deployment.

## The examples are the documentation for what the framework can do

With a project of this size, the example directory carries most of the practical information, and the set of directories is more informative than the prose. example/fedavg is the canonical case, a federated averaging run. example/mnist is the standard image classification workload, which is the usual way to check that a distributed training loop is behaving. example/privacy is the one that deserves attention, since differential privacy in a collaborative setting is the hardest problem and the one with the most legal weight. example/sparse_model, example/tree_model and example/wide_n_deep cover three quite different model shapes, and the fact that a single framework claims to handle all three is the interesting claim, because the coordination protocol is the part that should be model agnostic while the training loop is not. example/data_portal suggests a fourth kind of task, a place where data is exposed to the system rather than the model being trained. The cost of this breadth is that the framework has a large surface, and a new user is best off picking one example and treating it as the tutorial, which is what the example directory structure invites.

## Testing spans unit tests, integration tests and a CI script

The build and test story is conventional and complete enough to be useful. The Makefile includes a local config.mk if it exists and otherwise falls back to config.mk.template, which is the standard pattern for a project with machine specific paths. The ci target runs a single script:

```
ci:
	bash ci/ci_test.sh
```

The unit test target discovers its tests by find, matching files named test_*.py under the test directory, and turns each into a phony target so that make runs them one by one:

```
UNIT_TEST_SCRIPTS := $(shell find test -type f -name "test_*.py")
UNIT_TESTS := $(UNIT_TEST_SCRIPTS:%.py=%.phony)

test/%.phony: test/%.py
	python $^
```

The integration test target runs integration_tests.sh from the repository root, and the default test target runs both. Linting is pylint with a configuration in ci/pylintrc, applied to the fedlearner and example directories, which tells you the examples are held to the same standard as the library rather than being treated as scratch code. There are also docker-build and docker-push targets that build the Dockerfile and push it under the image name from the config, and a .dockerignore at the root. The Makefile has an unusual quirk worth knowing about, because the phony trick means the unit tests are driven by make rather than by pytest, so a test file that is renamed out of the test_ prefix pattern will simply stop running.

## The trademark policy is a licence with a second half

Most of the README of this project is not about machine learning, it is about trademark usage, and that is unusual enough to be worth reading rather than skimming. The licence field on the repository says Apache-2.0, and the code itself is straightforwardly licensed. The trademark policy adds conditions on top of that. Fedlearner welcomes everyone to build on or modify the open source software for their own project, and the licence of the software does not grant permission to use trademarks or product names in respect of the licensor. The permitted uses are narrow and specific: using a wordmark to refer to the program, product or technology, using a wordmark in text to indicate compatibility, and using a wordmark in text to indicate that your project is built on the technology. Everything else requires express permission, and the list of what needs permission is long. Combining a trademark with your own brand, trademark, product, project, service or domain name, any other commercial use, using it as a verb or noun rather than only as an adjective followed by the generic name, and using it in a modified, abbreviated or altered form, or in the plural or possessive form, all fall into that category. The policy also reserves the right to terminate or modify permission and to take action against non-conforming use. In practice this is the standard pattern for a company that ships a named framework, and it is the section a commercial integrator should read before naming anything.

## Version 1.5 with a date-stamped development suffix

The versioning is unusual and worth understanding before you pin anything. The setup.py computes the version at build time from a base string and the current date:

```
def get_version():
    base = "1.5"
    day = time.strftime('%Y%m%d', time.localtime())
    return '%s-dev%s'%(base, day)
```

So an installed development build reports something like 1.5-dev20260706 rather than a semantic version, and it changes on every build of the same commit. The release history explains why: the tagged releases are v1.5 from 2021-03-22, v1.5-rc3 from 2020-11-30 and v1.5-rc2 from 2020-10-29, with no release since. The last push was on 2026-07-06, so the branch is still receiving work, but the tagged history stopped years ago and there is no release cadence to follow. The distribution metadata also lists the operating systems as MacOS and POSIX Linux, and the licence classifier as the Apache Software License. The repository also carries a NOTICE file alongside the LICENSE, an AUTHORS file, a config.mk.template for local build configuration, and a metrics.ndjson file at the root. A team adopting this should expect to vendor the code rather than depend on a release, and to own the dependency upgrades that the pinned TensorFlow 1.15 line makes unavoidable.

## Conclusion

Fedlearner is a good fit for a research or engineering team whose real constraint is that two institutions cannot pool their records, and who want the training loop, the transport and the examples in one place rather than writing a federated averaging loop from scratch. It is the wrong tool for a project that can already share its data, since the whole machinery exists to avoid that, and it is the wrong tool if you need a framework that tracks current machine learning practice, because the pinned TensorFlow 1.15.2 and the release history that stops at v1.5 from 2021 make this a codebase you will be modernising rather than adopting. Before starting, read the example directories, since fedavg, mnist, privacy, sparse_model, tree_model and wide_n_deep each show a different shape of the problem, and check the protocol build, since the gRPC stubs are generated by the Makefile rather than checked in. The licence is Apache-2.0, and the last push was on 2026-07-06.

## FAQ

### What is Fedlearner and who is it for?

It is a collaborative machine learning framework that enables joint modelling of data distributed between institutions, written in Python. It suits teams where two organisations need to train a joint model but cannot pool their records, and the repository ships worked examples for federated averaging, MNIST, privacy, sparse models, tree models, wide and deep models and a data portal.

### What does Fedlearner require to build?

The Docker image starts from python:3.6 and installs the GMP, MPFR and MPC development libraries plus krb5-user, then runs make protobuf and make op. The requirements pin TensorFlow to 1.15.2, with pyspark 3.0.2, pandas 1.1.5, grpcio-tools, etcd3, influxdb, elasticsearch and the OpenTelemetry packages among the dependencies.

### How are the gRPC protocol stubs generated in Fedlearner?

The Makefile protobuf target runs grpc_tools.protoc over the protocols directory with the TensorFlow path on the include path, producing Python and gRPC Python output for the common and channel protocols, and it separately compiles the cluster and training service protos under the fedlearner/fedavg tree. The generated code is not committed to the repository.

### How do you run the Fedlearner tests?

The unit-test target discovers files named test_*.py under the test directory with find and runs each one as a phony make target. The integration-test target runs integration_tests.sh, the test target runs both, the ci target runs ci/ci_test.sh, and linting uses pylint with the configuration in ci/pylintrc against the fedlearner and example directories.

### What are the restrictions on using the Fedlearner name?

The licence of the software does not grant permission to use trademarks or product names. A wordmark may be used to refer to the program or technology, to state compatibility, or to state that a project is built on it. Combining the mark with your own brand or domain, any other commercial use, using it as a verb or noun, and modified, abbreviated, plural or possessive forms all require express permission from the licensor.

## Sources

- [bytedance/fedlearner on GitHub](https://github.com/bytedance/fedlearner)
- [Issues](https://github.com/bytedance/fedlearner/issues)
- [License: Apache-2.0](https://github.com/bytedance/fedlearner/blob/master/LICENSE)
- [README](https://github.com/bytedance/fedlearner/blob/master/README.md)
- [Releases](https://github.com/bytedance/fedlearner/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/bytedance-fedlearner