Open-source project
pavlin-policar/openTSNE avatar
pavlin-policar/openTSNE

openTSNE: a Python t-SNE library built around adding new points to an existing embedding

Extensible, parallel implementations of t-SNE

1,622 stars177 forksPythonBSD-3-Clause

At a glance

What is it?
openTSNE is a BSD-3-Clause Python implementation of t-SNE that ships two solver backends and supports transforming new data into a fitted embedding. It is aimed at people who need to reuse an embedding rather than recompute it, and the build story is the part to check before adopting.
Who is it for?
Adopt openTSNE if you need to project new samples into an embedding you already computed, or if you are embedding data sets large enough that the FIt-SNE backend and OpenMP threading matter. Stay with scikit-learn's TSNE if you only ever fit once on a small matrix and never revisit it, because the extra build requirements buy you nothing there.
Can I use it commercially?
Yes. BSD-3-Clause is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 7 days ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.

DEEP OPEN-SOURCE ANALYSIS

The problem openTSNE targets: embeddings you have to recompute

Plain t-SNE has an awkward property in production. You fit it once, get a two-dimensional layout, and then a new sample arrives. The standard answer is to refit everything, which changes the coordinates of every point you already plotted. Any downstream artifact keyed to those coordinates, a cluster label, a figure in a report, a saved index, becomes invalid.

openTSNE addresses that directly. The README lists "the ability to add new data points to existing embeddings" as one of the improvements it incorporates, citing Poličar, Stražar and Zupan on embedding to a reference t-SNE space. That reference-space framing is the point: the embedding becomes an object you keep, not a one-shot computation.

The audience follows from this. Single-cell work is the obvious case, and the README's own example figure is 44,808 mouse retina transcriptomes from Macosko et al. embedded with the multiscale kernel trick for global alignment. But the same shape of problem appears anywhere a fixed reference projection is useful: a batch of new samples scored against a reference cohort, or a dashboard where the axes must not move between refreshes.

Two solver backends and what the default actually is

openTSNE implements two algorithms, and the README is explicit about which one you get. FIt-SNE is the default, and the Barnes-Hut variant is the alternative. The citation block tells you to cite Linderman et al. for FIt-SNE and van der Maaten plus Yang, Peltonen and Kaski for Barnes-Hut. This is not a cosmetic distinction. They are different approximation strategies with different authors, and the library makes you acknowledge which one produced your figure.

The README describes FIt-SNE as interpolation-based, following the Nature Methods paper. Barnes-Hut is the tree-based acceleration from the 2014 JMLR paper, with the Yang et al. ICML work folded in. Both are approximations to the exact gradient, and both are what let the library claim scaling to millions of data points in the README's own wording.

One detail worth noticing: the README says FFTW3 is "heavily used in openTSNE" because the Fast Fourier Transform is central to the interpolation approach. That is a structural consequence of choosing FIt-SNE as the default, not an optional optimization. If you cannot install FFTW3, openTSNE falls back to numpy's FFT, which the README describes as "slightly slower" with the difference "only noticeable with large data sets containing millions of data points."

The build is the real adoption cost, not the API

The API surface shown in the README is small. Load iris from scikit-learn, then:

from openTSNE import TSNE embedding = TSNE().fit(x)

That is the whole hello world. No perplexity tuning, no learning rate, no initialization argument. The defaults are meant to be usable, and for a first look they are.

Installation is where the friction sits. Conda users get a binary through conda-forge:

conda install --channel conda-forge opentsne

pip users get the same from PyPI:

pip install opentsne

Source installs are different. The README instructs running `pip install .` in the root directory, and states plainly that openTSNE requires a C/C++ compiler on the system. This is not a pure-Python package. The binary files are compiled during installation, as the README says the command will "install the appropriate dependencies and compile the necessary binary files."

Threading has its own requirement. For multiple threads, the compiler must support OpenMP. The README notes that almost all compilers implement it, with the exception of older clang versions on OSX. So a macOS user on an old toolchain can install openTSNE successfully and still get single-threaded execution without any error. That is a silent degradation, and it is the kind of thing worth checking before you trust a timing.

Where openTSNE is the wrong choice

The most common case for t-SNE is exploratory: load a matrix, fit once, look at the picture, move on. If that is your workflow, openTSNE's distinguishing feature is dead weight. You never call transform, you never keep the embedding object, and you have taken on a compiled dependency and an OpenMP requirement for nothing.

The second case is reproducibility across environments. Because the default backend is FIt-SNE and the alternative is Barnes-Hut, and because FFTW3 presence changes the FFT path, two machines can produce different coordinates from the same input without either being wrong. The README does not present this as a bug, and it is not one, but it means an embedding is tied to the configuration that produced it. If your pipeline compares embeddings generated on different hosts, pin the backend and the FFT implementation deliberately.

The third case is scale in the opposite direction. The README frames the FFTW3 versus numpy FFT gap as only mattering "with large data sets containing millions of data points." For a few thousand rows, the backend choice and the FFT library are close to irrelevant, and the simpler option wins on setup time alone.

Finally, there is the question of what t-SNE is for. openTSNE makes embeddings reusable and fast; it does not make them interpretable. Distances between clusters in the output are not meaningful in the way a linear projection's distances are. Nothing in the library changes that, and the README does not claim otherwise.

scikit-learn's TSNE as the alternative, and the actual difference

The README's own example imports from sklearn to get the iris data, which makes the comparison natural. scikit-learn ships a TSNE class in sklearn.manifold. It is pure Python plus compiled extensions distributed as wheels, it installs with the rest of scikit-learn, and it has no separate FFTW3 or OpenMP story to manage.

The difference in approach is not accuracy. It is lifecycle. scikit-learn's TSNE is a fit_transform estimator: you hand it a matrix, it hands you coordinates. There is no documented path in the material here for projecting a new sample into an existing scikit-learn embedding without refitting. openTSNE is built around the opposite assumption, that the embedding persists and new points are mapped into it, which the README traces to the reference-space paper.

A second difference is backend exposure. openTSNE names its two algorithms and asks you to cite the one you used. scikit-learn's TSNE exposes a method parameter with Barnes-Hut and exact options. Both let you choose an approximation strategy; openTSNE's default is the interpolation-based method, which is the newer of the two lines of work.

If your only requirement is a scatter plot from a fixed matrix, scikit-learn is the smaller dependency and the shorter install. If your requirement includes the word "again" applied to new data, that is the gap openTSNE was written to fill.

Version history, maintenance and licence

The release record available here shows v1.0.0 in May 2023, v1.0.1 in November 2023, and v1.0.2 in August 2024. The repository's last push is dated September 2024, and it is not archived. That is a slow but live cadence: roughly one patch per year across the visible window, with no major version bump since 1.0. The project also has a published paper in the Journal of Statistical Software (2024, volume 109, issue 3), which is a stronger signal of documentation intent than a changelog alone.

For upgrade cost, the practical exposure is the compiled extension. A patch release can require a rebuild, and a rebuild requires the same compiler, OpenMP and optionally FFTW3 setup as the original install. If you installed via conda-forge or PyPI, that is handled for you. If you built from source in a container, your Dockerfile carries the toolchain and the rebuild is part of your image build, not a separate chore. Either way, the dependency is not something you can upgrade by editing a version string alone.

On licensing: openTSNE is BSD-3-Clause. That is a permissive licence, and the repository shows the standard badge pointing at opensource.org. The README also asks that you cite the underlying algorithm papers, FIt-SNE or Barnes-Hut depending on your choice. Citation is a scholarly norm rather than a licence term, but if you are publishing results, the two obligations arrive together and it is worth deciding which backend you used before you write the methods section. This is a description of what the repository states, not legal advice; check the licence text and your own counsel for anything binding.

What to confirm before you commit

Three checks decide whether openTSNE fits. First, does your workflow actually revisit the embedding? If you can name the code path that calls transform on new data, the library is earning its keep. If you cannot, it is not.

Second, what does your build environment provide? Run a source install once and confirm the compile succeeds and that OpenMP is active. On macOS with an older clang, the README warns that threading support may be missing, and that failure is quiet. Then check whether FFTW3 is available; the README says the numpy fallback is slower, and the difference shows up at the million-point scale it names.

Third, which backend will you cite and pin? FIt-SNE is the default and carries the Linderman et al. citation. Barnes-Hut carries van der Maaten and Yang et al. Both are legitimate; mixing them across runs of the same analysis is not, because the coordinates will not line up. Set the choice explicitly in your code and record it alongside the embedding, the same way you would record a random seed.

Editorial conclusion

Adopt openTSNE if you need to project new samples into an embedding you already computed, or if you are embedding data sets large enough that the FIt-SNE backend and OpenMP threading matter. Stay with scikit-learn's TSNE if you only ever fit once on a small matrix and never revisit it, because the extra build requirements buy you nothing there. Before committing, verify two things on your own machine: that your compiler provides OpenMP, and whether FFTW3 is present, since the README states the numpy FFT fallback is slower.

Official sources

  1. License: BSD-3-Clause
  2. pavlin-policar/openTSNE on GitHub
  3. Project website
  4. README
  5. Releases
Community notes

Community notes