Brain2Qwerty ships two MEG decoders, and only one can be reproduced today

What Brain2Qwerty reads, and who it is for

Every few years a brain decoding result appears that makes the rounds, and the practical question afterwards is always the same: can anyone else run it. brain2qwerty is Meta's answer to that for one specific result, decoding sentences that a person typed while lying in an MEG scanner. The repository is the code behind a Nature Neuroscience 2026 paper titled Non-invasive decoding of typed sentences from human brain activity, plus a 2026 preprint covering the second generation.

The audience is narrow. You need non-invasive recordings, which means a magnetoencephalography scanner, a shielded room and a participant. There is no consumer device in the loop and no implant. What the repository offers an engineer who will never touch an MEG machine is a well-organised example of a modern speech and text decoding stack: Lightning training, pydantic configuration, exact dependency pins, and a staged multi-loss schedule that is documented rather than implied.

V1: one embedding per keystroke, then a sentence model

The first generation lives in brain2qwerty_v1 and works at keystroke level. A convolutional encoder produces one embedding per keystroke window. Those embeddings are grouped by sentence identifier, taken from the trigger metadata on each segment, and fed into a sentence-level Transformer. A linear head emits per-keystroke predictions and the model is trained with per-keystroke cross-entropy.

Two details matter for anyone reading the code. Grouping requires care under distributed training, so the package carries a SentenceGroupedDistributedSampler that keeps all keystrokes of one sentence on a single rank. And the model class itself is thin: the architecture lives in configuration dictionaries rather than in a large model module.

The dataset is SpanishBCBL, registered under the study name Pinet2024Meg. The figure reported for this generation is an MEG character error rate of about 0.36 for the Conv plus Transformer model without a language model, with optional N-gram rescoring lowering it further.

V2: CTC, word-level contrastive alignment and a LoRA-adapted language model

The second generation in brain2qwerty_v2 drops keystroke supervision and decodes whole sentences end to end. It is three components trained together. A Conv and Conformer CTC encoder turns MEG signal into frame-level predictions. A word-level contrastive aligner built on SigLIP and dynamic time warping produces word embeddings that are aligned to text. A language model adapted with LoRA then generates the sentence from the CTC text plus the MEG word embeddings.

The three losses are combined as a weighted sum, with the weights gated by epoch so each head switches on at its own point in training and the remaining weights are renormalised. The CTC head also has an auxiliary logits path that blends two of its outputs. A CTCSpaceSegmenter groups encoder frames into pseudo-words, which is what gives the contrastive head something to align against.

This generation adds two metrics beyond the character error rate: word error rate, and a semantic error rate computed as embedding distance with RoBERTa-large. The agent guide in the repository notes that the CTC greedy character error rate is what drives checkpoint selection.

Two model behaviours had to be reimplemented locally because the upstream packages forbid them. ChannelPositions2D bypasses a guard that disallows 2D MEG layouts, needed for the Fourier positional embedding from the paper. ConvConformer subclasses the upstream ConvTransformer to restore an auxiliary CTC head and a per-frame output the word segmenter consumes.

One dataset you can download, one you cannot

This is the sharpest constraint in the repository and the reason the two packages are not equally useful.

The v1 recordings are public. The README links the SpanishBCBL dataset on HuggingFace. The v2 recordings, registered as PinetAudio2025 and referred to as EnglishBCBL, are under embargo until the paper is accepted. The agent guide describes them as not yet public.

Both datasets were collected by, and belong to, the Basque Center on Cognition, Brain and Language, not to Meta. So even the public half is someone else's data with its own terms.

Practically: you can train and evaluate v1 today if you have the compute. You can read, run the unit tests on, and learn from v2, but you cannot reproduce its published numbers, because the inputs are not available.

Setting up the environment and running something

The README gives no install command. It points at the project website, the Meta blog post and the two papers, and lists the two code directories. What the repository does provide is a pinned dependency set, and the pins are explained rather than incidental.

Python 3.12 or newer is required:

requires-python = ">=3.12"

The direct dependencies are pinned to exact versions, including the Meta lab packages that supply the data and training layers, and the deep learning core:

    "neuralset==0.2.2",
    "neuraltrain==0.2.2",
    "torch==2.6.0",
    "lightning==2.5.2",
    "transformers==4.52.4",
    "peft==0.18.1",

The comment above that list states the reason plainly: the project's goal is to reproduce published numbers, so every version that can influence numerics is frozen to the environment the results were produced in. The repository also ships a requirements.lock carrying the full transitive closure, and the agent guide points at that file for an exact environment.

Data, models and scheduling come from three external packages rather than from local code. neuralset handles the data path, from a study definition through event transforms to a segment dataset. neuraltrain supplies the model, loss and optimizer configuration classes. exca handles feature caching and can fan extraction out over SLURM.

To run anything you set the paths the code expects. The documented environment variables are BRAIN2QWERTY_STUDIES for the v1 data root, BRAIN2QWERTY_STUDIES_EN for the v2 data root, BRAIN2QWERTY_CACHE for the feature cache, BRAIN2QWERTY_RESULTS for checkpoints and prediction files, BRAIN2QWERTY_ROBERTA for the semantic metric model, WANDB_HOST for optional logging, and HF_HOME for the model cache.

Each package exposes the same four modes through its main module: debug for a one-GPU smoke run over a single timeline, train for the full run, eval with a checkpoint path, and cache to pre-warm the feature extraction. The agent guide recommends warming the cache before a full run because feature extraction is CPU bound. Full training is configured for eight GPUs and automatically caps to what is available, while evaluation runs on a single device so the prediction callback sees the whole test split.

What a 0.36 character error rate is not

The number worth internalising is the v1 MEG character error rate of about 0.36 for the model without a language model. That means roughly one character in three is wrong before any rescoring. It is a strong result for non-invasive decoding of typed sentences, and it is nowhere near a usable typing interface.

Other limits are structural. Full training assumes a multi-GPU node, which puts it out of reach of a laptop workflow. The repository implements only the final selected model: one scheduler, one loss path, no ablation switches, so anyone wanting to compare variants has to write them. There are no releases, so there is no tagged version to pin against, only the state of the default branch. The last push was on 2026-09-10, and the repository is not archived.

It is also the wrong starting point if your interest is applied assistive technology. Nothing here is packaged for real-time use, and the licensing rules out shipping it.

Non-invasive decoding against implanted electrodes

The obvious comparison is with invasive brain computer interfaces, where electrodes are implanted in the cortex. That route trades surgery for signal quality: the recorded signal is far closer to the source, so decoding accuracy is higher and the hardware is smaller, at the cost of a surgical procedure and its risks. Meta's bet here is the opposite one, that a sufficiently good model can extract typed sentences from recordings made outside the skull.

The cost lands on the hardware. Non-invasive decoding of this kind needs an MEG scanner, and the recordings used here come from a dedicated research centre. An implanted interface needs an operating room once and a portable device afterwards. For a researcher choosing a direction, that is the real fork: access to a scanner, or access to a clinical programme.

Inside the non-invasive camp, the repository also gives you a comparison between its own two generations: keystroke supervision with a sentence-level Transformer, against end-to-end CTC with a contrastive aligner and a language model. The second is more complex and, on the published numbers, better; it is also the one whose data you cannot get.

Licence, hardware and what you are agreeing to

The code is released under CC BY-NC 4.0. The README states this, and the package classifier in pyproject.toml records the licence as other or proprietary rather than a standard open source identifier. The practical effect, without pretending to give legal advice, is that non-commercial use, modification and sharing are permitted while commercial use is not. Any team evaluating this for a product should stop at that line and get a lawyer's reading of it.

The hardware ask is the second commitment. Eight GPUs for a full training run, a large persistent feature cache, and a HuggingFace cache to pre-download on offline nodes. The repository does document a CPU-only path for its fast unit tests, which are separate from the opt-in checks that touch live data.

What you get in exchange is a reproduction target rather than a library. The configuration files carry the full explicit values used in the papers, the dependencies are frozen to the original environment, and the agent guide asks that any change keep a debug run training cleanly. That is a coherent package for someone trying to stand on published numbers, and an awkward one for someone who just wants a decoder to call.

Editorial conclusion

This is for researchers with access to an MEG scanner and to the SpanishBCBL recordings, or for engineers who want a clean reference implementation of a Conv and Transformer decoder wired to Lightning and a pydantic experiment config. It is not for anyone looking for a communication aid to deploy: the reported MEG character error rate is around 0.36 before language model rescoring, the licence excludes commercial use, and full training is configured for eight GPUs. Verify one thing before committing time: whether you can obtain the dataset you need. The v1 data is on HuggingFace, the v2 data is under embargo until the paper is accepted, and without it the v2 package cannot reproduce its own numbers.

Frequently asked questions

Official sources

1. facebookresearch/brain2qwerty on GitHub
2. Issues
3. README