code2seq: generating a code summary from the structure of the code

The model treats a function as a tree, not a token stream

code2seq is the official implementation of the model described in the ICLR 2019 paper by Uri Alon, Shaked Brody, Omer Levy and Eran Yahav, which appeared as a poster at that conference. The idea the paper tests is that code has structure, and a model that consumes that structure can generate a description of what the code does rather than a single label describing what it is. The README contrasts this with the previous model in the same line of work, code2vec, which can only generate a single label at a time rather than a sequence, but which is much faster to train because of its simplicity. That is the trade-off in one sentence: code2vec answers what is this, code2seq answers what does this do, and it pays for the richer answer with a heavier training run.

The architecture is a TensorFlow network, and the repository includes a diagram of it. The important design decision for anyone extending it is stated in the README: the TensorFlow network is agnostic to the input programming language. The language-specific part lives entirely in the preprocessing extractors, so the training and inference code does not change when you add a language. The project says it can be easily extended to other languages for that reason, and contributions are welcome.

Three extractors, one network

The repository top-level entries show the shape of the implementation. There is a JavaExtractor directory, a CSharpExtractor directory and a Python150kExtractor directory, one per language with its own parser front end. The Java one requires a JDK, because any operation that parses a new code example, which includes creating a new dataset or manually examining a trained model, needs it. The C# path requires dotnet-core version 2.2 or newer. The Python one is named for the Python150k dataset rather than for a language target in general. Alongside them sit the shared pieces: extractor.py, the common entry point for turning source into the structural representation, preprocess.py, the code that builds the dataset files, reader.py for reading those files, model.py for the network, code2seq.py as the command line entry point, common.py for shared helpers, config.py for the hyper-parameters, and interactive_predict.py for the interactive session.

The split between extractor and model is what makes the language independence real rather than aspirational. An extractor parses a function into the structural representation the network expects, and everything after that point is shared. A baseline_tokenization directory sits alongside, which is the comparison point for evaluation, and train_python150k.sh alongside train.sh shows that the training entry point is a thin shell script that selects a dataset rather than a monolith.

Getting the preprocessed Java dataset

The README gives two routes to a dataset, and the first is to download the one the authors used. The java-large dataset holds about 16 million examples, compressed to 11G and expanding to 125GB when extracted, and the download is a plain wget plus tar:

mkdir data
cd data
wget https://s3.amazonaws.com/code2seq/datasets/java-large-preprocessed.tar.gz
tar -xvzf java-large-preprocessed.tar.gz

That creates a data/java-large sub-directory holding the training, test and validation files, along with a dict file for various dataset properties. The size is the first thing to plan for: this is not a dataset you fetch on a laptop with a spinning disk and 200GB free, it is an operation with a disk budget attached, and the compressed form is 11G on its own.

The alternative is to build your own from Java source files, which is what you would do to compare the model against another system on a different corpus. You edit preprocess.sh using the instructions in that file, pointing it at the right training, validation and test directories, and then run it:

> bash preprocess.sh

A separate preprocess_csharp.sh script exists for the C# path, following the same shape.

Training, and the model you can skip training

For most people the interesting path is the pretrained one. The authors trained a model for 52 epochs on the java-large data, and the README states plainly that this model is the same one used in the paper and the same one that serves the demo at code2seq.org. It downloads as a 137MB archive:

wget https://s3.amazonaws.com/code2seq/model/java-large/java-large-model.tar.gz
tar -xvzf java-large-model.tar.gz

One detail matters if you plan to resume from it. The trained model is in a released state, which the README explains means the training parameters were stripped from it, so it is good for inference and prediction but is not a checkpoint you continue training from.

Training from scratch is three steps. Edit train.sh to point at the right preprocessed data, by default the java-large dataset from the previous step. Edit config.py to set the hyper-parameters, which the README explains in the Configuration section. Then run the script:

bash train.sh

Training stops on its own after config.PATIENCE iterations without improvement on the validation set, which is an early stopping criterion on validation loss rather than a fixed epoch count. The 52-epoch figure for the released model is therefore a consequence of that criterion rather than a hard-coded schedule.

Evaluating and interrogating the model

Evaluation runs the same entry point with a model file and a test set. Given that iteration 52 is the chosen model:

python3 code2seq.py --load models/java-large-model/model_iter52.release --test data/java-large/java-large.test.c2s

While evaluating, a file named log.txt is written to the same directory as the saved models, with each test example name and the model's prediction, which is the artefact you need when you want to read through failures rather than look at a single aggregate score. For scoring against references, the README lists pip install rouge among the requirements, for computing rouge scores.

Manual examination uses the predict mode:

python3 code2seq.py --load models/java-large-model/model_iter52.release --predict

After the model loads, you edit the file Input.java and enter a Java method or code snippet, and you then examine the model's predictions and attention scores. The attention scores are the interesting part for anyone who wants to know why the model produced a given summary, since they show which parts of the input tree the decoder attended to.

There is a documented limitation in this path. Due to TensorFlow's limitations, if you use beam search, with config.BEAM_WIDTH above zero, then BEAM_WIDTH hypotheses will be printed but without attention weights. Without beam search, a single hypothesis is printed with its weights. In other words, the more thorough search you enable, the less interpretability you get, which is a real trade-off for a paper whose subject is interpretable structure-based generation.

The dependency that decides whether this runs at all

The requirements section is short and one entry dominates it: TensorFlow 1.12. The README gives a one-line version check with python3 -c, which is the fastest way to see whether your environment is even a candidate. TensorFlow 1.12 is a 2019 release, and the current TensorFlow line is several major versions later, so an environment with a modern TensorFlow will not satisfy this. Python 3 is the other requirement, and the README links a guide to installing it. The remaining entries are conditional: the JDK for the operations that parse new code, dotnet-core 2.2 or newer for a C# dataset, and the rouge package for scoring.

There is also a fork to be aware of. For a TensorFlow 2.1 implementation by Kolkir, the README points at a separate repository, and that is the practical answer for anyone who needs to run this on a current stack, at the cost of running code that is not the official implementation. The README's own framing, that it is a TensorFlow implementation of the network, is the honest description of what this repository is: a research artifact kept alongside its paper.

Where this model is the wrong tool

Three limits are structural. The first is the dependency age, and it is the one that ends the discussion for most production users: a model that requires TensorFlow 1.12 cannot be dropped into a current service, and the environment that runs it will need its own isolation. The second is the training cost, since the java-large dataset is 125GB extracted and training ran for 52 epochs, so the path from paper to working model is a machine with serious disk and compute, and the reasonable route for most people is the 137MB released model rather than training anything. The third is that code2seq has been overtaken by later work in the same research line, and the README says so itself. It points to Structural Language Models for Code from ICML 2020, which learns to generate missing code within a larger snippet, describes this as similar to code completion but able to predict complex expressions rather than a single token at a time, and has a demo of its own. The README also points to Adversarial Examples for Models of Code, which shows how slight mutations of an input snippet can make a model output a prediction of the attacker's choice, which is a warning worth reading before deploying any of these models on untrusted code.

The practical reading is that code2seq is a paper implementation to study, reproduce and extend, and a reasonable component for a summarisation dataset, not a summariser you would ship in 2026.

Licence, provenance and where to look

The repository is MIT licensed, with a LICENSE file at the root, and the CITATION.cff file points at the paper to cite. The project is published by tech-srl and the homepage is code2seq.org, which hosts the online demo alongside the paper PDF on OpenReview and the ICLR poster. The last push to the repository was on 2026-06-30, and there are no GitHub releases, so the tree is being worked on but nothing is versioned for download, which means anyone using this should pin a commit hash rather than a tag.

The README also points to two papers whose code was marked as soon, Adversarial Examples for Models of Code and Neural Reverse Engineering of Stripped Binaries, both of which build on code2seq and code2vec. That list is a good indication of the shape of the research group and of where the model was expected to be applied next, and it is also where a reader looking for the successor to this architecture should start.

Editorial conclusion

code2seq suits a researcher who wants the paper's model running rather than described, since the trained model, the preprocessing pipeline and the extractors are all in the repository and the Java-large data is downloadable. It is the wrong tool for production use, because the network needs TensorFlow 1.12, which is long past its end of life, and the wrong tool for a modern code summariser, since a transformer-based approach such as the Structural Language Models paper the README points to supersedes it. Verify first that you can run TensorFlow 1.12 in your environment, that you have the disk for the 125GB extracted java-large dataset, and note that the repository has no GitHub releases and the last push was on 2026-06-30.

Frequently asked questions

Official sources

1. Issues
2. License: MIT
3. Project website
4. README
5. tech-srl/code2seq on GitHub