SpeculativeDecodingPapers: a tagged reading list for speculative decoding

A paper list with a taxonomy, not just a pile of links

SpeculativeDecodingPapers contains a regularly updated paper list for speculative decoding, and its organising idea is that the entries are tagged rather than just accumulated. The README explains a keywords convention with four badge types. There is a SpecDec abbreviation badge, a badge for the conference or venue such as ACL2022, and a badge for the drafting method in speculative decoding, which is the category that actually tells you how a method works. The drafting badges seen in the entries are things like Self-Draft with a specialized FFN heads, Drafter with a smaller LM, Drafter with a specialized Non-Auto LM, and Self-Draft with jacobi decoding and with early-exiting. There is also a Main Features badge carrying names such as Blockwise Decoding, BiLD, SPEED and SpecDec.

So the list is not chronological. It is a set of orthogonal axes, which is the right way to organise a literature where two papers can differ in a dozen ways that matter simultaneously: what drafts, how the draft is produced, what the verification step looks like, and what the deployment target is.

The taxonomy is the repository's main contribution. The paper list itself is the visible part, and the classification is the part that lets you navigate it. The top level of the README has a Survey section with five surveys, then topic sections for Speculative Decoding for Seq2Seq, Speculative Decoding for LLMs, Multi-Token Prediction, Speculative Decoding for Diffusion LMs, Multimodal Speculative Decoding, Long-Context Speculative Decoding, Speculative Decoding for Mixture-of-Experts, Alignment, Benchmarks, Applications, Analysis, and Other Techniques, followed by a Blogs section and the contribution instructions.

That set of sections is itself an argument about the field. Multimodal, long context, mixture of experts, diffusion language models and alignment as separate sections tells you that speculative decoding has already spread out of the original text generation setting into most of the current inference landscape.

The survey section and the survey that anchors it

The Survey section lists five papers, which is unusual for a list this size and suggests the maintainers think the surveys are the right entry point. The first is Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding, by Heming Xia and colleagues from the Hong Kong Polytechnic University, Peking University, Microsoft Research Asia and Alibaba Group, published at ACL 2024 Findings, and it is the paper that carries the repository's own citation requirement.

The other four are Beyond the Speculative Game: A Survey of Speculative Execution in Large Language Models from April 2024, Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding from November 2024, Speculative Decoding and Beyond: An In-Depth Review of Techniques from February 2025, and A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Models from August 2025.

The dates matter for reading order. The list spans surveys published roughly eighteen months apart, and the progression in their titles is itself a summary of the field's movement: from speculative execution in large language models, to speculative decoding specifically, to techniques beyond it, and finally to parallel text generation as a category that includes diffusion language models. The repository also carries a timeline image in the assets directory, which is a good sign for a project whose subject is a chronology.

The survey's code link points at Spec-Bench, a separate repository, and the README's citation section asks readers to cite the survey. That pattern, a survey with its own benchmark repository, is the clearest sign that this list is attached to a research programme rather than being a hobby collection.

The Seq2Seq section and where the field actually started

The Speculative Decoding for Seq2Seq section is the oldest part of the literature and it is where the reading order has to start if you want to understand the mechanism. Its first entry is Blockwise Parallel Decoding for Deep Autoregressive Models by Stern, Shazeer and Uszkoreit from NIPS 2018, tagged as self-draft with specialized FFN heads and blockwise decoding.

The second is the paper that gave the technique its name, Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation, by Heming Xia, Tao Ge, Peiyi Wang and others, published at EMNLP 2023 Findings, with the code linked to the SpecDec repository, and tagged as a drafter with a specialized non-autoregressive language model.

After that come the variants that matter for understanding design space. Speculative Decoding with Big Little Decoder from NIPS 2023 uses a smaller language model as drafter. Accelerating Transformer Inference for Translation via Parallel Decoding from ACL 2023 is a self-draft using Jacobi decoding, which is the approach where the model drafts its own continuation without a separate model. SPEED, Speculative Pipelined Execution for Efficient Decoding, from ENLSP at NIPS 2023, is a self-draft with early exiting, and it is followed by a fast and robust early-exiting framework for autoregressive language models with synchronized parallel decoding.

The pattern to notice is that by 2023 the drafter is no longer assumed to be a separate small model. The 2018 blockwise paper already used the model's own feed-forward heads, and the 2023 work develops that into Jacobi decoding and early exit. The Big Little Decoder and the specialized non-autoregressive LM represent the other branch, where a second model does the drafting. Any reading list that mixes these without distinguishing them is not much help, which is what the drafting badges are for.

Tutorials, the paper list's own blog, and how to contribute

There is a Tutorial section above the papers, and it holds one entry: Speculative Decoding for Efficient LLM Inference, by Heming Xia, Cunxiao Du, Yongqi Li, Qian Liu and Wenjie Li, from January 2025, with a homepage, slides, a video, and a badge marking it as a tutorial on speculative decoding at COLING 2025. Having a tutorial that is separately linked and dated is a strong signal for a newcomer, because the papers assume the vocabulary that the tutorial supplies.

The Blogs section, linked as Blog and Project in the table of contents, collects non-paper write-ups and project posts, and the repository's title is a survey title, which means the author is the first author of the survey paper. The author list runs to nine people across four institutions, and the repository is the artefact of that survey work, which explains the level of taxonomic care in the tags.

The contribution section is part of the README too. It has a Contributors subsection and a Contributing to this paper list subsection, and a citation section at the end. A survey paper's repository with contribution guidance and a citation request is a working bibliography with an editorial process, which is rarer and more useful than a personal bookmark list.

The repository itself is minimal in the way a curated list should be. The top level contains a LICENSE, a README.md and an assets/ directory holding the logo and the timeline image. There is no code, no configuration and no build, and the primary language is not determined because there is no program. The licence is Apache-2.0, and the repository carries an Awesome badge, which is the listing standard this kind of resource is usually published under.

Where this list is the wrong tool

Three cases argue against it. The first is a practitioner who wants a speedup. Nothing here will make your serving stack faster on its own, and the entries that have code links point at separate research repositories, each of which will have its own hardware assumptions, its own batching requirements and its own integration work. A list of papers is a map of what has been tried, not a dependency you can add.

The second case is a reader who wants a linear narrative. The taxonomy is organised by technique, which is right for research and awkward for learning. Someone who has never read a speculative decoding paper will probably want the tutorial first, then the survey, then a specific family of methods, and the table of contents does not impose that order, so someone reading top to bottom will land in Blockwise Parallel Decoding without the context that makes the FFN head trick legible.

The third case is a reader whose question is not speculative decoding. The list has an Alignment section and a Benchmarks section and an Applications section, and it reaches into multimodal, diffusion language models, mixture of experts and long context, but it is a list about one technique. If your problem is serving throughput in general, or serving a mixture of experts model specifically, this list answers only the speculative decoding part of it.

There is also the ordinary maintenance risk of a curated list. A paper list tracks what has been published, and the taxonomy encodes a view of the field at a moment in time, so a category that seems complete today may not cover the method that appears next quarter. The list itself is the only artefact, so its accuracy depends on the maintainers continuing to read the new work, and the repository has no GitHub releases and no versioned schema for its tags.

On the practical side, the last push was on 2026-06-27, and the newest survey in the list is from August 2025, so the most recent additions predate the last push by several months. If you are entering the field now, the survey papers are the fastest way in, and the section on Multi-Token Prediction and the one on Speculative Decoding for Diffusion LMs are the two most likely to have moved on since.

Editorial conclusion

SpeculativeDecodingPapers suits a researcher who wants a structured map of the speculative decoding literature rather than a search query, and who will follow the linked code repositories to actually try the methods. It does not suit a practitioner looking for a drop in speedup or a maintained implementation, because the repository contains no code of its own beyond links, and it does not suit someone who needs coverage of a production serving stack. Before you rely on it, check the entry categories against your own question, since the list is organised by technique rather than by system, and note that the last push was on 2026-06-27.

Frequently asked questions

Official sources

1. hemingkx/SpeculativeDecodingPapers on GitHub
2. Issues
3. License: Apache-2.0
4. README