mryab/efficient-dl-systems: a ten-week course repo for GPU and distributed training work
Efficient Deep Learning Systems course materials
At a glance
- What is it?
- The repository is the 2026 iteration of an HSE University and Yandex School of Data Analysis course on efficient deep learning. It is teaching material organised by week, not a library, and the README frames it as a syllabus with seminars and home assignments rather than a supported tool.
- Who is it for?
- Adopt this if you want a structured ten-week path through CUDA, mixed precision, All-Reduce, FSDP2, torch.compile and inference serving, and you are willing to supply your own GPU hardware and environment. Do not adopt it if you need a maintained library, versioned releases or a support channel, because the README describes a course rather than a product.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 110 days ago.
- What is it written in?
- Mainly Jupyter Notebook, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem this repository addresses
Most PyTorch material teaches you to build a model. It rarely teaches you why the same model runs at different speeds on the same GPU, or how a training loop behaves once it spans several devices. The README describes a course built around that second question. Week 1 covers GPU architecture and the CUDA API alongside benchmarking. Week 2 moves to profiling with py-spy, PyTorch Profiler, Memory Snapshot and Nsight Systems. By week 3 the subject is data-parallel training and All-Reduce, and by week 5 it is FSDP and its optimizations. The intended reader is someone who already writes PyTorch and now needs the systems layer underneath it, which in practice means an ML engineer moving from notebooks to cluster jobs, or a student preparing for infrastructure work. The grading section lists home assignments on training pipelines and code profiling, distributed and memory-efficient training, and deploying and optimizing models for production, so the repository assumes you will write code, not just read slides.
How the materials are laid out and how the weeks build
The structure is a flat set of directories named week01_intro through week09_inference_algorithms, plus a week 10 guest lecture with no directory listed in the syllabus. Each week splits into a lecture and a seminar, and the README gives a one-line summary of each. The progression is deliberate. Week 4 introduces tensor, pipeline and sequence parallelism plus gradient checkpointing and offloading, then week 5 goes deep on PyTorch Device Mesh, DTensor, FSDP2 and Distributed Checkpoint. Week 6 shifts from distributed mechanics to arithmetic: kernel fusion, torch.compile, the GPU memory hierarchy and Liger kernels. Weeks 7 to 9 turn to serving, covering web service deployment, KV caching, continuous batching, FlashAttention variants and PagedAttention, then quantization, speculative decoding and KV cache compression. The primary language is Jupyter Notebook, which matches a seminar format where you run cells and observe timings. There is no package to install from this repository and no importable module documented in the README, so treat the notebooks as executable notes you adapt rather than as a dependency.
What you need before the first notebook runs
The README does not list installation steps, dependencies, a requirements file or a setup command, so anything beyond cloning the repository is unverified here. What the syllabus does tell you is the tool surface you will need: a CUDA-capable GPU for the week 1 and week 6 material, PyTorch with its distributed package for weeks 3 and 5, and profiling tools named explicitly in the week 2 and week 9 descriptions, namely py-spy, PyTorch Profiler, Memory Snapshot, Nsight Systems and nsys profile. Week 8 requires enough memory to hold a KV cache and to experiment with continuous batching, which in practice means a serving-capable GPU rather than a laptop integrated chip. The repository is on the main branch for 2026, and the README points to separate branches for 2025, 2024, 2023 and 2022, with 2021 living in a different repository, yandexdataschool/dlatscale_draft. If a notebook fails on your machine, the first thing to check is whether you are on the branch matching your PyTorch version, because the 2026 branch will track newer APIs such as FSDP2 that older releases do not provide.
Where the course format limits you
This is teaching material, and the README never claims otherwise. There are no releases, no changelog, no issue triage commitment and no API stability promise. Notebooks that demonstrate a technique are not the same as a reference implementation you can vendor, and the grading section makes clear that assignments are graded through the institution rather than through the repository. A second limitation is coverage shape: the syllabus is broad by design, one week each for FSDP, inference algorithms and deployment, so a topic like quantization gets a lecture slot rather than a deep implementation guide. If your problem is a specific production regression in a serving stack, ten weeks of survey material will not resolve it. The repository is also tied to a yearly teaching cycle. The 2026 branch is the current one, and the existence of five earlier branches suggests material is revised rather than frozen, so a notebook you rely on may be rewritten the following year.
How it compares with a framework like DeepSpeed
DeepSpeed is a library you install and call; it ships optimizers, ZeRO stages and inference kernels behind a configuration file, and its behaviour is versioned and documented as a product. This repository takes the opposite approach. It explains the mechanisms, such as All-Reduce, gradient checkpointing and sharded data parallelism, using PyTorch primitives and profiling tools, and leaves you to assemble them. That difference matters when you are debugging: with a library you read its source and release notes, while with this course you re-derive the concept and then decide which library to adopt. The two are complementary rather than competing. Someone who has worked through week 5 on DTensor and FSDP2 will read a DeepSpeed configuration with more context, and someone who only knows the library API will struggle to explain why a particular sharding choice is slow. If you want a drop-in training stack today, this is the wrong artefact. If you want to understand the stack well enough to choose between them, the week-by-week sequence is the point.
Licence, reuse and what maintenance looks like
The repository is MIT licensed, which permits reuse, modification and redistribution provided the copyright notice and permission notice are retained. That is permissive enough for internal training decks or adapted seminar notebooks, though the README does not state how attribution should be handled for course branding, and this is not legal advice. Maintenance is best read from the commit history rather than from releases, since none were retrieved. The last push is dated 2026-05-28, and the README states this branch corresponds to the 2026 iteration, which is consistent with a repository updated around a teaching calendar. There is no upgrade path to manage in the software sense: you track a branch, and when the next iteration lands you compare your local edits against the new notebooks. The practical cost is the time to re-check that the APIs your notes depend on, particularly FSDP2 and torch.compile behaviour, still match the PyTorch version you run.
Who should clone it, and what to confirm first
Clone it if you are comfortable with PyTorch and want a sequenced route through GPU performance work, from benchmarking and profiling in weeks 1 and 2 to distributed training, compilation and serving by week 9. It suits self-study engineers, internal training programmes and anyone preparing for ML infrastructure interviews, because the syllabus names the tools rather than gesturing at them. Skip it if you need a supported dependency, if you have no access to a CUDA GPU, or if your interest is narrow enough that one week of survey will not help. Before committing time, open the week directories on main and confirm the notebooks are present and runnable in your environment, check that the assignment topics align with what you actually need to build, and glance at the 2025 branch to see how much the material changed in a year. If the diff between 2025 and 2026 touches the exact weeks you plan to study, budget for re-reading them next cycle.
Editorial conclusion
Adopt this if you want a structured ten-week path through CUDA, mixed precision, All-Reduce, FSDP2, torch.compile and inference serving, and you are willing to supply your own GPU hardware and environment. Do not adopt it if you need a maintained library, versioned releases or a support channel, because the README describes a course rather than a product. Verify first that the week directories you care about exist on the main branch for 2026, that the assignment topics match your level, and that the past-version branches (2021 through 2025) do not already cover the material you need in a form closer to your toolchain.
Community notes