harleyszhang/llm_note: a Markdown reading list for LLM inference and GPU kernel work

What llm_note actually is: a curated index, not a framework

The repository describes itself as "LLM notes, including model inference, hpc programming note, transformer model structure, and vllm framework code analysis notes." That sentence is the whole product. The README is a table of contents in Chinese, pointing at Markdown files stored in numbered directories such as 1-transformer_model, 2-llm_compression, 3-llm_infer, 4-hpc_basic and 5-framework. There is no package to install, no CLI, no Python entry point exposed in the material, and no release has been retrieved. The primary language is listed as Python, which most likely reflects code blocks inside the notes rather than a library surface.

The audience is narrow and clearly stated. The topics list cuda-programming, kv-cache, llm-inference, transformer-models, triton-kernels and vllm. The reading path runs from transformer and llama1-3 architecture write-ups, through SmoothQuant and AWQ quantization, into FlashAttention 1, 2 and 3, prompt caching, CUDA graphs, tensor parallelism, and finally framework-level analyses of TGI, vLLM and LightLLM. If you already write PyTorch training code and want to understand why an inference server behaves the way it does, that ordering is sensible. If you want a library that serves a model, this is the wrong repository.

The paid course sits between the notes and the runnable framework

The first section of the README is not a note. It is a course advertisement for a self-built LLM inference framework, priced at 499, described as built on Triton plus PyTorch with a PyTorch-like syntax for writing GPU kernels. The claims listed there include support for FlashAttention V1, V2 and V3, GQA, PageAttention, fused operators such as KV linear layer fusion, and adaptation to qwen3, qwen2.5, llama3 and llava1.5. The README also states a speedup of up to 4x versus the transformers library on llama3 1B and 3B models, and shows GIFs of streaming output for llama3.2-1.5B-Instruct, Qwen2.5-3B and Llava1.5-7b-hf.

Treat that whole block as vendor material. The speedup figure is the author's claim about a course project, not a benchmark you can reproduce from this repository, and no methodology, hardware or batch configuration is given. The framework code itself is not in this repository as far as the supplied material shows. So the correct mental model is: llm_note is the free notes layer, the course is the paid implementation layer, and the README deliberately blurs the two by putting the sales section first.

The mechanism is a directory taxonomy, and it is the useful part

There is no runtime architecture to describe. The organising mechanism is the numbered folder scheme, and it does real work because it separates concerns that are usually mixed together in LLM reading lists. Folder 1 is model structure: transformer paper notes, a transformer implementation walkthrough, llama1-3 structure, ViT, GPT1-3, sinusoidal position encoding, and an MLA implementation and optimisation note. Folder 2 is compression, split into paper reading and source-code analysis for SmoothQuant and AWQ. Folder 3 is inference optimisation, subdivided into performance analysis (translations of LLM inference survey papers and a service framework comparison), algorithm-level work (online softmax, the three FlashAttention papers plus a combined summary, prompt cache, vLLM CUDA graph), and parallelism (tensor parallelism). Folder 4 is high-performance computing, with a Triton series numbered 0 through 5 and a large CUDA block. Folder 5 is framework source analysis for TGI, vLLM and LightLLM.

One defect is visible in the README itself. The AWQ paper entry links to the same SmoothQuant paper file, and the vLLM optimisation entry is written with a doubled path prefix, llm_note/5-framework/..., which suggests a copy-paste error. Those two links are worth checking before you trust the index.

Getting to the content: clone, read, and check the links first

There is no install step in the supplied material. The only commands implied by the repository are the standard ones:

git clone https://github.com/HarleysZhang/llm_note cd llm_note

From there you read Markdown files directly, for example 1-transformer_model/llama1-3模型结构详解.md or 3-llm_infer/fast_algorithm/flashattention-2论文解读.md. The README gives no virtual environment, no requirements.txt, no Makefile, and no test command. That is consistent with a notes repository, but it means you cannot verify any of the described kernels by running them. If you want to execute Triton kernels, the README's own reference list points elsewhere: kernl, unsloth and Liger-Kernel are cited as sources of Triton kernel implementations, with unsloth described as rewriting all its kernels in OpenAI Triton.

The one configuration-shaped detail in the material is the recommended reading order in section 4.3, which names GPU Architecture and Programming as the first document, the CUDA Tutorial as the systematic course, learn-cuda for advanced asynchronous material, and the CUDA C++ Programming Guide (version 12.6) as a reference to consult rather than read start to finish. The author is unusually blunt about two books: the Chinese translation of the CUDA C Programming Guide is called out as poorly translated and outdated, and the domestic CUDA programming text by 樊哲勇 is described as verbose but usable for Chinese readers.

Where the repository stops being the right tool

The first limitation is provenance. The material lists no licence, no CONTRIBUTING file, and no citation guidance. Notes that quote or closely paraphrase papers and framework source sit in an ambiguous position, and the repository gives you nothing to resolve that. If you need to reuse the text or the code blocks in a commercial setting, the absence of a licence is a blocking problem, not a detail.

The second is freshness. Framework analyses of vLLM, TGI and LightLLM age faster than the underlying papers. The vLLM CUDA graph note and the vLLM inference flow walkthrough describe internals that have moved repeatedly. The last push recorded is 2026-08-19, which tells you the repository is alive but not which files were touched. Nothing in the material gives per-file dates, so you cannot tell whether a given framework note reflects the version you deploy.

The third is that it is a reading aid, not an answer. A note on online softmax or on the Roofline model explains a concept; it does not tell you whether your kernel is memory-bound on your hardware. For that you need the profiler, and the repository's own CUDA performance analysis note points in that direction without providing a tool.

Compared with DefTruth/CUDA-Learn-Notes and the framework docs themselves

The README's own reference list gives the honest comparison. DefTruth/CUDA-Learn-Notes is described as CUDA kernel programming notes with practical code, and the author says he intends to study it seriously. The difference in approach is concrete: that project pairs notes with runnable kernels, while llm_note pairs notes with links to other people's kernels. If your goal is to write a fused kernel, starting from a repository where the kernel is in the tree and can be compiled is a shorter path.

The second alternative is the primary documentation. For vLLM internals, the project's own docs and source are the reference; llm_note's value is the Chinese-language walkthrough that tells you which parts to read first. For FlashAttention, the papers are the source and the notes are the summary. Neither alternative replaces the other. A notes repository earns its place by compressing the search, and this one does that with a taxonomy that is genuinely more organised than most topic lists.

Maintenance, licence and what to verify before you depend on it

Maintenance cost here is low in one direction and high in another. Cloning and reading costs nothing and there is no dependency graph to break. But the repository is a moving target with no changelog and no releases, so any link you build into your own internal documentation can rot silently. The README's own broken links (the AWQ entry pointing at the SmoothQuant file, the doubled llm_note/5-framework path) are evidence that link integrity is not enforced.

On licensing, the material states nothing. There is no licence identifier in the supplied facts, and the README does not mention one. Do not assume permissive reuse of the prose or the embedded code. If you plan to mirror the notes internally or quote them at length, check the repository for a LICENSE file first, and if none exists, treat the content as all rights reserved until the author says otherwise. That is a factual observation about the material, not legal advice.

The notes are also written in Chinese, which the README never states explicitly. English-speaking readers will need translation, and the paper translations inside the repository are Chinese translations of English papers, so you are reading a translation of a translation for some entries.

Editorial conclusion

Adopt it as a reading list if you are an engineer moving from PyTorch-level LLM work toward kernel-level inference and want a curated path through FlashAttention, Triton, CUDA and vLLM internals. Do not adopt it if you need runnable code with a stated licence, or if you are looking for a maintained framework: the README points to a separate paid course for the actual inference framework, and no licence file is referenced anywhere in the supplied material. Before relying on any file in it, open the repository and confirm three things yourself: whether a LICENSE file exists, whether the linked note paths resolve, and whether the vLLM and TGI analyses still match the versions you run.

Official sources

1. harleyszhang/llm_note on GitHub
2. Issues
3. README