External-Attention-pytorch: 37 attention blocks, no training harness
🍀 Pytorch implementation of various Attention Mechanisms, MLP, Re-parameter, Convolution, which is helpful to further understand papers.⭐⭐⭐
At a glance
- What is it?
- FightingCV's External-Attention-pytorch collects 37 attention modules, 25 backbones, 6 MLP designs and 3 re-parameter blocks as standalone PyTorch classes. It is a reading library and a copy source, not a pipeline: no training loop or benchmarks in the usage notes, no releases, and an install path whose name disagrees with its own packaging metadata.
- Who is it for?
- Use it as a reading library and a source of copyable blocks; do not treat it as evidence that any mechanism works.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Activity is slowing. The repository last received commits 8 months ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on November 13, 2026, and from our analysis. They are not legal advice.
Star history
- xmu-xiaoma666/External-Attention-pytorch12,185 ★ · +2
| Repository | 2026-09-22 | 2026-11-13 |
|---|---|---|
| xmu-xiaoma666/External-Attention-pytorch | 12.2k | 12.2k |
2 Stars in the last 52 days, measured from the daily GitHub snapshot. Compare with other projects
Editorial analysis
The git path and the pip path differ by one word in the import
Two install routes are offered, and the gap between them is a single token. The pip route is one line:
pip install fightingcv-attentionThe git route copies the tree instead:
git clone https://github.com/xmu-xiaoma666/External-Attention-pytorch.git
cd External-Attention-pytorchAfter that the two routes part only at the import statement, where fightingcv_attention becomes model. Everything else in the demo is identical, down to the same constructor argument and the same input tensor. The side effect is that two module paths can be live in one environment at the same time, so a fix you make in the cloned copy never reaches the installed copy, and a stale installed copy can shadow what you are editing. A third file, README_pip.md, is the usage reference for the modules built into the pip package. Neither route pins a commit, so what lands on your disk is whatever master holds at the instant you run the command.
setup.py names the distribution fighingcv, not fightingcv-attention
The packaging metadata disagrees with the install line. setup.py sets the distribution name to fighingcv, with the two halves of the intended word transposed, while the usage notes tell you to install fightingcv-attention. The two files never mention each other's name, so which distribution a public index actually serves cannot be confirmed from this repository. The same file sets the version to 1.0.0, and there are no GitHub releases for it to attach to. setup.py also declares license="Apache" and classifies the package under the Apache Software License, while the repository metadata reports MIT and a LICENSE file sits in the root. The consequence for a reader is that the import you can verify yourself is the git one; the pip name is something you have to take on trust, and if you intend to ship a model that embeds one of these blocks you have to resolve the license mismatch before you can say which terms the code travels under.
install_requires is commented out, so torch arrives on its own
The dependency line in setup.py is commented out, so the install declares no requirements whatsoever. The usage snippets open with import torch, from torch import nn and from torch.nn import functional as F, and nothing in the metadata arranges for any of them to be present. Follow the pip route into a clean virtual environment and the first import that fails is the one you typed yourself, not the one you asked pip to fetch. The only environment constraint in setup.py is python_requires set to >=3.7.0, with no upper bound and no pin on any torch release. The consequence is that a working setup here means you already had a working torch, and the repository contributes nothing to the part of installation that breaks first on a fresh machine. The same holds for numpy and for whatever helper libraries individual module files reach for. Note also that packages=find_packages(".") with package_dir pointing at the root means the whole model tree is collected, so what you get installed depends on how find_packages walks the current checkout.
The index is a catalog of 37 attention blocks, not a pipeline
The tree is organized as series inside model/: attention, backbone, MLP, re-parameter, and a convolution section. The attention index runs to 37 entries, from External Attention at number one through GFNet, Coordinate attention, MobileViT, UFO, DAT, CrossFormer, MOATransformer, CrissCrossAttention and Axial attention at number 37, and each entry receives its own usage snippet. The backbone index names 25 designs from ResNet to ConvNeXtV2, the MLP series names six including MLP-Mixer, ResMLP, gMLP and sMLP, and the re-parameter series holds RepVGG, ACNet and Diverse Branch Block. The index lists modules with a usage snippet for each, and does not itself describe training or evaluation code. Treat the result as blocks you wire into a loop you already own, not something you launch and watch converge.
Every snippet ends in import *, so the file's namespace is the API
The demo, copied exactly as published, is:
import torch
from torch import nn
from torch.nn import functional as F
# 使用 pip 方式
from fightingcv_attention.attention.MobileViTv2Attention import *
if __name__ == '__main__':
input=torch.randn(50,49,512)
sa = MobileViTv2Attention(d_model=512)
output=sa(input)
print(output.shape)The wildcard is the documented way in, and it is also the sharp edge. Every name the module file binds at its own top level, including its own copies of torch, nn and F, lands in your namespace after the star import and replaces the binding you made moments earlier. The set of names a star import hands you is something you learn by opening the module source. The consequences show up in a long research session: two modules that happen to define a similarly named class clobber each other, and the class you instantiate may not be the one on disk. Replacing the star with explicit imports costs nothing and is what the same snippet would need in any shared codebase.
The demo prints a shape that no file records
The whole verification story fits in four lines: build a random tensor of shape 50 by 49 by 512, construct MobileViTv2Attention with d_model=512, pass the tensor through it, print output.shape. The demo does not state the resulting shape, so there is nothing to check your run against without executing the snippet yourself. The stated purpose of the codebase is to help you understand papers from the code side, and for that goal a faithful reimplementation of the arithmetic is the point rather than a leaderboard. The consequence for a reader is narrow and worth stating plainly: this tree can tell you what a block does to a tensor, and cannot tell you whether the block is worth putting in a model. When accuracy decides your architecture, the answer has to come from the paper or from your own training run.
master moves, 1.0.0 does not, and the default page opens in Chinese
The default branch is master and the most recent push is dated 2026-03-16. The repository is not archived, but it also has no GitHub releases, so the version string in setup.py has no release behind it and nothing ties a commit to a revision of a paper. The default page opens in Chinese and links to README_EN.md, and the pip route points to a third file, README_pip.md. The promotional material is inert: the author's introduction, the WeChat account and the group invitation all sit inside HTML comment markers and do not render on the project page, and the invitation asks readers to identify themselves with company, school, direction and ID. What survives is a sense of who maintains the tree, not a maintenance commitment, and the root carries no issue template, no contribution guide and no support statement. It contains FightingCVimg/, LICENSE, three README files, main.py, model/ and setup.py.
Editorial conclusion
Use it as a reading library and a source of copyable blocks; do not treat it as evidence that any mechanism works. Before you depend on it, confirm three things: that the class you instantiated is the class on disk, since every snippet ends in a wildcard import; that the distribution name and license match what you assumed, because setup.py says fighingcv and Apache while the repository reports MIT; and that the code you copied is the code behind any number you plan to quote, since the last push is dated 2026-03-16 and there are no releases to pin. Teams with a working training harness and a habit of reading source get the most out of it. Teams looking for a reproducible training recipe will not find one in this tree.
Frequently asked questions
What is the difference between internal and external attention?
The repository ships both as code rather than as definitions. The attention index opens with External Attention, then Self Attention, then Simplified Self Attention, and gives each its own usage snippet, so the difference can be read directly out of the two implementations.
What is external attentional focus?
The name used in the index is External Attention, entry one of the attention series, one of 37 attention blocks in total. The README demo shows the usage pattern with MobileViTv2Attention, d_model=512 and a 50 by 49 by 512 tensor.
Why is attention used in neural networks?
The stated purpose of the codebase is to implement attention mechanisms in PyTorch so that papers become readable from the code side, which is why 37 attention blocks sit next to 25 backbones, 6 MLP designs and 3 re-parameter blocks. The case for any single block rests on the paper that introduced it.
How does attention actually work?
By reading the implementation. Each of the 37 entries in the attention index names a class and shows a snippet that instantiates it and passes a tensor through it, from External Attention through MobileViTv2, DAT, CrossFormer and CrissCrossAttention to Axial attention.
Official sources
Add this badge to your README
If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.
[](https://hysenlabs.com/projects/xmu-xiaoma666-external-attention-pytorch)Star history chart for your README
[](https://hysenlabs.com/projects/xmu-xiaoma666-external-attention-pytorch)Was this useful?