TextAttack: A Python Framework for Composing NLP Adversarial Attacks from Named Components
TextAttack 🐙 is a Python framework for adversarial attacks, data augmentation, and model training in NLP https://textattack.readthedocs.io/en/master/
At a glance
- What is it?
- TextAttack bundles attack recipes, transformations, constraints, search methods and goal functions into a single command-line and Python interface. It is useful when you want to reproduce a published attack by name, and awkward when you want an attack the recipe list does not cover.
- Who is it for?
- Adopt TextAttack if you need to reproduce a published attack by recipe name, run it against a HuggingFace-style classifier, or generate augmented CSV data without writing the perturbation search yourself.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 31 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem TextAttack addresses: attacks are pipelines, not scripts
An adversarial example for a text classifier is not a single algorithm. It is a transformation that proposes a candidate edit, a set of constraints that decide whether the edit is acceptable, a search method that decides which candidate to try next, and a goal function that decides whether the model's prediction has actually flipped. Published attacks differ in all four places. Alzantot's attack uses counter-fitted word embedding swaps, a genetic algorithm and a language model perplexity constraint. BAE uses BERT masked token prediction with a Greedy-WIR search and a USE sentence encoding similarity constraint. Writing each of these from scratch means reimplementing the same scaffolding.
TextAttack's answer is to name the four slots and let a recipe fill them in. The README describes the framework as covering adversarial attacks, data augmentation and model training, and the recipe table makes the decomposition explicit: each row lists a goal function, a set of constraints, a transformation and a search method. That table is the clearest statement of what the project actually is. It is a component library with a registry, not a collection of standalone scripts.
The audience follows from that. Researchers who need to compare a new attack against a published baseline benefit from running the baseline as a named recipe. Engineers who want to know whether a deployed classifier is brittle need the attack to run against their own model, which means the model wrapper matters as much as the recipe. People who only want more training data are served by the augment command, which is a different entry point into the same transformation machinery.
How a TextAttack run is assembled at runtime
The CLI exposes the four slots directly. The README gives this example:
textattack attack --model lstm-mr --num-examples 20 --search-method beam-search^beam_width=4 --transformation word-swap-embedding --constraints repeat stopword max-words-perturbed^max_num_words=2 embedding^min_cos_sim=0.8 part-of-speech --goal-function untargeted-classification
Three things are worth reading off that command. First, components are named by string identifiers and resolved at runtime, so the set of available transformations and constraints is a registry rather than a fixed API. Second, constraints are passed as a space-separated list and are combined, which means the search only accepts a candidate edit if every listed constraint passes. Third, parameters are attached to a component name with a caret, as in max-words-perturbed^max_num_words=2 and embedding^min_cos_sim=0.8. That syntax is the mechanism by which a generic component becomes a specific instance without editing code.
A recipe is a named preset over the same slots. textattack attack --recipe textfooler --model bert-base-uncased-mr --num-examples 100 selects the TextFooler configuration and runs it against a BERT model fine-tuned on MR. The recipe table in the README documents what each preset contains, which is what makes the presets auditable: you can see that a2t uses a Greedy-WIR gradient search while alzantot uses a genetic algorithm, and that both enforce a percentage-of-words-perturbed constraint but differ on the language model perplexity constraint.
The goal function is where the model's behaviour enters. untargeted-classification means the attack succeeds when the predicted label changes to anything other than the original. The table also lists Untargeted {Classification, Entailment} as a goal function family, so entailment tasks are handled by a different goal function rather than by a different attack loop. This is the design decision that makes the framework extensible: adding a task type means adding a goal function, not forking the search.
Installation, cache location and the TA_CACHE_DIR variable
Installation is a single pip command:
pip install textattack
The README states that Python 3.6 or later is required and that a CUDA-compatible GPU is optional but improves speed. Once installed, the framework is reachable either as the textattack command or as python -m textattack.
TextAttack downloads files to ~/.cache/textattack/ by default. The README lists what lands there: pretrained models, dataset samples, and a configuration file named config.yaml. The cache path is overridden with the TA_CACHE_DIR environment variable, and the README gives this form:
TA_CACHE_DIR=/tmp/ textattack attack ...
That variable is the one piece of configuration most likely to matter in practice. Attack runs pull pretrained models and dataset samples on first use, so on a shared machine or inside a container the default location may be read-only or may fill a small home partition. Setting TA_CACHE_DIR before the first run avoids both problems, and it also lets you pre-populate the cache in an image build step so that later runs do not need network access.
The config.yaml inside the cache directory is the other configuration surface the README mentions, but it does not document its keys. If you need to change behaviour beyond the cache location, the documented route is the command-line component syntax rather than that file.
Two further entry points are documented. textattack augment operates on a CSV file, and the README points at the examples/ folder for scripts covering training, attacking and augmentation. For parallel execution across multiple GPUs, the README documents a --parallel option on the attack command and notes that attacking Keras models in parallel requires examples/attack/attack_keras_parallel.py instead.
Where the recipe abstraction breaks down
The recipe list is the boundary of the framework's convenience. If the attack you want is in the table, you get it with one flag. If it is not, you are back to composing transformations, constraints, search methods and goal functions by hand, and the framework only helps to the extent that your attack decomposes into those four slots. An attack whose core idea is not expressible as a constrained search over token edits, for example one that operates on the input at a level the transformations do not cover, has no natural home in this design.
The constraint list is also a hard filter, not a soft penalty. In the beam search example, repeat, stopword, max-words-perturbed, embedding and part-of-speech are all enforced together, so a candidate edit must satisfy every one of them. Adding constraints shrinks the search space, and the README's own note that parallel execution helps performance for some attacks is a signal that these searches are not cheap. Nothing in the supplied material gives timing figures, so treat the cost as unknown until you measure it on your own model and dataset.
Target model support is the other boundary. The examples in the README use models named through the framework's own naming scheme (bert-base-uncased-mr, distilbert-base-uncased-cola, lstm-mr), and the parallel-attack caveat for Keras implies a separate code path for non-PyTorch models. If your classifier is served behind an HTTP API rather than loaded as a local model object, the material here does not describe an adapter for that case.
Finally, the release cadence is uneven. v0.3.10 landed in March 2024 and v0.3.11 in August 2026, roughly two and a half years apart, with v0.3.9 before that in September 2023. The version numbers stay in the 0.3.x line. A long gap between releases is not itself a defect, but it does mean that any dependency you pin against TextAttack should be checked against the version you actually install rather than against the current release date.
TextAttack against a plain augmentation library
The obvious alternative for the data-augmentation half of the project is a general-purpose text augmentation library such as nlpaug, which offers synonym replacement, back-translation, contextual word insertion and similar operators without any notion of a target model. The difference in approach is the goal function.
nlpaug-style augmentation is model-free. It takes a sentence and returns a perturbed sentence, and the only quality control is whatever constraints the operator itself imposes. TextAttack's augment command sits inside the same architecture as its attacks, so a transformation can be paired with constraints and, in the attack path, with a goal function that queries a model. That means augmentation can be steered by model behaviour: you can generate examples that are hard for a specific classifier rather than merely different from the original. The README lists augmented data as a way to increase model generalization and robustness downstream, which is the intended use.
The trade is setup cost. A model-free augmenter needs a sentence and a rule. TextAttack's model-aware path needs a model wrapper, a dataset, and a search budget, and it will download pretrained artifacts into ~/.cache/textattack/ on first run. If your goal is simply more paraphrases of a training set, the model-free library is the shorter path. If your goal is examples that expose a particular model's weaknesses, the goal function is the part you cannot get from a model-free augmenter, and that is the reason to accept the extra machinery.
The same comparison applies on the attack side. A hand-written attack script for one model and one dataset can be shorter than the equivalent TextAttack command, because it hardcodes the pieces TextAttack makes configurable. TextAttack pays off when you want to swap one of those pieces, or run several recipes against the same model for comparison, without rewriting the loop.
Licence, maintenance and what to check before adopting
TextAttack is MIT-licensed. That is a permissive licence, so incorporating it into a commercial pipeline and modifying it are both permitted subject to the licence terms, which require the copyright notice and permission notice to be included in copies or substantial portions. This is a description of the licence identifier in the repository metadata, not legal advice; if the framework ends up distributed with your product, have counsel review the actual LICENSE file and any third-party model or dataset terms that the downloads bring with them. The pretrained models and dataset samples that TextAttack fetches into its cache carry their own licences, and the MIT licence on the framework does not extend to them.
Maintenance cost has two parts. The framework itself is a pip dependency in the 0.3.x series, so upgrades are version bumps rather than migrations, but the release history shows long gaps, which means a bug you hit may sit unfixed for a while. The larger ongoing cost is the cache. Pretrained models and dataset samples accumulate in ~/.cache/textattack/ and are re-downloaded on any machine that does not have them, so CI images and shared workstations need either a persistent volume or a pre-populated TA_CACHE_DIR. That is a concrete piece of operational work, not a hypothetical one.
Before adopting, verify three things against your installed version. Run textattack list attack-recipes to confirm the recipe you need is present, since the README's table documents the set but the installed version is the authority. Check whether your model can be loaded through the framework's model naming, using the examples in the README as the template; if it cannot, the framework's attack path is closed to you even though the augmentation path may not be. And confirm the constraint set you intend to use is enforceable on your data, because constraints like part-of-speech consistency and language model perplexity assume the inputs are well-formed sentences rather than fragments, code or log lines.
Editorial conclusion
Adopt TextAttack if you need to reproduce a published attack by recipe name, run it against a HuggingFace-style classifier, or generate augmented CSV data without writing the perturbation search yourself. Do not adopt it if your target is not a text classifier or seq2seq model reachable through its model wrappers, or if you need an attack that is not in the recipe list and cannot be expressed as a combination of its existing transformations, constraints, search methods and goal functions. Before committing, run textattack list attack-recipes on your installed version, check that the recipe you need is present, and confirm which of its components you can substitute.
Community notes