AlphaGen: a reinforcement learning loop that writes stock factors as expressions

A research codebase, published with its paper

AlphaGen is the code for a paper titled Generating Synergistic Formulaic Alpha Collections via Reinforcement Learning, accepted at KDD 2023 in the Applied Data Science track, and the README states that some extensions on that work are also included in the repository.

That framing sets the expectations correctly. This is a research codebase released alongside a publication, with the paper's experiments reproducible from it, and the extensions layered on afterwards. The publication is on the ACM Digital Library with a DOI, and the repository also carries a citation entry for a second paper, HARLA, described as a hybrid approach to formulaic alpha discovery with large language model assistance.

The subject is formulaic alpha, which is the quant term for a factor expressed as a formula over market data. The goal of the paper is not one good factor but a collection whose members are synergistic, which is why the toolkit contains a way to compute the correlation between two factors and a way to combine a pool of them linearly and score the combination against a target.

The audience is a quant researcher or a student reproducing the work, and a second audience that the repository serves deliberately: anyone who wants the search machinery without the data pipeline. The repository layout is split into a core library, a qlib-specific data layer, a generic layer modelled on gplearn, an LLM layer, vendored baselines, and the scripts that run the experiments.

The AlphaCalculator interface is the seam worth copying

Before anything else, the repository asks you to choose between two ways to reproduce the results, and the second one contains the most reusable design in the project.

Choice 1 is the builtin alpha calculation pipeline, which requires the Qlib library and locally stored stock data. Choice 2 is to adapt to an external pipeline, on the assumption that you have better implementations of alpha calculation, by implementing an adapter of alphagen.data.calculator.AlphaCalculator.

The interface is a set of abstract methods, and what they say about the domain is more interesting than the signatures themselves:

class AlphaCalculator(metaclass=ABCMeta):
    @abstractmethod
    def calc_single_IC_ret(self, expr: Expression) -> float:
        'Calculate IC between a single alpha and a predefined target.'

    @abstractmethod
    def calc_single_rIC_ret(self, expr: Expression) -> float:
        'Calculate Rank IC between a single alpha and a predefined target.'

    @abstractmethod
    def calc_mutual_IC(self, expr1: Expression, expr2: Expression) -> float:
        'Calculate IC between two alphas.'

Three ideas are baked in. There is an information coefficient between an expression and a target, and a rank version of it, because a factor that correlates with returns but is dominated by a few extreme values is a different animal from one that correlates robustly. There is a mutual information coefficient between two factors, which is the synergy the paper is about. And there are pool methods that combine alphas linearly with weights and score the combination against the target, which is what turns a set of factors into a portfolio of them.

The interface also carries a warning that is easy to skip and expensive to ignore: the values evaluated from different alphas may have drastically different scales, so you should normalize them before combination.

The data comes from baostock, not from Qlib's own source

The data pipeline has a wrinkle that the README explains in a footnote, and it is the kind of decision that says a lot about how the authors thought about reproducibility.

The builtin pipeline requires the Qlib library and local-stored stock data. The first instruction is to follow Qlib's own data preparation process, because some of the metadata, but not the actual stock price and volume data, is needed from Qlib.

The actual stock data used comes from baostock, and the reason given is concerns on the timeliness and truthfulness of the data source used by Qlib. That is a researcher's objection, not a technical one: a factor mined on stale or adjusted data is a factor mined on a different dataset than the one you will trade.

The data is downloaded by running the script data_collection/fetch_baostock_data.py, and the newly downloaded data is saved into ~/.qlib/qlib_data/cn_data_baostock_fwdadj by default. The path can be customized, but you have to use the correct path when loading, and the README points at the exact place: in alphagen_qlib/stock_data.py, in the function StockData._init_qlib, the path should be passed to qlib with qlib.init(provider_uri=path).

The forward-adjusted suffix in the default directory name is worth noticing, since it says the series is adjusted for corporate actions, and the name of the data set therefore depends on that choice. Anyone using the tool on a different market, a different adjustment convention or a different data vendor is doing a different experiment, and the adapter interface in Choice 2 is where that difference has to be absorbed.

Running the loop: one script per experiment

The README is unusually clear about where the moving parts are. All principal components of the experiment are located in train_maskable_ppo.py, which tells you the shape of the code before you read it: a masked PPO agent, a state built from expressions and their scores, and a policy that emits the next expression to evaluate.

The parameters listed for building a run are the ones you would expect and the ones you will actually have to choose. The data side has instruments as a set of instruments, start_time and end_time as the data range for each dataset, and target as the target stock trend, for example a 20 day return rate. The training side has batch_size as the PPO batch size, features_extractor_kwargs as the arguments for the LSTM shared net, device as the PyTorch device, save_path for checkpoints, and tensorboard_log for TensorBoard.

The experiments are run as modules, and the README insists on that form:

python -m scripts.NAME ARGS...

with python -m scripts.NAME -h to see the arguments. Three scripts are named. scripts/rl.py runs the main experiments of AlphaGen and HARLA. scripts/llm_only.py is an alpha generator based solely on iterative interactions with an LLM. scripts/llm_test_validity.py tests how the system prompt affects the valid alpha rate of an LLM.

After a run, the model checkpoints and the alpha pools are located in save_path. The model is compatible with stable-baselines3, and the alpha pools are formatted in human-readable JSON, which is a detail worth appreciating: the output of a search over thousands of candidate expressions is a file you can read, diff and check into version control rather than a pickle. The TensorBoard logs are in tensorboard_log.

The LLM branch is a separate package, not a rewrite

The alphagen_llm package is described as containing LLM client abstractions and a set of prompts useful for LLM-based alpha generation, and it also provides some LLM-based automatic iterative alpha-generation routines.

The structure tells you the design intent. The LLM work is a client abstraction, which means the model is behind an interface rather than embedded in the search loop, and a prompt set, which means the prompts are data that can be inspected and changed rather than string literals scattered through the code. The two scripts that use it, llm_only.py and llm_test_validity.py, cover two distinct questions: can an LLM alone produce useful factors, and how much does the system prompt change the rate of valid expressions.

The second question is the more interesting one to a practitioner, because valid alpha rate is the metric that decides whether an LLM-driven generator is usable at all. A model that produces syntactically valid expressions most of the time but rarely ones that survive the pool and correlation filters is not a generator, it is a suggestion engine. The requirements list carries openai and num2words, and the prompts and the client abstraction are where a different provider would be plugged in.

The paper entry in the README for HARLA, a hybrid approach to formulaic alpha discovery with large language model assistance, is the natural citation for this branch, and it is the closest thing to a description of how the pieces fit together.

The baselines are vendored and modified, and the trading code is experimental

Two baselines are included as modified copies, and the reason for modifying them rather than installing them is the usual one: the comparison has to be fair.

The first is gplearn, which implements Genetic Programming, a commonly used method for symbolic regression, and the authors maintained a modified version to make it compatible with their task. The corresponding experiment script is gp.py. The second is DSO, Deep Symbolic Optimization, described as a mature deep learning framework for symbolic optimization tasks, of which the authors maintained a minimal version for compatibility, with dso.py as the experiment script.

Vendoring a baseline and changing it is normally a smell, and it is worth being explicit about why it is defensible here. A genetic programming or deep symbolic search baseline that does not see the same expression space, the same evaluation budget or the same normalisation as the proposed method is not a baseline, it is a different algorithm with a handicap. The cost is that the vendored code has to be maintained, and the benefit is that the comparison in the paper can be reproduced from this repository alone.

The trading part is labelled experimental, and the README is brief about it. Some trading strategies based on Qlib are implemented, with backtest.py and trade_decision.py as demos. The word demo and the word experimental together are a clear signal about how much weight to put on that directory. The factor search is the research contribution; the trading layer is there to show what the factors could be used for, and it is not documented as something to run with money.

No licence file, and dependency pins from the paper's era

Two practical matters sit outside the research, and both matter before you start.

The first is the licence. The repository ships no licence file in its top level listing, and the licence metadata is not classified either, so there is no stated grant for reuse, modification or redistribution of this code. The paper is published and the code is public, but public and permissively licensed are different things, and anyone planning to build on it commercially needs to resolve that with the authors rather than assume.

The second is the dependency set, and it is the reason a modern environment will need work:

baostock==0.8.8
gym==0.26.2
matplotlib==3.3.4
numpy==1.20.1
pandas==1.2.4
qlib==0.0.2.dev20
sb3_contrib==2.0.0
stable_baselines3==2.0.0
torch==2.0.1
shimmy==1.1.0
fire
openai==1.2.3
num2words

These are exact pins from the era of the paper, and they predate the Python versions most people run today. NumPy 1.20 and pandas 1.2 in particular are from a period when the supported Python floor was lower, so a current interpreter will not simply accept them. The research stack is also spread across three frameworks at once, gym, stable-baselines3 and sb3_contrib, with shimmy bridging the API change between them, which is a second sign of a codebase pinned to one moment in the dependency timeline.

The last push to the repository was on 2026-06-04, and the project has no GitHub releases, so there is no tagged history to check out and no changelog of fixes. The code is the artefact, and the versions above are the versions it expects.

Editorial conclusion

AlphaGen fits a quant researcher who wants to reproduce or extend the KDD 2023 work, or who wants the factor search loop separated from their own data and backtest stack through the AlphaCalculator adapter. It does not fit a team that needs a turnkey trading system, since the trading scripts are explicitly experimental, and it does not fit a modern Python environment without work, because the dependency pins come from the paper's era. Before you start, read the AlphaCalculator interface and decide which of the two reproduction choices you are using, and check the licence situation, since the repository ships no licence file.

Frequently asked questions

Official sources

1. ICT-FinD-Lab/alphagen on GitHub
2. Issues
3. README