RLTrader: A Book's Companion Codebase for Reinforcement Learning Stock Trading

What RLTrader Is Actually For

This repository is the companion code for a book, titled in Korean as a guide to deep learning and reinforcement learning stock investment using PyTorch and Keras. The README lists the book's purchase links to Kyobo Book Centre and Yes24 before it lists anything technical, which tells you the intended audience: readers who have the book open beside them. The stated topics are deep learning theory, reinforcement learning theory, how to apply RL to stock investing, how to build an RL-based investing system, how to acquire and process real stock data, how to train on it, how to use a trained model, and how to customize the system.

That is a curriculum, not a product. The problem it solves is the absence of a worked, end-to-end example that connects a reinforcement learning agent to real Korean equity data with realistic feature engineering. Most RL tutorials use Atari or a toy gridworld. Here the state representation is a long list of price, valuation, moving-average and investor-flow ratios. If you want to see what an RL state vector looks like when the domain is a stock rather than a game screen, this is the point of the repository.

Who it is for: developers and quantitatively inclined readers who already know some Python and want to see the mechanics of an RL trading loop. Who it is not for: anyone looking for a library to import into a live trading stack. The README's own framing is educational, and the code is organized around a book chapter structure (the run script is named run_e3.cmd, and the main mode flag takes a --ver v3 argument).

The State Vector Is the Most Interesting Part

The v3 dataset definition in the README is where the design intent is clearest. Each stock row carries date, open, high, low, close, volume, per, pbr and roe, followed by a set of derived ratios: open_lastclose_ratio, high_close_ratio, low_close_ratio, diffratio, volume_lastvolume_ratio, and then moving-average ratios at 5, 10, 20, 60 and 120 day windows for both close and volume. After that come three investor-flow blocks, each with a raw value, a diff, and moving averages at the same five windows: ind (individual), inst (institution) and foreign.

That is a deliberately wide feature set. Rather than asking the agent to learn that a 20-day moving average matters, the preprocessing hands it the ratio directly. The trade-off is real: the agent receives a state vector in which most dimensions are already normalized and correlated with each other, which makes credit assignment harder and makes it easy to fool yourself about how much the network actually learned. The README does not discuss this, and it does not describe the reward function at all, which is the single most consequential design choice in any RL trading setup.

Market data is kept separate and joined at load time. The v3 market file contains date plus market_kospi_ma5_ratio, market_kospi_ma20_ratio, market_kospi_ma60_ratio, market_kospi_ma120_ratio, and the same four windows for bond_k3y. The README says v3 and v4 files should be used by combining market data with stock data, and points to data_manager.py for the merge logic. The v4 market section is listed as v3 plus additional fields, with the additional fields themselves written as TBD. That is an unfinished documentation entry, and you should treat v4 as under-specified until you read the code.

Running It: The CLI and the Pinned Environment

The entry point is main.py. The README gives one worked example:

python main.py --mode train --ver v3 --name 005930 --stock_code 005930 --rl_method a2c --net dnn --start_date 20180101 --end_date 20191231

Read the flags as a description of the system's axes. --mode selects train or, presumably, an inference mode. --ver selects the dataset version. --name is a run label and --stock_code is the Korean ticker (005930 is Samsung Electronics). --rl_method a2c selects the advantage actor-critic agent. --net dnn selects the network architecture, with an LSTM path implied by the troubleshooting section, which mentions an lstm/strided_slice error. --start_date and --end_date bound the training window in YYYYMMDD form.

The environment is the friction point. The README lists Anaconda 3.7+, TensorFlow 2.7.0, plaidML (plaidml-keras 0.7.0 and mplfinance), and PyTorch. The development environment section states Python 3.6+, PyTorch 1.10.1, TensorFlow 2.7.0 and Keras 2.7.0 bundled with TensorFlow. There are then separate conda recipes for TF 1.15 (Python 3.6, cudatoolkit 10.0, cudnn 7.3.1) and TF 2.5 (CUDA 11.2, cuDNN 8.1, with explicit Windows PATH entries). Note that the documented CUDA and cuDNN versions do not line up across these recipes. You will be reconciling them yourself.

Profiling is supported and documented: python -m cProfile -o profile.pstats main.py ... followed by python profile.py. That is a genuine convenience, and it suggests the authors expect training runs to be slow enough that you will want to profile them.

The Data Is Not in the Repository

This is the constraint that determines whether you can use RLTrader at all. The training data is distributed through the Quantylab Naver cafe, under a menu labeled for RLTrader training data. It is not in the repository, and the README does not provide a direct download link, only a pointer to the cafe. If you cannot access that cafe, or if the data has been reorganized since the README was written, the CLI example above will fail on data loading regardless of how correctly you set up the environment.

The README also notes that v1 through v4 datasets exist and that v3 and v4 are the ones to use, with v3 adding selected market data to stock data and v4 adding a larger volume of market data on top. Since the v4 additions are listed as TBD, the practical target is v3 unless you read data_manager.py to find out what v4 actually contains. Plan for a manual data acquisition step before any code runs, and treat that step as the first thing to validate rather than the last.

The two documented troubleshooting entries are worth noting because they are environment traps rather than logic bugs. The first is a NotImplementedError about converting a symbolic Tensor (lstm/strided_slice:0) to a numpy array under TF 1.15, resolved by pinning Python 3.6. The second is an AttributeError on f.attrs['keras_version'].decode('utf8'), resolved by installing h5py==2.10.0. The second one is a version-skew problem between Keras and h5py that has nothing to do with this project specifically, but it will stop you dead if you load an old saved model.

Where RLTrader Is the Wrong Tool

The most obvious failure mode is expecting backtest results to mean something. The repository trains an agent on a fixed date range and, per the README, supports using a trained model afterward. It does not document walk-forward validation, transaction cost modeling, slippage, or a held-out evaluation protocol. An A2C agent trained on 2018-01-01 to 2019-12-31 for a single ticker has seen one market regime. Nothing in the material suggests the code guards against overfitting to that window.

The second limitation is scope. The data schema is Korean-market specific by construction: per, pbr and roe are standard, but the investor-flow blocks (individual, institution, foreign) and the market series (KOSPI, 3-year Korean bond) are tied to that market's disclosure conventions. Porting to another market means rebuilding the feature pipeline, not just swapping a CSV.

The third is that this is a book's code, and books age. The most recent release listed is v2.1 from September 2021, with v2.0.1-alpha and v2.0-alpha from March and April 2020. The environment pins (TensorFlow 2.7.0, PyTorch 1.10.1, Python 3.6) sit well behind current tooling. The repository is not archived, and a push is recorded in July 2026, but the release cadence and the TBD entry in the v4 data description both indicate that the maintained surface is narrow.

Finally, the primary language is Jupyter Notebook. That matters if you intend to import RLTrader modules into your own code: the README says this is possible and points to main.py as the reference, which implies the notebook content is exposition and the reusable surface is the Python modules behind main.py.

Alternatives and the Difference in Approach

The most direct alternative is FinRL, which is also an open source reinforcement learning library for finance, but it is structured as a library with a gym-style environment abstraction rather than as companion code to a specific text. That difference shows up immediately: with FinRL you install a package and instantiate an environment against your own data; with RLTrader you clone a book's repository, obtain datasets from a cafe, and run main.py. FinRL's abstraction makes it easier to swap markets and algorithms. RLTrader's lack of abstraction is precisely what makes it readable as a teaching artifact, because there is no indirection between the explanation and the code.

For the narrower question of backtesting rather than RL, backtrader and vectorbt occupy different ground entirely: they simulate strategies you write by hand and give you the evaluation machinery (costs, slippage, portfolio accounting) that RLTrader's README does not describe. If your actual goal is to know whether a trading rule works, an RL agent is an unnecessarily indirect way to find out, and a backtesting framework will answer the question with far less compute.

The honest summary of the comparison: RLTrader is better than a general RL library at showing you what an RL state vector looks like for equities, and worse than a backtesting framework at telling you whether a strategy is any good.

Licence and the Cost of Keeping It Running

RLTrader is MIT licensed, which permits commercial use, modification and redistribution provided the copyright notice and permission notice are retained. That is permissive and imposes no copyleft obligation on your own code. Two caveats that are not legal advice: MIT covers the code in this repository, not the training data hosted on the Naver cafe, whose terms are separate and which you should read before redistributing anything derived from it. And the book itself is a commercial product; the repository is its companion, not a substitute.

Maintenance cost is dominated by the dependency pins. TensorFlow 2.7.0 and PyTorch 1.10.1 with Python 3.6 means you are building an environment that current package resolvers will fight you on, and the documented CUDA/cuDNN combinations differ between the TF 1.15 and TF 2.5 recipes. Expect to spend your first session on environment construction rather than on the model. The plaidML dependency (plaidml-keras 0.7.0) is a further pin to a project whose relevance has faded since the README was written, and it is listed alongside mplfinance for plotting.

Upgrade cost is the more interesting number. Because the merge logic lives in data_manager.py and the entry point is a flat CLI in main.py, the coupling between framework version and project code is likely concentrated rather than spread out. That is good news for anyone attempting a port to a current TensorFlow or PyTorch release, and it is consistent with the repository's role as a reference implementation rather than a dependency you would vendor.

Editorial conclusion

Adopt RLTrader if you are working through the book and want the code to match the text, or if you need a readable reference implementation of an A2C agent wired to Korean market features. Do not adopt it as a trading system: the training data is distributed through a Naver cafe rather than the repository, the environment pins are years old, and the v4 market data list is literally marked TBD in the README. Before anything else, verify that you can obtain the v3 or v4 dataset from the Quantylab cafe, because without it main.py has nothing to train on.

Official sources

1. License: MIT
2. Project website
3. quantylab/rltrader on GitHub
4. README
5. Releases