# hok_env: a reinforcement learning environment built on Honor of Kings > hok_env is Tencent AI Lab's open environment for the Honor of Kings MOBA, providing a Python SDK that talks to the game's gamecore, a reinforcement learning training framework and a PPO implementation, together with the paper and baseline results for the Honor of Kings Arena benchmark. It is aimed at research teams working on competitive multi-agent reinforcement learning. **tencent-ailab/hok_env** — Honor of Kings AI Open Environment of Tencent - Repository: https://github.com/tencent-ailab/hok_env - Website: https://aiarena.tencent.com/aiarena/en/open-gamecore - Stars: 920 · Forks: 109 - Language: Python - License: Apache-2.0 - Published: 2026-10-20 · Updated: 2026-10-20 · Language: en - Canonical page: https://hysenlabs.com/projects/tencent-ailab-hok-env ## A real MOBA as the environment, not a simulation of one hok_env is the open environment of the MOBA game Honor of Kings. The README is precise about what the repository contains: mainly the hok_env SDK, a reinforcement learning training framework, and an implementation of the PPO algorithm based on that training framework. The SDK is used to interact with the gamecore of Honor of Kings. The gamecore is the important word. This is not a reimplementation of a game written for the purpose of reinforcement learning. It is the actual game engine, running as a simulator, and the SDK is a Python interface that speaks to it. That distinction is the project's entire research value, and it is also why the setup is heavier than a typical Gym environment. The paper behind it is Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learning, published at NeurIPS Datasets and Benchmarks 2022 with an arXiv entry at 2209.08483. The abstract sets out the challenge the environment is designed to expose. It is a multiagent problem with one agent competing against its opponent, and it requires generalization because it has diverse targets to control and diverse opponents to compete with. The paper describes the observation, action and reward specifications for the domain, provides the open-source Python interface for communicating with the game engine, gives twenty target heroes with a variety of tasks, and presents initial baseline results for RL-based methods with feasible computing resources. The generalization angle is what separates it from a training environment. An agent that beats one opponent on one map has learned a policy, not a skill, and the paper's closing section is about the challenges that imposes and possible remedies. ## Three pieces in one repository The repository layout shows how the pieces are separated. There is an `hok_env/` directory for the SDK itself, an `rl_framework/` directory for the training framework, and an `aiarena/` directory holding the training code, with `aiarena/1v1` for the one against one training code that the README points to. Then there is the operational material. A `docs/` directory holds the cluster training instructions in `cluster.md` and the workaround for running the Windows gamecore on Linux in `run_windows_gamecore_on_linux.md`. A `dockerfile/` directory and a `build.sh` cover container builds, and `3rd/` holds third party components. There is an `archive/` directory for older material, and two licence files at the top level, `LICENSE` and `GAMECORE.LICENSE`, which is a hint that the gamecore itself is licensed separately from the code around it. The entry point for a new user is the unit test. The README points to `hok_env/hok/hok1v1/unit_test/test_env.py` for the basic usage of the 1v1 environment, and the 3v3 mode has its own equivalent that can be run directly: ``` python3.8 -c "from hok.hok3v3.unit_test.test_env import run_test; run_test()" ``` That command is worth reading twice, because it tells you the package layout as well as the entry point. The import path is `hok.hok3v3.unit_test`, so the SDK is importable as a `hok` package with a subpackage per mode, and the unit test module exposes a `run_test` function that can be called without arguments. The 3v3 documentation lives at doc.aiarena.tencent.com and is a separate site from the main documentation. The README also flags the current supported heroes, and there are twenty of them, from lubanqihao and miyue through libai, makeboluo, direnjie, guanyu, diaochan, luna, hanxin, huamulan, buzhihuowu, jvyoujing, houyi, zhongkui, ganjiangmoye, kai, gongsunli, peiqinhu and shangguanwaner. ## Nothing runs until you apply for the gamecore This is the step that catches people out, and it is not in the installation section, it is in the gamecore installation section. You need to apply for the licence and the gamecore on the open gamecore page at aiarena.tencent.com. The README says to put the `license.dat` file under the folder `hok_env_gamecore/gamecore/core_assets`. So the gamecore is not in the repository, and it is not downloaded by any script in the repository. It is distributed under an application process, and a licence file gates its use. That is a reasonable arrangement for a commercial game engine, and it makes the boundary between the open source work and the proprietary work explicit. The consequence for an evaluator is significant. You cannot clone this repository and start training. You cannot even test the SDK, because the SDK talks to the gamecore. The first thing anyone does is apply for access, and the second thing is run the simulator by hand to confirm it starts before touching the training code. The homepage of the project is the open gamecore page at aiarena.tencent.com, which is where the application lives, and the documentation site is at aiarena.tencent.com/hok/doc. The CHANGELOG.md at the top level is where the release history is recorded. ## The gamecore is a Windows binary, and that shapes the deployment The Running Requirement section lists the constraints, and they are unusual for a reinforcement learning environment. Python must be greater than or equal to 3.6 and less than or equal to 3.9. The operating system must be Windows 10 or 11, or Linux with wine, in order to deploy the Windows gamecore server. Docker is needed to deploy hok_env on Linux containers, and on Windows, WSL 2 is required. There is also a Python 3.8 command in the README, which fits inside that range. The README explains why: the gamecore of hok_env runs on the Windows platform, and the package hok_env needs to be deployed on Linux platforms to interact with the gamecore. So the simulation is a Windows process and the learning code is a Linux process, and they talk to each other. The project ships a Docker image for training on your own computer, and a further version will release a gamecore server compatible with Linux. For cluster training, the README points at a workaround by running the Windows gamecore on Linux, documented in `docs/run_windows_gamecore_on_linux.md`. That is the shape of the project today, and it is worth being blunt about the cost. A Python ceiling of 3.9 rules out most current environments, since 3.9 predates the structural typing changes that later Python versions brought. The Windows gamecore means the training machine needs Windows, or WSL2 with a Windows host, or wine. And the cross-platform split means the ZMQ-based interface between the simulator and the client carries the traffic of every frame. ## Starting the simulator and reading its output The README documents the manual test, and it is short enough to be the first thing you run. Open a command prompt, change into the gamecore binary directory, add the library directory to the path and start the remote simulator with its configuration file: ```shell cd gamecore\bin set PATH=%PATH%;..\lib\ .\sgame_simulator_remote_zmq.exe .\sgame_simulator.common.conf ``` The executable requires one parameter, the path to the config file. The configuration itself is small and tells the simulator which scene to load, where the assets are, and which heroes the two players control: ```json { "abs_file": "../scene/1V1.abs", "core_assets": "../core_assets", "game_id": "kaiwu-base-35401-35400-6842-1669963820108111766-217", "hero_conf": [ { "hero_id": 199 }, { "hero_id": 199 } ] } ``` Two hero entries means a two player match, and the same `hero_id` on both sides is a mirror match, which is the natural starting point before you introduce asymmetry. The README then shows the expected console output, and it is worth checking the specific lines rather than the whole block. You should see `PlayerNum:2`, the two `PlayerInfo` lines with camp, hero and skill identifiers, `init_ret:0`, `start_ret:0`, and a closing `SGame Simulator End` line with the frame count and the elapsed time. If `init_ret` is not 0, the simulator did not initialise and nothing downstream will work. The run also writes output files, an `.abs` file with the replay data, a `.json` file, a `.stat` file and a `_detail.stat` file, all named after the game id. ## The 1v1 and 3v3 modes, and where the training code lives The two modes are documented separately, and the README's structure makes the distinction clear. For 1v1, the observation and action space specifications are on the quickstart page at aiarena.tencent.com/hok/doc/quickstart/index.html. The usage example is `hok_env/hok/hok1v1/unit_test/test_env.py`, and the training code is in `aiarena/1v1`. For 3v3, the README leads with the update that 3v3 mode is now available, and the docs live at doc.aiarena.tencent.com under the hok3v3 environment path. The difference in the research question is real. The paper describes the environment as a multiagent problem with one agent competing against its opponent, and the 3v3 mode changes the credit assignment and the opponent modelling problem, since the team has to learn to play with and through teammates rather than as a single unit. Cluster training is the third piece, and `docs/cluster.md` covers using the hok_env environment with the integrated `rl_framework` for cluster training. The framework in `rl_framework/` is what the PPO implementation sits on, and the README describes the repository as containing an implementation of the PPO algorithm based on that framework. The overall picture is a research stack rather than a library. You get an SDK that speaks to a real game engine, a training framework, a reference algorithm, unit tests for each mode, and the paper's baselines. The cost is that every layer has to be working before you learn anything, and the gamecore gate is the first of those layers. ## Apache-2.0 for the code, a separate licence for the gamecore The repository licence field says Apache-2.0, and the top level carries both a `LICENSE` file and a `GAMECORE.LICENSE` file. That pairing is the point. The code you can read, the SDK, the training framework, the PPO implementation and the examples are under Apache-2.0, which is permissive and carries an explicit patent grant. The gamecore binary is licensed separately, and its use is gated by the application and the `license.dat` file. So the licence position is: the research contribution is open, the simulator is licensed. Anyone writing a paper on competitive reinforcement learning can build on the SDK and the training code, but running anything at all requires access to the engine. On maintenance, the numbers are worth reading carefully. The tagged releases are v2.0.4 and v2.0.3, both on 2023-12-28, and v2.0.1 on 2023-08-31, whose release name is the 3v3 baseline. The last push was on 2026-05-18. That is a gap of more than two years between the newest tag and the last commit. It is consistent with a project whose published artefacts are a paper, a benchmark and a fixed set of baselines, and whose repository receives occasional maintenance rather than continuous development. The README's mention of a further version releasing a Linux compatible gamecore server is a promise about the future rather than something you can rely on today. For anyone planning to build on this, the sensible reading is that the environment is a stable research artefact with a known state rather than a moving target, which is exactly what you want if you intend to publish results on it and compare them with others who used the same version. ## Conclusion Adopt hok_env if you are doing competitive multi-agent reinforcement learning and want a real MOBA rather than a toy environment, and you are prepared to deal with a Windows gamecore, a licence application and a Python version that predates 3.10. Skip it if you want a self-contained environment that runs on a modern Linux box, because the gamecore is Windows only in this release and the project says a Linux compatible gamecore server will come in a further version. Verify first that you can obtain the gamecore licence, that your host runs Windows or WSL2 or wine, and that the simulator starts and reports `init_ret:0` before you invest in a training run. The licence field on the repository says Apache-2.0, and the last push was on 2026-05-18. ## FAQ ### What is hok_env? It is the open environment of the MOBA game Honor of Kings from Tencent AI Lab. The repository mainly includes the hok_env SDK used to interact with the gamecore, a reinforcement learning training framework, and an implementation of the PPO algorithm based on that framework. ### How do I get the Honor of Kings gamecore for hok_env? You need to apply for the licence and the gamecore on the open gamecore page at aiarena.tencent.com. Then you put the `license.dat` file under the folder `hok_env_gamecore/gamecore/core_assets`, since the gamecore is not distributed with the repository. ### What are the system requirements for running hok_env? Python greater than or equal to 3.6 and less than or equal to 3.9, Windows 10 or 11 or Linux with wine to deploy the Windows gamecore server, Docker to deploy hok_env on Linux containers, and WSL 2 on Windows. The gamecore runs on Windows while the package is deployed on Linux to interact with it. ### How do I check that the gamecore works? Open a command prompt, change into `gamecore\bin`, add `..\lib\` to the path and run `.\sgame_simulator_remote_zmq.exe .\sgame_simulator.common.conf`. The README says a successful start shows `PlayerNum:2`, `init_ret:0` and `SGame Simulator End` with the frame count and elapsed time. ### What is available in the 3v3 mode of hok_env? The README announces that 3v3 mode is now available, and it can be exercised with `python3.8 -c "from hok.hok3v3.unit_test.test_env import run_test; run_test()"`. The 3v3 documentation is on doc.aiarena.tencent.com, and cluster training instructions for the hok_env environment with the integrated `rl_framework` are in `docs/cluster.md`. ### What licence does hok_env use? The licence field on the repository says Apache-2.0 for the code, with a separate `GAMECORE.LICENSE` file covering the gamecore. The releases are v2.0.4 and v2.0.3 on 2023-12-28 and v2.0.1 on 2023-08-31, and the last push was on 2026-05-18. ## Sources - [License: Apache-2.0](https://github.com/tencent-ailab/hok_env/blob/master/LICENSE) - [Project website](https://aiarena.tencent.com/aiarena/en/open-gamecore) - [README](https://github.com/tencent-ailab/hok_env/blob/master/README.md) - [Releases](https://github.com/tencent-ailab/hok_env/releases) - [tencent-ailab/hok_env on GitHub](https://github.com/tencent-ailab/hok_env) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/tencent-ailab-hok-env