IQL-PyTorch

A minimal PyTorch port of Implicit Q-Learning, the offline reinforcement learning algorithm, with a script to run it on D4RL tasks. The paper's own implementation is JAX-based; this one is for PyTorch users.

This repository is a minimal PyTorch implementation of Implicit Q-Learning, an offline reinforcement learning algorithm, along with a script to run IQL on tasks from the D4RL benchmark. The README notes that the paper's authors published an official implementation based on JAX, and this one is intended as an alternative for PyTorch users. That keeps the door open for anyone working in PyTorch.

The implementation is being validated against the results stated in the paper as compute permits. It provides results for the MuJoCo locomotion tasks with normalized return at the end of training, averaged with standard deviation. A table compares this implementation with the official one on environments like halfcheetah-medium-v2 and hopper-medium-v2, with values shown for both columns. The comparison keeps the two columns side by side so any gap is visible at a glance.

The README is careful about what it claims. Performance is mostly similar to what is stated in the paper, but slightly worse on a few tasks. The results came from a small simplification using a deterministic policy and least-squares value fitting. It is presented as an honest comparison rather than a claim of exact parity.

The general recommendation is to use the authors' official implementation. This repository is presented as a practical alternative for developers who work in PyTorch. The scope is a minimal implementation with the D4RL script, and the README does not document a wider feature set beyond that.

• Source 1 · Project repository
• Source 2 · README