OpenAI Gym (2016)
The paper introduces OpenAI Gym, a toolkit for developing and comparing reinforcement learning algorithms. It provides a standard set of environments, a common interface, and a platform for benchmarking algorithm performance across diverse tasks.
Background
OpenAI Gym was a landmark open-source toolkit released by OpenAI in 2016 that standardized the way researchers develop and compare reinforcement learning (RL) algorithms — a branch of machine learning where agents learn by trial and error, receiving rewards for good actions. Before Gym, every lab built its own simulation environments, making it nearly impossible to reproduce or fairly compare results. Gym provided a common "gym" of tasks (Atari games, robotic control simulations, board games, etc.) where any algorithm could be plugged in and evaluated the same way. This paper introduced the Gym API and catalog of environments, which quickly became the industry standard. The project later evolved into Gymnasium (maintained by the Farama Foundation) after OpenAI pivoted away from maintaining it. This paper is a citation classic in AI/ML because it effectively created the infrastructure for the modern RL research pipeline.