Although there is a variety of reinforcement learning (RL) teaching material, we in practice identify a few remaining gaps: 

• Textbooks provide the essential introduction to the field, often with nice conceptual examples. However: 

  • • Students often struggle to translate between equations and code. 

• Public (documented) codebases are great for research. However: 

  • • They implement complicated algorithms that conflate many separate algorithmic ideas. 

    • They often contain much boilerplate code (e.g., for logging), 'hiding' the key RL essentials. 

• Environment packages (based on Gymnasium) provide a unified framework for testing. However: 

  • • Most environments conflate several challenges (stochasticity, partial observability, exploration, etc.), and are not tailored to teach a particular aspect/challenge of RL.

    • Most environments are high-dimensional, and therefore not suitable for fast interactive experimentation. 

The above problem creates a gap for: 

• Educational reinforcement learning environments, which are:

  • • designed to illustrate a particular aspect/challenge of reinforcement learning. 

    • (ideally) have tunable parameters to vary the strenght of that particular challenge. 

    • allow for a fast experimentation cycle. 

• An associated set of interactive notebooks, which: 

  • • illustrate each challenge and possible solution approaches, connecting equations and code in a single document. 

    • allow for (fast) interactive experimentation by the student. 

EduGym exactly implements the above solution: 

• We selected a set of core topics in reinforcement learning (see the list under Notebooks), 

• designed an environment that illustrates the particular challenge, and

• made an interactive notebook that explains the challenge and possible solution approaches (using its associated environment). 

As such, EduGym may be used as a companion to classic textbooks, (online) courses and existing documented RL codebases.