, we explored how to extend Reinforcement Learning (RL) beyond the tabular setting using function approximation. While this allowed us to generalize across states, our experiments also revealed an important limitation: in simple environments like GridWorld, approximate methods can struggle to match the stability and efficiency of tabular approaches. The main reason is that learning a good representation is itself a difficult problem—one that can outweigh the benefits of generalization when the state space is still relatively small. To truly unlock the power of function approximation, we therefore need to move to environments where tabular methods are no longer viable. This naturally leads us to multi-player games , where the state space grows combinatorically and generalization becomes essential – and at the same time perfectly fits into this post series, as so far we did not manage to learn any meaningful behavior on more complex multi-player environments.…