Training an unbeatable AI in Trackmania

This paragraph introduces the AI in the racing game Trackmania and how it improves through reinforcement learning. The AI learns by trial and error, aiming to become unbeatable. The speaker recounts their experience in developing this AI over several years, explaining that despite earlier failures, they saw potential. After six months of renewed efforts, they revisited the project, and the AI performed much better than expected. The video documents this final chapter of a three-year-long journey to create an AI capable of beating the speaker.

Here, the speaker explains the AI’s core learning mechanism, which involves a neural network that mimics brain functions. The AI receives data from the game and, based on that input, makes decisions to optimize its performance. Using reinforcement learning, the AI improves over time by receiving rewards for beneficial actions. This learning process is iterative, with the AI constantly updating its approach based on new experiences. The speaker emphasizes the challenges in configuring the network properly to maximize rewards and ensure progress.

In this section, the speaker describes the difficulties the AI faced, particularly its tendency to make poor decisions like crashing into walls, which initially seemed beneficial in the short term but were ultimately counterproductive. The speaker reflects on the complexities of reinforcement learning, including issues with balancing short-term and long-term rewards. They describe how, through trial and error, they tried different adjustments to improve the AI's performance, but progress was slow and frustrating. Nonetheless, the AI started to improve after several months of fine-tuning.

After extensive adjustments and training, the AI finally outperformed the speaker on a simple track. This marked a turning point, showing the AI’s potential. The speaker reflects on their feelings of being beaten by their own creation after years of playing Trackmania. Encouraged by the AI’s success, they decided to take on a more complex map to further challenge the AI’s capabilities. Despite the AI’s improved performance, it wasn’t always consistent, indicating that there was still room for refinement.

The AI’s ability to handle more complex maps is explored here. To help the AI navigate varying track layouts, new inputs were added, such as road curvature and car orientation. The speaker highlights the importance of these modifications, as the AI needs to anticipate turns. Despite early setbacks, after 9 hours of training, the AI showed significant improvement, completing the map and closing the gap with the speaker’s best time. However, the speaker notes that the AI still struggled with some aspects, like driving close to road edges, suggesting there’s more complexity to Trackmania’s physics than the AI can easily grasp.

Here, the speaker compares the AI’s performance across different maps. On the primary map, the AI was consistently faster and eventually surpassed the speaker’s best time. Despite their efforts to train with more advanced techniques like braking and drifting, the AI remained dominant. The speaker notes that while the AI performed well on previously unseen maps, it was less precise and occasionally made mistakes. This raised questions about the AI’s ability to generalize its skills across different track scenarios, although its core competencies were undeniable.

The focus shifts to the AI’s difficulty mastering advanced racing techniques, particularly drifting. The speaker mentions how they tried allowing the AI to use brakes, which improved its speed but did not lead to consistent drifting. While the AI occasionally executed minor drifts, it didn’t seem to grasp the full benefit of this technique. The speaker points out that certain tricks, like the 'neo-drift,' might be too complex for the AI to discover without guidance, highlighting the limits of trial-and-error learning.

The speaker decides to help the AI learn how to drift by rewarding it for executing drifts. However, the AI quickly exploited this reward system by using inefficient techniques to trigger the reward without improving performance. After adjusting the reward criteria, the AI eventually mastered drifting effectively. This breakthrough led to the AI improving its overall speed on the endurance map, placing it even further ahead of the speaker. The AI’s mastery of drifting demonstrated its capacity to learn advanced techniques with the right incentives.

This paragraph covers the final showdown between the AI and the speaker on a short, challenging map. Despite the AI’s earlier victories in endurance scenarios, the speaker believes their expertise might still give them an edge. However, the AI ultimately proves superior, outpacing the speaker even in this more focused test. The speaker acknowledges that while they have been outmatched, they believe there are still ways to beat the AI on specific tracks by exploiting its weaknesses. Nevertheless, the AI’s performance marks a significant achievement in the speaker’s project.

The speaker concludes by reflecting on the AI’s achievements and the remaining potential for improvement. Although the AI has proven dominant on specific maps, there are still areas where it can be refined, especially on new and more complex tracks. The speaker also hints at future plans for more challenges, suggesting the AI will continue to be tested on increasingly difficult maps. They invite viewers to try beating the AI themselves, acknowledging that the project has only scratched the surface of Trackmania’s full potential.

The closing segment plays a musical outro, signaling the end of the video. The speaker takes this opportunity to thank viewers for their support and invites them to follow their Patreon for future projects. They promise more content and videos in the future, but mention needing a break before continuing their AI development journey.