This thesis aims to investigate a geometric approach to enhance the implementation of Value Gradients on long trajectories. The research focuses on addressing the vanishing and exploding gradient issue in Value Gradients that limits their variance reduction capabilities.

Modular-Baselines is a Reinforcement Learning (RL) library, based on Stable-Baselines3, with the objective of improving flexibility and providing necessary components in RL Research. Components are framework agnostic in the sense that they do not rely on a specific framework. That said, Modular baselines includes both Pytorch and JAX implementations of some of the agents.