Скачать или смотреть First Successful Model Trained with Soft Actor-Critic (SAC)

This model is purely a stress test with no direct practical application.
Its purpose is to explore whether a reinforcement learning agent can learn the mechanics of shifting the body’s weight when lifting a leg.

The key challenge lies in the dynamics: the robot’s legs are heavy, so when a leg is lifted, the body naturally begins to tilt in the opposite direction.
To maintain balance, the controller must learn to preemptively shift the body in the opposite direction of the leg’s motion,
effectively canceling the downward acceleration as the leg begins to rise.

But that’s only half the solution. Once the leg reaches the apex of its motion, the direction of acceleration reverses.
When the acceleration reverse, if the other three leg are ankled on the ground,
the controller must also prepared to reposition the body to counteract the the next leg motion
by predicting several hundred frames into the future in order to not flops.
This just the main idea, but there are lot more nuances to the problem.
Maintaining balance throughout the entire cycle requires precise timing and predictive motion.

Notice that this only happens with a slow-moving animation cycle,
for a fast animation the angular momentum of the body does not have time to till the body to fall before the leg reach the apex, therefore the body is in a dynamic balance.

Interestingly, other algorithms like TD3, DDPG, and PPO all failed to learn this behavior under the same conditions.
Only SAC was able to successfully adapt and solve the task.

While this specific model doesn’t serve a direct practical use, it’s an important step toward training more complex and capable agents in the future.