Moving objects along one axis


I am trying to do reinforcement learning on a problem similar to the cartpole problem. It is intended as a multi-agent cartpole problem, and I will therefore use the contraption seen below (which in real life is intended to be gripped by two robotic arms). However, I intend to simulate the problem without robots and make the object move by itself. I am intending to use a continuous action space and set the velocity in either direction on one axis with a maximum of 100 mm/s and a moving distance of maximum 40 mm. My question is:

How can I simulate that movement on this contraption? All the parts are rigid bodies with collision enabled, except for the pole which is simply just a rigid body. It is connected to the “cart” with a revolute joint and the physics work as intended, EXCEPT for when I try to move it in the GUI during a simulation, in which case the physics start to break and the object flies away. What is the best possible way for me to achieve this?

Thanks in advance!