About Creating a New Task Questions

I’ve read the documentation and the sample code, and I’ve tried to create my new reinforcement learning tasks, but I don’t understand the principles of the whole framework yet.
I’ve taken some tensors and used them to build actions and rewards, but so far I haven’t been able to run them successfully because there are too many errors about tensors and dimensions, similar to the following:

RuntimeError: mat1 dim 1 must match mat2 dim 0

def quat_mul(a, b):
assert a.shape == b.shape
~~~~~~~~~~~~~~~~~~~~~~~~~ <— HERE
shape = a.shape
a = a.reshape(-1, 4)
RuntimeError: AssertionError:

RuntimeError: The size of tensor a (9) must match the size of tensor b (2) at non-singleton dimension 1

RuntimeError: shape mismatch: value tensor of shape [9] cannot be broadcast to indexing result of shape [9, 3]

I don’t even know where to start troubleshooting errors, I’m still using the PPO algorithm in the example, and I haven’t changed the network structure.
Are there any points or tips that I should be aware of, and can I get some advice on how to build new reinforcement learning tasks?

Thank you very much.

We have documentation on the RL framework for Isaac Gym to help you get started. They are located in docs/rl/framework.html, you can look into the “Class Definition” and “Creating a New Task” sections. The error you posted looks like it’s caused by a call to quat_mul(a, b) where tensors a and b have mismatching shapes. The quat_mul(a, b) method computes element-wise quaternion multiplications between two tensors of quaternions, so both a and b tensors must have a matching shape of n x 4, where n is the batch size of the tensors. If you are having trouble debugging through the pytorch jit functions, you can try removing the @torch.jit.script annotations so that you can set debug points within those functions and get more useful errors at run-time. For dimension mismatch errors, it may be helpful to check the dimensions of the tensors that you are passing to make sure they have the expected shapes.

Thanks for the replies, I’ve solved quite a few problems.
I’ve never worked with so many tensors at once, and it was a little difficult at first!
Also, I would like to know the usage of the get_axis_params function. I found many instances that use this function, but I couldn’t find it in the documentation.
Thank you very much!

get_axis_params() is meant to be an utility function when parameterizing the up axis (such as switching between Y-up and Z-up). It likely won’t be needed for most cases where parameterization of the up axis is not required. The idea of get_axis_params() is to create an array and assign index y of the array to value x, where x and y are parameters to the function, along with a few other parameters that you can specify. You can see implementation details for the function in python/isaacgym/torch_utils.py.

1 Like