Please provide all relevant details below before submitting your post. This will help the community provide more accurate and timely assistance. After submitting, you can check the appropriate boxes. Remember, you can always edit your post later to include additional information if needed.
Isaac Sim Version
[o] 4.2.0
[o] 4.1.0
4.0.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):
Isaac Lab Version (if applicable)
1.2
1.1
1.0
Other (please specify):
Operating System
Ubuntu 22.04
Ubuntu 20.04
Windows 11
[o] Windows 10
Other (please specify):
Hello,
I am currently using a Meta Quest 3 to run VR simulations in Omniverse Isaac Sim and have encountered the following issue.
After changing the main camera in the simulation and starting the VR session, the initial view comes from the updated camera as expected.
However, as the VR session progresses, the simulation camera’s position data is not reflected correctly, and the view remains fixed at its initial state.
While moving the VR headset updates the view in real-time, the camera attached to the robot’s head does not seem to affect the VR perspective. This suggests that the VR rendering system prioritizes the VR headset’s position data over the simulation camera’s position data.
Is there a way to modify the code or settings in Omniverse Isaac Sim to ensure that the simulation camera’s position data takes priority over the headset’s position?
If anyone has experienced a similar issue or knows how to address this, your advice would be greatly appreciated.
Thank you for your help!
Yes you are correct. The VR headset view is always from the human users point of view, viewing the scene and moving through the space. It is never a fixed camera view. It is just a free floating perspective camera. If you want to “watch” a fixed camera view from the robots perspective, simply view the camera, normally and not through VR.
VR has always been designed for YOU the human user to explore you scene in vr. You can view your robot. You cannot “be” your robot.
Just curious but does the robot have a complete 360 degree free rotation camera ?
We are currently attempting a simulation where a robot is controlled through VR. The ultimate goal is for the user to remotely control the robot through its point of view using VR. There are many other challenges as well, but we want to tackle them step by step, starting with this one.
Ok understood. Let me ask and see if there is some kind of work around for your situation.
Ok having talked to our top VR expert at the company, he shares my concerns with the concept here. We try to make the VR experience for humans, not for robots, and the problem is motion sickness. Ironically, locking the view in VR, but not locking the human head in reality, will cause massive motion sickness. The user will try to move his/her head naturally, and when the robot’s head does not follow, you will get massive nausea. FIxing the position of the VR viewer, creates the issue.
He recommended the same as me. Have the robot use its 360 camera, yes, but the human driver only views its output in 2D on a normal screen, and then rotates his angle of view manually.