Hi, since the last update of the USD Composer application in Omniverse I am able to stream content to the Magic Leap 2 through remote rendering.
Now I’m struggling to get the controller input into Omniverse. Can anyone point me in the right direction to get controller button states show up in Omniverse?
Would appreciate any help!
Yes it should work with USD Composer 2023.2, there’s a magic leap extension they should turn on. Can you see if you can find that extension in the extension manager ? Let me know.
Thanks Richard.
Yes I have the extension enabled, but I’m unsure how to make use of it. My virtual controller doesn’t show up when I look through the glasses when I start AR and the buttons don’t trigger anything.
Are there any nodes I need to create in Action Graph to get the controller data?
@mathradecs i don’t own ML or any HMDs, so this is just me doing some due diligence. i dug around and while what i did find doesn’t address your question, it could have some nuggets of information from the team at ML:
i saw you have reached out to the dev over at the ML forum (i went there to look for additional info myself) and are waiting for their response, so hopefully they can provide more concrete information. in the interim, the GTC video linked above sounded to me like ML has future plan to release their extension/connector at some point down the road, which have full controller support. what was shown in this demo showcased the early access program of their extension at around @10:53.
then @14:52, Connor mentioned two new extensions as part of their official extension: xr.profile.magicleap and xr.system.openxr. so this is the part that i would defer to Nvidia and/or ML team.
having said all that, i couldn’t test or prove anything out without the hardware in question. what i did try to do was to look into the content of this Composer extension; and, it includes a model asset that’s meant to be the ML2 controller:
Thanks for your response @Simplychenable.
Yeah I remember this video and it has some very useful information. It seems like it is supposed to work out of the box, but unfortunately I can only stream and not interact with the scene. I also found the assets in the extensions, so everything should be good to go. I hope this will be fixed soon.
@Richard3D do you have any new information on the topic?
We will be testing the latest build with all the controllers and I will post the updates.
Just curious but have you tested the older version of Composer ?
No I haven’t. Please correct me if I’m wrong, but I think the old one doesn’t have the possibility to connect XR devices.
The old version does have the ability to connect to a wide variety of headsets yes. Specifically your Magic Leap 2 I am not sure. Our testing is ongoing
