Multiple Camera Feed via REB_Camera

Hi there,

I currently have a setup where I am able to send and view the camera image feed via the REB_Camera component, over to a Jupyter notebook with PyAlice on the Isaac SDK.

However, I don’t see any options that allows me to associate REB_Camera to specific Camera components, but instead it just follows the current selected camera.

Is it possible to set up a REB_Camera for multiple cameras, or have multiple REB_Camera for each Camera? Or will I have to manually write a middle layer that will generate the numpy images and send them over via PyAlice instead?

Hi Kelvin,

Unfortunately, multi camera is not supported in the current version (2020.2)
Please stay tuned as this will be supported in our next release in May.

Kindly,
Liila

So, just to clarify the question for my specific application, there is no way to obtain the views from multiple REB_Cameras simultaneously using Isaac SDK?

This will be possible with the next release coming out June 10th

Related question, will this also enable multiple ROS camera feeds?

yes, it will support multi camera with ROS, we even will have an example for it.

Kindly,
Liila

has the release date been pushed back again?

2 Likes

Hi Phil,

Isaac Sim 2021.1 is released, only the NGC container is not out yet and will be updated on Monday.

Kindly,
Liila

ltorabi,
As you mentioned that Isaac Sim 2021.1.0 container image not updated yet (and still points to 2020.2.2_ea Isaac Sim container) - will check Monday for the update.
Also will Isaac Sim 2021.1.0 be available without the use of container?
Thanks

Hi Xavier,
yes, you can run Isaac Sim locally if you have a RTX GPU.

Please go to Omniverse website, download the launcher, and then install Isaac Sim.

Kindly,
Liila

Oh great. Thanks. I thought Omniverse was for Windows only. Just downloaded for the Linux system. Will try/launch Isaac Sim from there.

Thank you Liila.

Hey, I have a question related to this topic. Is it possible within the omniverse IsaacSim to have the camera render specific layers? Like what is being done in Unity. Essentially you can select a layer, for example all the walls, and the camera will show that ignoring everything else that stands in the middle. (Unity’s culling masks, HDRP custom pass)

@eliabntt94 Its possible to do something like this when running isaac sim via a python app and stepping manually. You can set the visibility of a prim, update the renderer so the visibility updates and then publish an image by manually ticking the ROS camera component.

Yeah, I’ve seen that you can step. I wanted to try yesterday but I had some crashes with IEditor. Therefore, I tried to reinstall everything from scratch but I encountered an error with Nucleus (a fix should come soon).

Do you have any fast-forward tip for this? I’ve seen that you can “update” the scene but that applies for both the physics and the editor. My idea of proceeding is more or less this:

def editor_update(dt):
    Get the camera
    Save the data
    Disable the visibility of the prims
    Save the data again

# start simulation
kit.play()

# assign callbacks
update_sub = kit.editor.subscribe_to_update_events(editor_update)

But IDK (since I’ve not tried yet) if the new data gets rendered inside the “subscriber”.

Sorry! I didn’t realize you were asking about REB components (my brain read it as ROS), there is actually a sample that should do exactly what you need

python_samples/isaac_sdk/pose_estimation.py

Specifically it hides the environment, publishes an image of the cube, then unhides the environment and publishes the image with everything.

docs are here:

https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/sample_python_isaac_sdk.html#pose-estimation

That’s great, but I’ve the feeling that if I run “update” and, let’s say, a robot in the scene has a non zero velocity, the robot moves in that update step. Am I wrong?

Yes, if you call update() simulation will also happen.

I believe there is a way to force an update to only do rendering and that is by doing something like

kit.update(physics_dt=1.0, physics_substeps=0)

you can test this by adding it to the other kit.update tests in
python_samples/simple/time_stepping.py

This is a bit of a workaround at the moment with the current release. With the next release stepping physics vs stepping rendering separately will be more user friendly.

Ok perfect. I’m a bit over smth else today but as soon as I have an answer I’ll get back to you ;) Thanks!

hi @Hammad_M I tried this but it seems like that this works only when the simulation is run in headless mode. When I press the play button I see the physics update step != 0 (and the robot is affected by gravity). Seems that physics_substep cannot be set to 0, nor physics_dt. Any tip?

I’ve tried to set physics_dt = 0.0000001 and that seems to work but it is not ideal clearly.
I’ve seen that there is a carb.kit.get_settings() + set_setting(’/physics/maxNumbSteps’,0) possibility but it crashes bc carb has no “kit” module