Does Create Movie Capture support multi camera?

Hi, all
Movie Capture Settings UI can only setup one camera and then Capture Sequence/Submit to Queue/Capture Current Frame.
Is there any plan to support mulit camera? Or maybe my problem, not finding the correct usage.

i think only way is to animate your camera with keyframed positions and then render image seq.

I know what you are looking for, a typical architect rendering workflow.
@WendyGram could this feature to added to the wishlist?

I appreciae you taking the time to reply me.
Sorry, my English is a little rusty.

In fact, I want use OV to simulation unmanned aerial vehicle(UAV) which has multi camera. In order to do that, I use Unity and write C# scripts link to camera object, but Unity can not render multi camera real-time that is why use Nvidia OV.

I see source code of OV Create Movie Capture, form omni.renderer_capture call acquire_renderer_capture_interface(), and then call render_capture.capture_next_frame_rp_resource(filePath, viewport)

from omni.kit.viewport call get_viewport_interface(), and then get current viewport. I know i can create multi viewport and setup different cameras after i see scripts/test_multiview.py

But render_capture seems a singleton object, so call capture_next_frame_rp_resource() with different camera can not acquire same frame image

I watch the video of Nvidia Drive Sim which based on OV Kit, it shows multi viewport from different camera on car. I believe OV Kit can do what i want to do, just i didn’t find the solution to the problem.

wow! this is hardcore!
@WendyGram can sure help you with this by connecting the right devs for you.

Thanks @pekka.varis! Hello @user76666! I contacted the developer for Movie Capture to help! I should hear back soon. I will also make a feature request ticket as soon as he gets back to me!

Hi @user76666 , it looks to me that you want to capture a movie with multi cameras at the same time, right? This unfortunately is not a designed case for movie capture tool and the capture extension source you have looked into. For the capture extension, it’s designed to capture a viewport window, which is rendered with one camera at a time. You can change camera in your script during capture so that movie capture will capture with the new camera but it comes in sequence, not parallel to the previous camera used. So maybe, you can try to capture the same simulation and animation use different cameras, and then you will get a series of movies from different cameras and then making you final movie with them? Does this work for you?

Hi @quchen , I appreciae you taking the time to reply me.
Your username looks like Chinese name, If I’m right, we can talk in Chinese.
It seems not work for me, because I need capture a scene with different cameras. It means these cameras need capture at same frame.
I want simulation unmanned aerial vehicle(UAV) which has multi camera. UAV can fly in the scene, collect images from these cameras and send images to some algorithm and then use algorithm result control UAV.

Then I guess you may want to do this with multiple viewports, and capture those viewports with their own cameras. You can try to create a new viewport via the [Window → New Viewport Window] menu and set its own camera. In theory it should be doable by creating a new extension for your needs with the capture API you found.

That’s what I thought of before.
The premise is renderer_capture can acquire from different viewport, but I only know this API:acquire_renderer_capture_interface() to acquire renderer_caputre, and there is no doc about render_capture.
How can i config which viewport used with renderer_capture? Or acquire different renderer_capture from different viewport?
Thank you in advance.

I hope this code snippet can help you a bit:

self._viewport = omni.kit.viewport_legacy.get_viewport_interface()
# to get default viewport window
viewport_window = self._viewport.get_viewport_window()
# to create a second viewport window
self._viewport_handle_2 = self._viewport.create_instance()
self._viewport_window_2 = self._viewport.get_viewport_window(self._viewport_handle_2)
# for a viewport window, you can get/set its camema:
viewport_window.get_active_camera()/set_active_camera(camera_name)
# for the image capture api, I guess you can reuse the one in omni.kit.capture.viewport's extension.py
def _capture_viewport(self, frame_path: str):
        viewport_window = self._viewport_interface.get_viewport_window(None)

        def _capture_helper(viewport_rp_resource):
            self._renderer.capture_next_frame_rp_resource(frame_path, viewport_rp_resource)

        if self._to_capture_render_product:
            self._renderer.capture_next_frame_using_render_product(viewport_window.get_id(), frame_path, self.options._render_product)

        omni.kit.viewport_legacy.deferred_capture(viewport_window, _capture_helper, is_hdr=self.options._hdr_output, subscription_name="omni.kit.capture capture helper")

Sorry for getting back to you so late.
I can’t found omni.kit.viewport_legacy in Kit. So I use other way to capture frame, but also not work

self._view_port_factory = omni.kit.viewport.get_viewport_interface()
self._renderer = omni.renderer_capture.acquire_renderer_capture_interface()
def _capture_viewport(self):
viewports = self._view_port_factory.get_instance_list()
viewport_names = [self._view_port_factory.get_viewport_window_name(vp) for vp in viewports]
  for i in range(len(viewports)):
      vp = self._view_port_factory.get_viewport_window(viewports[i])
      viewport_rp = vp.get_drawable_ldr_resource()
      frame_path = str(self._capture_count) + viewport_names[i]+ ".png"
      self._renderer.capture_next_frame_rp_resource(frame_path, viewport_rp)

If I create multi viewports, it only save first viewport frame. I guess omni.renderer_capture can only capture a single viewport.
So… I found omni.syntheticdata and write demo follow omni.syntheticdata test. It can save different viewport frame. But unfortunately, frame saved not continuous(save frame at update_event_stream callback). And I can’t find samples or doc for how can I control viewport frame update
I have two questions:

  1. Does omni.renderer_capture can capture different viewport? How can do that?
  2. How can omni.syntheticdata.sensors.get_rgb(viewport) get continuous frame? In a other word, can I control viewport update, or can I use viewport callback when frame update finish?