Access to GPU Texture Data of Render Output

[i.bahceci]

Hello,

I have been trying to capture the rendered GPU texture data from omniverse viewport. I have tried C++ and Python API of Omniverse-Kit without luck so far. My aim is that, I get the GPU texture handle at the end of rendering so that I can use that texture for post processing e.g. using DirectX or Vulkan. Is it possible ?

[ben115]

Hi @i.bahceci I am trying the same thing and am stuck. Did you solve this? How can we access the camera buffer directly using kit?

I am using:

camera = rep.create.camera(position=(0, 0, 1000))
render_product = rep.create.render_product(camera, (1920, 1080))

# Get the hydra_texture
hydra_texture = render_product.hydra_texture
resource = self.hydra_texture.get_drawable_resource()

but that is as far as i have got.

[Richard3D]

It’s pretty easy to access the RTX viewport buffer directly with code, but it does not mean you can get access to the individual textures as you’re trying to do. It’s just a way to capture the whole viewport to disk.

[AshleyG_NVIDIA]

Hey @ben115

There is no officially supported way to get the GPU handle directly from hydra_texture in Python
But you can try reading to the CPU to get a NumPy image and RGB data:

import omni.replicator.core as rep

image = await rep.capture.capture_image(render_product)

[ben115]

Thanks! Will this also work with this snippet from the docs? Or is there a more direct way to do this in c++? I need to access the camera buffer without creating a viewport.

Overview — Omniverse Kit

def renderer_completed(event: carb.events.IEvent):
    if event.type != omni.kit.hydra_texture.EVENT_TYPE_DRAWABLE_CHANGED:
        carb.log_error("Wrong event captured for EVENT_TYPE_DRAWABLE_CHANGED!")
        return

    # Get a handle to the result
    result_handle = event.payload['result_handle']
    # And pass that to the HydraTexture instance to get the AOV's that are available
    aov_info = hydra_texture.get_aov_info(result_handle)
    print(f"Available AOVs: {aov_info}")

    # Get an object for a specific AOV and include the GPU texture in the info
    ldr_info_array = hydra_texture.get_aov_info(result_handle, 'LdrColor', include_texture=True)
    print(f"LdrColor AOVs are: {ldr_info_array}")

    ldr_texture = ldr_info_array[0].get("texture", None)
    assert ldr_texture is not None

    ldr_color_res = ldr_texture.get("rp_resource")
    ldr_color_gpu_tex = ldr_texture.get("rp_resource")
    print(f"LdrColor[0]: {ldr_color_res}, {ldr_color_gpu_tex}")

    # YOU CANNOT USE THE RESOURCE OUTSIDE OF THIS FUNCTION/SCOPE
    # ldr_color_gpu_tex must be consumed now or passed to another object
    # that will use / add-ref the underlying GPU-texture.

event_sub = hydra_texture.get_event_stream().create_subscription_to_push_by_type(
    omni.kit.hydra_texture.