Fast Pointcloud rendering

marcelto · October 26, 2023, 7:24pm

Hi,

I need fast pointcloud rendering in isaac-sim, however it is pretty slow using the standard cameras, and I am seeing that when getting the pointcloud, it also returns the segmentation and the rgb. Is it possible to turn these off to get faster pointcloud rendering?

Thank you,

nikepupu9 · October 30, 2023, 9:37am

If you know the camera parameters, you can use this script:

github.com

arnold-benchmark/arnold/blob/main/utils/compute_points.pyx

import numpy as np
import cython
from cython.parallel import parallel, prange

@cython.boundscheck(False)
@cython.wraparound(False)
def compute_points(int height, int width, float[:,:] depth, float fx, float fy, float cx, float cy):
    cdef points_cam = np.zeros((height, width, 3), dtype=np.double)
    cdef double[:, :, :] result_view = points_cam
    cdef int v, u, k
    cdef float d
    with nogil, parallel():
        for v in prange(height, schedule='static'):
            for u in range(width):
                d = depth[v,u]
                
                # https://docs.omniverse.nvidia.com/app_isaacsim/app_isaacsim/reference_conventions.html
                # rotate around x axis 180
                # point[2] = -d *100.0
                # point[0] = -( u - cx ) * point[2] / fx

This file has been truncated. show original

marcelto · October 30, 2023, 12:29pm

Thank you! This should work. However, a bottleneck for my data generation using isaac sim, is that on my current GPU I can only have 12 cameras otherwise it consumes too much GPU memory, so I can only collect environments from 12 environments in parallel. Is that normal, or am I using your cameras incorrectly?

Thank you again

nikepupu9 · October 31, 2023, 8:25am

which version are you using? I think for 2023 there is a huge speed up for multi-camera.

marcelto · October 31, 2023, 2:28pm

I am using 2022.2.1. Is that right? I haven’t seen any mention of the multi-camera in the release notes, but maybe I missed it? Would you have some pointers by any chance?

Thanks again