YCB Writer not working with PathTraced Rendering

Hi,

Playing around with the offline_pose_generation example (6. Offline Pose Estimation Synthetic Data Generation — Omniverse Robotics documentation), I faced a problem with the YCB writer. If I switch to PathTraced Rendering, I get an error within the generation process. In addition, the images are not saved, even if the logs output that the images are generated.

I am working on the latest Release Isaac Sim 2022.1.1.

Example logs:
ID: 4/59
2022-09-05 09:02:29 [60,424ms] [Error] [omni.graph.core.plugin] Assertion raised in compute - index 1431655765 is out of bounds for axis 0 with size 3
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.core-1.4.3+lx64.r.cp37/omni/replicator/core/ogn/python/_impl/nodes/OgnWriter.py”, line 256, in compute
writer._write(writer_payload)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.core-1.4.3+lx64.r.cp37/omni/replicator/core/scripts/writer_registry.py”, line 157, in _write
self.write(data)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.isaac/omni/replicator/isaac/scripts/writers/ycb_video_writer.py”, line 154, in write
self._write_semantic_segmentation(data, render_product_path, annotator)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.isaac/omni/replicator/isaac/scripts/writers/ycb_video_writer.py”, line 249, in _write_semantic_segmentation
segmentation_data_remapped = np.take(semantic_id_to_class_index_map, semantic_seg_data)
File “<array_function internals>”, line 6, in take
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/kit/extscore/omni.kit.pip_archive/pip_prebundle/numpy/core/fromnumeric.py”, line 191, in take
return _wrapfunc(a, ‘take’, indices, axis=axis, out=out, mode=mode)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/kit/extscore/omni.kit.pip_archive/pip_prebundle/numpy/core/fromnumeric.py”, line 58, in _wrapfunc
return bound(*args, **kwds)

ID: 5/59
ID: 6/59
ID: 7/59
2022-09-05 09:02:31 [62,222ms] [Error] [omni.graph.core.plugin] Assertion raised in compute - index 1431655761 is out of bounds for axis 0 with size 3
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.core-1.4.3+lx64.r.cp37/omni/replicator/core/ogn/python/_impl/nodes/OgnWriter.py”, line 256, in compute
writer._write(writer_payload)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.core-1.4.3+lx64.r.cp37/omni/replicator/core/scripts/writer_registry.py”, line 157, in _write
self.write(data)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.isaac/omni/replicator/isaac/scripts/writers/ycb_video_writer.py”, line 154, in write
self._write_semantic_segmentation(data, render_product_path, annotator)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.isaac/omni/replicator/isaac/scripts/writers/ycb_video_writer.py”, line 249, in _write_semantic_segmentation
segmentation_data_remapped = np.take(semantic_id_to_class_index_map, semantic_seg_data)
File “<array_function internals>”, line 6, in take
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/kit/extscore/omni.kit.pip_archive/pip_prebundle/numpy/core/fromnumeric.py”, line 191, in take
return _wrapfunc(a, ‘take’, indices, axis=axis, out=out, mode=mode)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/kit/extscore/omni.kit.pip_archive/pip_prebundle/numpy/core/fromnumeric.py”, line 58, in _wrapfunc
return bound(*args, **kwds)

ID: 8/59
ID: 9/59
ID: 10/59
2022-09-05 09:02:32 [63,754ms] [Error] [omni.graph.core.plugin] Assertion raised in compute - index 1414878545 is out of bounds for axis 0 with size 3
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.core-1.4.3+lx64.r.cp37/omni/replicator/core/ogn/python/_impl/nodes/OgnWriter.py”, line 256, in compute
writer._write(writer_payload)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.core-1.4.3+lx64.r.cp37/omni/replicator/core/scripts/writer_registry.py”, line 157, in _write
self.write(data)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.isaac/omni/replicator/isaac/scripts/writers/ycb_video_writer.py”, line 154, in write
self._write_semantic_segmentation(data, render_product_path, annotator)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/exts/omni.replicator.isaac/omni/replicator/isaac/scripts/writers/ycb_video_writer.py”, line 249, in _write_semantic_segmentation
segmentation_data_remapped = np.take(semantic_id_to_class_index_map, semantic_seg_data)
File “<array_function internals>”, line 6, in take
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/kit/extscore/omni.kit.pip_archive/pip_prebundle/numpy/core/fromnumeric.py”, line 191, in take
return _wrapfunc(a, ‘take’, indices, axis=axis, out=out, mode=mode)
File “/home/mwiedema/.local/share/ov/pkg/isaac_sim-2022.1.1/kit/extscore/omni.kit.pip_archive/pip_prebundle/numpy/core/fromnumeric.py”, line 58, in _wrapfunc
return bound(*args, **kwds)

Hi there,

thank you for reporting this, I can replicate the error and we will investigate the possible cause. For me however I only get the error once and the images and the data was successfully written. Were there any other parameters changed so I can try to fully replicate this?

Cheers,
Andrei

UPDATE: Interestingly I am not getting the error running on Linux with a dedicated GPU. Would it be possible to share your nvidia-smi output? Or even better the full log output when you get the error?

Hi Andrei,

Thanks for your reply! I am running the example on a Linux machine with dedicated GPU, too. I am using Ubuntu 22.04 with a RTX A5000 GPU.

nvidia-smi output:

  nvidia-smi
  Thu Sep  8 12:27:01 2022       
  +-----------------------------------------------------------------------------+
  | NVIDIA-SMI 510.85.02    Driver Version: 510.85.02    CUDA Version: 11.6     |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |                               |                      |               MIG M. |
  |===============================+======================+======================|
  |   0  NVIDIA RTX A500...  Off  | 00000000:01:00.0  On |                  N/A |
  | N/A   65C    P5    31W /  N/A |    865MiB / 16384MiB |      2%      Default |
  |                               |                      |                  N/A |
  +-------------------------------+----------------------+----------------------+
                                                                                 
  +-----------------------------------------------------------------------------+
  | Processes:                                                                  |
  |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
  |        ID   ID                                                   Usage      |
  |=============================================================================|
  |    0   N/A  N/A      3165      G   /usr/lib/xorg/Xorg                312MiB |
  |    0   N/A  N/A      3636      G   ...ome-remote-desktop-daemon        3MiB |
  |    0   N/A  N/A      3669      G   /usr/bin/gnome-shell              256MiB |
  |    0   N/A  N/A      4219      G   ...AAAAAAAAA= --shared-files       49MiB |
  |    0   N/A  N/A      6882      G   /usr/lib/firefox/firefox          235MiB |
  |    0   N/A  N/A     14955      G   gnome-control-center                3MiB |
  +-----------------------------------------------------------------------------+

The log-file:
isaac-logs.txt (52.5 KB)

And the config-file:
ycb_config.json (2.8 KB)

In addition, I checked some configs for the generation again.

  • Using “PathTracing” as parameter: Error appears
  • Using any other renderer param (even non-sense like “abc”): No error

And you are right, the images are generated for me now, independent of having this error message or not. It seems I have overseen the generated data last time or whatever…

Regards
Marvin

Hi Marvin,

thanks for the detailed update. It seems the error appears when an integrated GPU (e.g. Intel) is available as well . I will create a bug request concerning the error. Regarding the render parameter, when you write “abc” the renderer just defaults to raytrace, hence no error.

Cheers,
Andrei

Hi,
I got the same errors using default renderer “RayTracedLighting”.
Is there any ways to fix this?

Is the data being written?

If the errors show, the data won’t be written.
Only part of frames showed the errors.

Do you by any chance have a dual GPU setup? E.g. Intel + Nvidia? Could you share your nvidia-smi results?

I’m using vGPU.
nvidia-smi

Thank you for your report, I will update the bug report accordingly and ask around for any solution.