Rendered Image not visible : Camera Sensor Standalone python example

[ksu1rng]

In order to use the camera in a project, I was following this tutorial. According to the instructions, the image should be rendered after a few simulation steps, but it doesn’t appear.

[ksu1rng]

Can you please update on this thread?

[rthaker]

Hi @ksu1rng - Are you seeing any errors or warnings when you run the standalone python script?

[ksu1rng]

Hello,

I am getting the following text message as a kind of warning in Isaac Sim GUI while running the camera sensor standalone python example.

“SdRenderVarToRawArray missing valid input renderVar TargetMotionSD”

Are you able to render the images as shown in tutorial ?

[ksu1rng]

Hello,

Can you please update on this ?
I just want to know, according to the instructions, the image should be rendered [pop up window will appear] when this script runs. But it doesn’t happen.

[ahaidu]

Hi there,

if you comment out all the print statements (just to avoid filling up the terminal) do you get the following warning?

UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()

Best,
Andrei

[ksu1rng]

Hi,

Yes I am getting the following warnings that you’ve mentioned

2023-07-24 11:37:18 [15,964ms] [Warning] [carb.flatcache.plugin] UsdRelationship /Render/RenderProduct_Replicator.orderedVars has multiple targets, which is not supported

2023-07-24 11:37:20 [17,470ms] [Warning] [omni.syntheticdata.plugin] SdRenderVarToRawArray missing valid input renderVar TargetMotionSD

standalone_examples/api/omni.isaac.sensor/camera.py:201: UserWarning: Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  plt.show()

Do you know, work around this ?

Best,
Sanjeev

[ksu1rng]

Hi,

I am also facing another issue. In order to move forward with my current project, I decided to save images using the OpenCV library instead of showing via matplotlib.pyplot. To install the OpenCV library, I used the following command: ./python.sh -m pip install <library_name> within the existing Python environment of isaac-sim.

I attempted to save the image from the camera using the get_rgba() method on the camera instance. I tried to store only the RGB image using cv2.imwrite("file_name", camera.get_rgba()[:, :, :3]). However, I am not getting the exact colors in the image.

Expected color of the ground-plane is “Blue”
But in saved image, I am getting “somewhat orange shade”

Do you know the reason for this ?

Regards,
Sanjeev

[ahaidu]

You can try installing a GUI backend such as qt using ./python.sh -m pip install pyqt5 or see if the save_rgb example snippet helps:

def save_rgb(rgb_data, file_name):
    rgb_image_data = np.frombuffer(rgb_data, dtype=np.uint8).reshape(*rgb_data.shape, -1)
    rgb_img = Image.fromarray(rgb_image_data, "RGBA")
    rgb_img.save(file_name + ".png")