Noise in Fisheye Camera input

Hi Nvidia developers,

I am working on a localization algorithm that uses fisheye camera input for feature detection and tracking. The fisheye camera input coming from Isaac Sim is very noise, which is causing the algorithm to fail. Here is the code I am using to initialize the camera.

camera = Camera(prim_path=prim_path)
camera.set_projection_mode("perspective")
camera.set_projection_type("fisheyePolynomial")
camera.set_stereo_role("mono")
camera.set_resolution((640, 480))
camera.set_frequency(30)
camera.set_fisheye_polynomial_properties(
            nominal_width=640,
            nominal_height=480,
            optical_centre_x=333.2475990547461,
            optical_centre_y=217.78326171321265,
            max_fov=163.857,
            polynomial=[0.0, 0.003569348092203986, 7.553851071502936e-08, 0.0, 0.0, 0.0]
)
camera.set_clipping_range(0.1,30.0)

Here is a sample image from the camera feed:

Can you suggest a way to fix/reduce this noise? I am okay if the simulation slows down or the image needs to be changed from color to grey scale.

Thank you!

just another user of OV, so take my input as a grain of salt.

i dont use fisheye cameras often, but have you tried capturing at a larger resolution to see whether render size played a role in the “noise” amount? and it’s likely not the case, but those artifacts remind me of images with little to no antialiasing. oh, and by the way, what GPU are you using?

Hi,

I tried increasing the resolution, but it didn’t help. I also tried changing the rendering settings to DLAA for anti-aliasing. It did slight positive change, but not enough for my use case.

I am using TU102 [GeForce RTX 2080 Ti Rev. A]

which renderer are you using? RTX-Realtime or Interactive?

RTX-Realtime

just out of curiosity, which version of Issac Sim are you on?

2022.2.1

Hi @vishrut - Apologies for the delay in response. Please let us know if you are still having any issues or questions after using the latest Isaac Sim release.