Camera simulation focus distance and Projection Type

Omniverse Isaac Sim
1

Hi,
I have a question regarding the Focus Distance property of the camera. I have different real world cameras which I want to simulate as close as possible. I only have the datasheet, so no distortion values available. My question is how I should choose the Focus Distance value?

For example it I want to model this https://mvpaerospace.com/wp-content/uploads/2023/04/mvp_camera_datasheet_v2.pdf camera with this optics: Lensagon BMK2119C – Lensation GmbH?

I looked into the LeopardImaging → Hawk-> LI-AR0234CS-GMSL2-OWL

camera which is already modeled in Isaa sim but could not figure out how to find the value to use for the Focus Distance

Furhtermore, to figure out which projection type to use: So far I used the stated resolution, pixel pitch and vertical/horizontal FOV from the datasheet and then calculated which of the fisheye models (projection types ) had the lowest error. Is there another way to better figure this out and then model the camera as close as possible?

Ont the Max FOV property, is it correct that this only plays a role for “true” fisheye cameras, so if my vertical/horizontal FOV is below 100° I can just set this value to a big number. Or how does it come into effect if I use e.g fisheyeEquidistant (with around 80° horizontal/vertical FOV)?

I use
2023.1.1

Thank you very much for the support

3

Hi @yhofmann welcome to the community.

Please switch to Isaac Sim 4.5 as this is the latest stable version. Is there any reason that you have to use 2023.1.1 version?

4

HI @zhengwang,
I use OmniLRS (GitHub - OmniLRS/OmniLRS: Omniverse Lunar Robotics Simulator) which currently still uses 2023.1.1, however, they plan to move to 4.5. This is planned already for some month but progresss is rather slow.

What is the advantage of 4.5 over 2023.1.1 regarding camera simulation?

5

Please refer to our release notes for sensors:

Isaac Sim 4.5: Release Notes — Isaac Sim Documentation
Isaac Sim 4.2: Release Notes — Isaac Sim Documentation
Isaac Sim 4.1: Release Notes — Omniverse IsaacSim latest documentation
Isaac Sim 4.0: Release Notes — Omniverse IsaacSim latest documentation

Also this camera is already part of Isaac Asset Sensor Assets — Isaac Sim Documentation. This documentation also contains its feature and specification. We are using focus distance 0.6m and fisheyePolynomial projection type.

6

HI @zhengwang,
Thanks for the links.
Regarding the OWL, yes I saw this documentation. I was wondering why the focus distance deviates and also for the cameras SG5_IMX490C_5300_GMSL2_H110SA and SG8_AR0820C_5300_GMSL2_H60SA (part of Isaac Assets) which have fisheye orthographic and equidistant projection type how to know why this is the correct projection type, such that I can also use the method to figure out my projection type for the cameras I use. Also how to get the focus distance parameter, since this and the projection type is not stated in their datasheet.

Thats why I proposed the method of calculating the error for the three different projection types, as with the pinhole ptojection type the image is only sharp at a narrrow band

image
image793×325 68.6 KB

I also looked at the existing SG8_AR0820C_5300_GMSL2_H30S1A , which uses the pinhole projection type, however there nothing is sharp ind the render.

Thanks for the feedback

7

Also when looking at the MaxFoV parameter, does the documentation show that this is the diagonal FoV Camera — Omniverse IsaacSim latest documentation, however when looking at any of the exisitng Sensing cameras (e.g the SG5-IMX490C-5300-GMSL2-Hxxx) is the value set to the bigger one of vertical or horizontal FoV.

How should this parameter now be used?

8

Thanks @yhofmann . I also noticed that focus distance is different in our document and the datasheet. I talked to the internal team and they said the values are from the Leopard imaging company. So it could be a bug/mistake.

In the future, we will have a documentation to show how you can populate the values from datasheet and hopefully that would be helpful.

10

Hi @zhengwang,
Thanks for talking to the internal team. Can you give me an ETA on this documentation?
Will this also include the point about which projection model to use? Can you already tell me if my approach is right or point me in the right direction / reference helpful material?

11

@yhofmann unfortunately I don’t have the ETA. But I will update you once I have the ETA.
Yes it will include all of the camera parameters including the projection model.

You approach is on the right track. The values we are using in our asset are also from the datasheet provided from corresponding companies. There might be some mistake causing the value mismatch.

12

Hi,
I wanted to ask for clarification regarding points of the camera:

  1. Angular field of view: Is it correct that the following formula determines the angular field of view if pinhole is used? (f = focal length, H = horizontal aperture)
    image
  2. Can you confirm the following units
    focal length = mm
    Aperture = mm
    Focus distance = m
    Nominal width/height = px
  3. Which of the parameters under Fisheye Lens need to be set (accurately) if I use pinhole as a projection type?
    image
    image402×375 11.6 KB

Thanks

13

I’m struggeling a lot with the focus distance. I have now done extensive research on depth of field but still struggle to find out which value I should put in for a specific camera for the focus distance. This “moves” the sharp part (depth of field) forward and backwards.

An answer on how to figure this value out would be greatly appreciated, thanks.

14

@yhofmann shall I close this topic? It looks like your question has been answered in this another topic Depth of field rgb camera not correct

15

@zhengwang you can close it

1 Like