Rectification of supported Entron F008A120RM0AV2 Camera Sensor

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.10.0
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description

Hi,
I noticed that the frames captured from the Entron F008A120RM0AV2 camera sensor are not rectified enough. As you can see in the attached image below: poles, trees and buildings that are far from the center of the frame are not straight. I’m currently not performing any image processing in my application, just grabbing frames from multiple cameras and streaming them.

My questions are as follows:

  1. Is there any rectification performed within the drive isp pipeline for this camera module? If there is, is it possible to modify it and enhance the rectification?
  2. If I perform intrinsic calibration using your tools ( DriveWorks SDK Reference: Static Camera Calibration Tutorial) and obtain the parameters for pinhole model to utilize the dwRectifier (based on your sample DriveWorks SDK Reference: Video Rectification Sample) for real time usage, how can i use it during recording using your recorder tool? I need to be able to record rectified data.

Thanks,
Alon

Dear @al.es,
Recorder tool does not record rectifier data.
Is camera data captured using sample_camera with default rig.json(/usr/local/driveworks/data/samples/sensors/camera/camera/rig.json) or any changes camera parameters from in the rig.json?
Does DriveWorks SDK Reference: Video Rectification Sample sample works with your recorded video and gives you rectified output?
Can you try using rectification APIs in sample camera to get rectifier output on display?

Hi,

Here’s the rig I’m using:
rig_6_cameras.txt (5.6 KB)

I’m running with my own project, but I tried running with sample_camera with default rig and got the same results.
Didn’t try to implement the rectification sample yet, was hoping for a config solution.
Can you please reply my questions:

  1. Would you say the rectification observed in the images I shared is as expected?
  2. Assuming this can only be solved by using the rectifier of driveworks, what is the best practice for rectifying data recorded by the recorder tool?

Can you please reply my previous question as well:

  1. Is there any rectification performed within the drive isp pipeline for this camera module? If there is, is it possible to modify it and enhance the rectification?

Thanks,
Alon

Sample_camera, recorder tool does not perform rectification. You need to use rectifier APIs on captured frames.

Can you try rectification sample using the recorded video as suggested.

Hi,
I tried using the rectification samples as you suggested.

  1. I ran the rectifier sample (DriveWorks SDK Reference: Video Rectification Sample ) successfully using this command:

sudo ./sample_video_rectifier --rig=/target/vehicle/rig/ar0820_ftheta_1_8mp_calib_virtual.json --video=camera_front_center.mp4 --camera-name=camera:front:center
attached below an image and terminal prints displaying the output, the FPS is sufficient and also the rectification as a start.


rectifier_terminal.txt (8.2 KB)

  1. Then, I tried running the rectifier LDC sample (DriveWorks SDK Reference: Video Rectification with LDC Sample ) using this command:

sudo ./sample_video_rectifierLDC --rig=/target/vehicle/rig/ar0820_ftheta_1_8mp_calib_virtual.json --video=camera_front_center.mp4 --camera-name=camera:front:center
and got bad results. as you can see in the image below, the FPS is very low and the result is noisy. also attaching terminal prints.


rectifier_ldc_terminal.txt (6.5 KB)

I’d like to use the NvMedia LDC engine and offload from the GPU so can you please give some insights on how to get it to work?

for both cases I used calibration data that I generated using your static calibration tool, using the A0 14x9 checkerboard. attaching the rig file as well.
ar0820_calib_rig.txt (1.1 KB)

Could you please share the needed files to repro the issue and debug? Also, please share the steps followed for calibration to verify with core team.

Dear @al.es
Any update can be provided for this topic?

Hi,
You can close this topic.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.