Birdeye view with dwRectifier on pinhole model

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after clicking “+ Create Topic”):
Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.4.1.7402
other

Host Machine Version
native Ubuntu 18.04
other

Hi,

We try to obtain a birdeye view using dwRectifier_setHomographyFromRotation on a pinhole camera.
It works fine for OCAM.

Is it supported?

Is there a better way of doing it?
Thx.

2

Hi @sacha ,

Have you tried sample_video_rectifier? Do you see any problem of this on the application?
The applicaiton uses pinhole. Thanks.

4

Hi @VickNV,

Actually, yes, we tried. But sample_video_rectifier uses only OCAM models, not pinhole.

5

Per below snippet from /usr/local/driveworks-3.5/samples/src/imageprocessing/geometry/rectifier/main.cpp, it can use pinhole model. Can’t it?

bool RectifierApp::initCameras()                                                                                                                                                                                                     
{
...
    if (RAD2DEG(m_fovX) >= 180)
    {
...
    }
    else
    {
...
        CHECK_DW_ERROR(dwCameraModel_initializePinhole(&m_cameraModelOut, &cameraConf, m_context));
    }
6

Agree. However, that’s the output camera model, not the input one.

7

Did you see any problem with pinhole camera model as input camera ? Thanks.

8

Yes, that’s the whole point. The homography does not seem to work to create the bird’s eye view for pinhole input camera.

9

Could you elucidate the problem more? Have you seen any error/abnormal messages with pinhole input camera? Thanks.

10

There is no error/abnormal message.
The resulting image after the homography is black when the input camera is pinhole.
It works well when the input cam is OCAM.

11

Please help to provide more details of the problem so we can help you. Thanks.

  1. Does the pinhole model you use produce an image when not applying homography?
  2. The homography is multiplied by the camera rays in world space, so it doesn’t matter the camera model, are you sure the calibration parameters you have are correct?
12
  1. Yes, it does, when you put the homography to the identity matrix, it displays an image.
  2. Yes, verified in Matlab. BTW, why is not available the tangential distortion for pinhole models?
13

Could you share the video (or a piece of it) and the rig? We need to run locally to understand what’s wrong. Thanks.

14

No, we currently don’t support tangential distortion in DriveWorks pinhole cameras, only radial distortion. Until now, we didn’t see this is required to improve accuracies significantly. In general, you should consider using FTheta camera model and not pinhole cameras (note that with our industry camera partners, we don’t use tangential distortions, only FTheta-based radial distortion).

15

Could you share a place where we could drop the files or provide an email adress that we can share on our OneDrive?

16

Can you send with a personal message of the forum? Thanks.

17

I have got your rig.json and video_NvidiaS0_conv.h264 via person message.
Please provide your command and its output messages so I can make sure reproduce your issue.
Thanks.

18

The command is
./sample_video_rectifier --camera-name=TEST_S0 --rig=rig.json --video=video_NvidiaS0_conv.h264 --fovX=60 --fovY=34

The goal is to display a bird’s eye view. Thank you,

19

Is the command to reproduce the issue? The rectified video on the right side looks normal.

20

After pressing a couple of times “J” to look down, you cannot get a correct image.

21

Is this what you see? I’m running on host system.

image
image1280×430 217 KB