Rectifier: Homography, top down/bird's eye view

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.4.1.7402
other

Host Machine Version
native Ubuntu 18.04
other

Hi, I’m currently looking into sample_video_rectifier and its transformations, more specifically, homography matrices.

In RectifierApp::updateHomography() there are a few things I do not understand. For example, how does camOutRotationMatrix work, that is, how does it define rotations? And of what?
Same thing goes for normalToPlane.

I was actually trying to add rotations on input, not just translations after pressing keys. The image I’m trying to create is top view or so called birds eye view image of undistorted/rectified sample image, but I don’t understand the way rotations work…

Maybe you could reference me to some coordinate system/rotations documentation or explain perhaps how could one rotate and translate cameras so that the output would be a top down/bird’s eye view projection?
Maybe an in detail explanation of what is transformation matrix?

I also tried using dwRectifier_setHomographyFromRotation() with roll, pitch and yaw from rig.json file and it also produces (for me) unexpected results.

Thanks

2

Hi @dmatic ,

Please take a look at the source of computeHomography() is in /usr/local/driveworks/samples/src/framework/MathUtils.cpp. Thanks.

4

Hi,

that actually helped, tnx.

I do have to ask though, what is the order of rotations and how are your coordinate systems defined?

In my case, I simply tried to “correct” cameraOutRotationMatrix by multiplying it with my own simple rotation matrix (3x3) with rotation around one axis. I got what I needed eventually, but with the y-axis looking in the opposite direction. Isn’t vehicle coordinate system defined with y axis (increase) looking to the left of the car? Where should vehicle c.s. and camera c.s. be oriented in order to have all roll, pitch and yaw equal to zero?

It’s a bit hard for me to explain it without drawing it. I just want to say that I think I have a general idea of how computeHomography() works, I just need to understand your angle and rotation definitions.

5

Please refer to Coordinate Systems for the information. Thanks.

6

Unfortunately, this is insufficient info for my case. I will explain in detail:

So I’m editing main.cpp of /usr/local/driveworks/samples/src/imageprocessing/geometry/rectifier. The only thing that is different from the original main.cpp file that came with driveworks is the RectifierApp::updateHomography() method.

What I changed is: I created a new rotation matrix in which I define exactly one rotation and I pass it as an argument in computeHomography() instead of camOutRotation matrix. Also, I removed user input translations so that on key pressed I would only get the desired rotation.

RectifierApp::updateHomography()
(example unchanged above)

 dwMatrix3f R_test;
 getRotationMatrix(&R_test, 90, 0, 0);
 
 // normal to the output plane
 float32_t normalToPlane[3] = {-1, 0, 0};
 float32_t distanceToPlane = m_transformation.array[3 * 4 + 0];
 // define camera output translation, same as the input camera + user input
 float32_t camOutTranslate[3] = {m_transformation.array[12], 
                                 m_transformation.array[13],
                                 m_transformation.array[14]};
 
 computeHomography(&m_homography, m_transformation, R_test, camOutTranslate, normalToPlane, distanceToPlane);
 dwRectifier_setHomography(&m_homography, m_rectifier);

(example unchanged below)

And what I got is following:

  • getRotationMatrix(&R_test, angleValue, 0, 0); rotates image around Z axis (Base Vehicle Coordinate System)
  • getRotationMatrix(&R_test, 0, angleValue, 0); rotates image around X axis (Base Vehicle Coordinate System)
  • getRotationMatrix(&R_test, 0, 0, angleValue); rotates image around Y axis (Base Vehicle Coordinate System)

How is that possible? Is it maybe defined with this normalToPlane? How is normalToPlane related to rotations in this sample? Where is that plane and what does it represent?

Also, when using both getRotationMatrix(&R_test, 0, 0, 90) and getRotationMatrix(&R_test, 0, 0, -90) I get image that points to the sky and never to the ground. How is it possible that with both 90 and -90 it points to the sky? Am I missing another coordinate system?