Image pixel coordinate to real world coordinate

When possible, please provide the following info:
Hardware Platform: DRIVE AGX Xavier™ Developer Kit
Software Version: DRIVE Software 10
Host Machine Version: Ubuntu 18.04
SDK Manager Version: 1.1.0…


Now I am able to use LaneNet or DriveNet to find the lanemark or object boxing in the image coordinate. So, I would like to move one step forward to project the pixels in the image coordinate to the real world coordinate.
I suppose I need following steps:

  1. Create rig configuration of the camera
  2. Calibrate the camera
  3. Use the calibration result to bring pixel to ray.
  4. Calculate the intersection of the ray to the ground to get the world coordinate.

So here are my questions:

  1. I saw in the manual " dwLaneMarking" struct has a member of “worldPoints”. But it was zero all the time. Is it possible to let the LaneNet generate this value and use it as world coordinate?
  2. If the answer to question 1 is no. Then how can I do the transformation? I found one thread but it was for PX2.
    How to transform points from image space to world space?
    Moreover, there is a function “Mat4_Rxp” which I have no idea what it is. If this approach is good, then how shall I adopt it in DRIVE AGX Xavier?


Hi @hanyang.zhuang,

May I know which sample you are using to check worldPoints?

Hi @VickNV.

Basically I use the LaneDetection sample and adapt it into my application. I’m sure that I didn’t configure rig or calibrate the camera in my code. I check the SDK manual and the explanation of worldPoints in the struct is empty. So I wonder what the meaning of this value, and how can I use this value?

worldPoints will be available in LandmarkDetector API and its sample of the next release.

For Mat4_Rxp(), it is defined in /usr/local/driveworks/samples/src/framework/Mat4.hpp.