SV3DT World Coordinates (ground plane) Origin OR how to generate projection matrix for a customer video/camera?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Thanks for sharing a great work on MOT (SV3DT).

Regarding the camera calibration process, as I know there are 3 coordinates World, Camera, Image Plane. For this particular sample/example camInfo, I am very curious, where is the World (ground plane) coordinate system (origin).

If camera coordinate origin = world coordinate origin, then the camera matrix has the following form (similar, not exactly the same)

f  0  cx  0
0  f  cy  0
0  0  1   0

However, the sample projectionMatrix_3x4 in camInfo (cam_matrix = IntrinsicMatrix x ExtrinsicMatrix, ExtrinsicMatrix transforms a point in 3D world to camera coordinates) suggests that camera coordinate origin is not the same as that of the world coordinates.

It would be very nice, if you can briefly describe how to generate the projection matrix for a customer video/camera.

Thanks for your time.

3

Please refer the guide on the projection matrix here: Gst-nvtracker — DeepStream documentation 6.4 documentation (nvidia.com)

4

Dear @kesong ,
We are facing the same issue when we would like to employ SV3DT in our application.

We have examined the document that you link already. Basically, it links to opencv-calib. In that document, as our understanding, the projection matrix is composed of the intrinsic matrix and extrinsic matrix, PM = A [R|t]. The former can be obtained from the camera calibration process. However, we have difficulty on how to obtain the later.

We are really interested in its (SV3DT) unique feature, and we would like to make use of it.

If you know any source that can help us proceed this step, we would really appreciate your support.