Deepstream 360 Parking Image Calibration

Hi, I’m having some issues with calibrating my snapshots to generate the perception spreadsheet. I’m following this guide: https://devblogs.nvidia.com/calibration-translate-video-data/ and the one on the deepstream sdk guide.

  • "The first step in calibration is to get snapshot images from all cameras." How do I get these dewarped snapshots and what are the resolutions needed?
  • "The first step in calibration is to get snapshot images from all cameras." How do I get these dewarped snapshots and what are the resolutions needed?
  • And with QGIS, there's this line "Make a note of the longitude and latitude of the origin (in this case, the center of the building)." Does this mean that I have to get the coordinate of the center of the referenced TIF image?
  • On the nvaisle/nvspot.csv files, what do ROI and the H values represent?
  • How do I find the gx,gy coordinates? I tried just importing the latitute and longitude but they don't work.

Thank you.

Hi velushomaz,

• “The first step in calibration is to get snapshot images from all cameras.”
How do I get these dewarped snapshots and what are the resolutions needed?
A: in the deepstream_360_degree_smart_parking application, there is a model that transfers fish-eye images to dewarped images. For your only application, the “dewarped image” is the one that object detection and tracking are applied on. If you have a fish eye camera (like the 360 degree ones), probably you need to dewarp; If you have a regular camera (like a roadside traffic camera), probably you don’t need to dewarp. The calibration is to transfer the detected object location from camera domain to map domain so the first step is to get the snapshot images with the resolution that the detection will be applied on.

• And with QGIS, there’s this line “Make a note of the longitude and latitude of the origin (in this case, the center of the building).”
Does this mean that I have to get the coordinate of the center of the referenced TIF image?
A: The “global coordinates” don’t necessarily have to be the latitude and longitude. It can be any meaningful coordinates in the map domain with a user specified origin. In the cases where lat and long are not easy to get or are not optimal for the post analytics due to its scale, you may want to use your own “global coordinates” with user a specified origin. The origin doesn’t need to be the center of the maps of course.

• On the nvaisle/nvspot.csv files, what do ROI and the H values represent?
• How do I find the gx,gy coordinates? I tried just importing the latitute and longitude but they don’t work.

A: H values represent homography matrix, ideally the H values should not be part of the config files, they should be instantiated (during start) based on the values of the image and global coordinates mapping, however for ds-360 they were added as part of the config.

ROI are specific to aisle and parking spot.

gx, gy are part of the csv. [Relating to the Qs below on how to find gx, gy]. Please see header in the csv.
As above mentioned, H represents mapping from image to global plane. Each such mapping and ROI are camera specific.

Hey Kayccc,

So for the 360 sample application the csv with H and gx, gy values were hard coded and will only work with the supplied playback videos?

In order to use the application for any camera, I would need to use something similar to the below python cv code from (https://www.learnopencv.com/homography-examples-using-opencv-python-c/):

‘’’
pts_src and pts_dst are numpy arrays of points
in source and destination images. We need at least
4 corresponding points.
‘’’
h, status = cv2.findHomography(pts_src, pts_dst)

‘’’
The calculated homography can be used to warp
the source image to destination. Size is the
size (width,height) of im_dst
‘’’

im_dst = cv2.warpPerspective(im_src, h, size)

Thanks

Thanks for this fairly detailed explanation. Please consider putting more information like this into the calibration article.

Do you mean this is done automatically by the plugin at start or do we have to implement this ourselves? So to get it working without code modification do we have to supply H values in the config or not?

Thanks