How to use a custom deep learning network on Drive PX2?

Autonomous Vehicles DRIVE PX 2 General
1

Hello,
I am trying to use my own CNN network on Drive PX2 with Sekonix GMSL cameras.
However, I couldn’t find related documents and tutorials.

How can I capture the images from GMSL cameras?

Based on the documents (docs.nvidia.com), the images are captured to NvMdeia images.
And how can I use the NvMedia images as an input of my CNN network?

Finally, is it possible using pytorch framework on Drive Px2?

Thank you in advance.

2

Hi,

The document is within driveworks sample which is located at /usr/local/driveworks/doc/.

You can open a GMSL camera with ./sample_camera_gmsl:
►DriveWorks Samples
►Camera Sensor Samples
►GMSL Camera Capture Sample

To run a customized CNN, you can check ./sample_object_detector sample:
►DriveWorks Samples
►Deep Neural Network (DNN) Samples
►Basic Object Detector Sample

Thanks.

3

I’ve looked at that sample and I’ve been having trouble figuring out how to modify it with my own deep learning models. Let’s say, for example, I wish to use a saved .uff or .caffemodel model to inference with the Drive PX’s live cameras, how would I change that code to accomplish it?