Image Sensor Capture Best Practice

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after creating this topic):
Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.6 and DriveWorks 4.0
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.8.3.10426
other

Host Machine Version
native Ubuntu 18.04
other

I’m looking for best practice advice. I’m concerned mainly with power consumption and effective processor utilization. I didn’t find details on this in the NvMedia documentation, the SIPL documentation, or sample source code.

I’m bringing camera data in from 11 cameras to a Drive AGX Xavier. This data is then published to an AI algorithm. I noticed in the documentation for NvMedia, the getImage function is a blocking function. I presume this means that the function does not return until a new frame has arrived since the last call. What is the best practice for pulling this data in without locking the processor from the interpretation of other sensor data?

Should each image sensor processing be launched on its own thread prior to a call to this function? What are other concerns to keep in mind?

Below is a snippet from the getImage function documentation:

image
image975×253 92.2 KB

Any help is appreciated
Thanks!

3

Dear @michael.honey,
Did you check DW sample_camera sample for your use case? You need to update the rig file as per your need.

4

Yes. sample_camera executes a for loop that iterates over each camera for every frame render. However, this application is not processing the image data beyond changing its format and rendering it. This approach doesn’t seem well-suited to maximizing useful computation and minimizing power-usage. Is there a way to receive something like an event driven callback that can trigger computation?

Thanks

5

Hi @michael.honey
Please check dwSensorCamera_getImageAsync() and the camera module document, and see if it helps your case. Thanks.

6

I’ll dwSensorCamera_getImageAsync() experiment with that. Thank you VickNV!