Streaming rendered AGX output

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Dear fellow developers,

Because of the pandemic we must work from home while the Nvidia Drive AGX is still on location. For better debugging and more user-friendly usage we want to make a web-interface for controlling the drive and see what the AGX get as input and given as an output. As an input we are using our own TCP server that gets its images from a virtual city. The stream is streaming to both the web-interface as well as the AGX. But now we also want to get the output of the AGX to our web application (including all the perception lines). Has this been done before? Is there any example code available? This would help us a lot.

At this moment we think we have 2 solutions:
1. Create a flatarray of the current image + perception lines and send it to our web-interface where the JPG encoding happens.
2. Create a JPG steam directly on the AGX so that we can use it on our web-interface without having to encode it on our client side

We prefer option 2 if possible. Can anyone help us?

Thanks in advanced and stay healthy!

Please provide the following info:
Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVEā„¢ Software 10.0 (Linux)
NVIDIA DRIVEā„¢ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVEā„¢ AGX Xavier DevKit (E3550)
NVIDIA DRIVEā„¢ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.4.0.7363
other

Host Machine Version
native Ubuntu 18.04
other

3

Hi @n.r.j.kat,

Are you using Roadrunner application, DriveWorks DRIVE Perception or any others?
Could you elaborate which part of this is using NVIDIA software?
Thanks!

4

We use Driveworks Path Perception. Instead of a hardware camera we implemented our own custom camera that fetches frames over the network. These frames are then used by the Driveworks API to make commands we can send back to the client application. Together with this data we would also like to send the frame back that has been processed with the lines drawn on top

5

Do you want to send back dwPathDetection objects and rendering in sample_path_perception application?
Are you asking about DriveWorks visualization module supports rendering to a file or reading back pixel data?

6

What we want is to paint dwPathDetection onto the frame then transfer the painted frame over the network

7

Please refer to below header files on your host sytem and related sample applications for this. Thanks!

/usr/local/driveworks/samples/src/framework/ScreenshotHelper.hpp
/usr/local/driveworks/targets/*64-Linux/include/dwvisualization/image/FrameCapture.h

8

Thanks! This is what we where looking for. Thanks, thanks, thanks! :)

1 Like