Sharing cuda memory between containers x86 and jetson

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
,
1

Hi,

In my company we deploy our software in a very decoupled manner based on several docker containers communicating. A part of the workflow involves video decoding, video resizing and inference with a triton server.
Looks like a perfect use case for using deepstream but we actually need a service to manage frames from the decoding to the inference. So far we simply use shared memory which of course creates a big overhead because of all the copies done.

We need something that would answer these demands:

  • should be gpu shared memory
  • one writer, several readers
  • scalable for video streams (up to 60 fps)
  • should be able to be read by different containers
  • should work on gpu servers and jetsons
  • bonus, deepstream could actually directly write the frames into it

What would be the best approach ?
So far the information I have gathered are:

  • nvIPC is not available on jetson
  • nvsci could be a solution ?
  • using cuMemExportToShareableHandle and cuMemImportFromShareableHandle ?

But honestly the lack of examples of usage of the two last propositions are making them really hard to grasp.

Thanks in advance!

4

Hi,

If you need to share GPU memory with two processes, please give nvsci a try.
But we don’t have an available example for nvsci+inference.

Maybe another possible alternative is to share the streaming video via RTSP.
If this works for you, we already have some Deepstream samples for this use case:

Thanks.

1 Like
6

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.