How to efficiently communicate for processes? And how to dynamic allocate resource like CPU to specific process?

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

I am developing a multi-process project on Xavier NX, which have processes for multi-sensor data collection, data fusion and inference by ML models. Currently I use torch.multiprocessing.Queue() to transmit data from the collection process to the processing process, but I found that it will have a fluctuating latency, normally about 20ms. It will become extremely high (more than 1000ms) especially when the CPU resource is limited.

Therefore, I wonder if there is any tool for multi-process communication? Or is there any better framework for my project? I used to try thread in thethreading library but the overall latency is worse due to the PIL in Python I think. Another question is that is any possible approach to allcoate resource CPU based on the current system status? For example, when the data collection latency is high, the system can automatically allocate more CPU to the data collection process. I notice that there are a lot of work on resource allocation in the mobile system community, but most of them are not open source.

Thank you so much!!

3

Help!!! Thank you so much!!!

4

Hi,

Do you need it with Python?

If C++ implementation is acceptable, please check our NvSCI library:

Thanks.

5

Well, maybe Python woule be better? Because I need to use Pytorch for deep learning models in my inference process.

6

Hi,

Unfortunately, we only have the C++ library for the IPC use case.
Thanks.

7

Is there any other implementation for continuous sensing and inference in parallel? Such as multi-process or multi-thread?

8

Hi,

Do you need this for both CPU and GPU?
For GPU, another option is the MPS from CUDA 12.5 which is available for the Orin series.

Thanks.

9

Is there any solution for CPU or GPU? And my device is Xavier NX, is there any available method?

10

Help!!! Thank you!

11

Help!!!

12

Hi,

We have both NvSci and MPS for the Orin series.
But it isn’t available for the older device.

Thanks.

13

So no any other solutions for Xavier?
Another question, is ROS work for my project?

14

Hi,

There are some ROS containers (tag: r35.x) available for Xavier below:

Thanks.