Multiple PGIE in different GPUs with NEW NVSTREAMMUX

[juan.isern95]

Hi everyone,

- Hardware Platform (Jetson / GPU): RTX2080TI
- DeepStream Version: 6.2 (python bindings)
- TensorRT Version: 8.5.2-1+cuda11.8
- NVIDIA GPU Driver Version (valid for GPU only): 530.30.02
- Issue Type( questions, new requirements, bugs): Question

I’m trying to use the two GPUs of my server (2xA30) to balance the workload of my pipeline, which has two primary models (PGIE) in the same pipeline and supports several sources and uses the NEW NVSTREAMMUX.

I have come up with two different options:

1. Select a different gpu-id for each PGIE configuration. However some configuration problem appears and the model on GPU 1 does not start, this error appears in the log:

0:02:03.874800967 1 0x7f5cc40114c0 WARN nvinfer gstnvinfer.cpp:1480:gst_nvinfer_process_full_frame:<primary-inference-9> error: Memory Compatibility Error:Input surface gpu-id doesnt match with configured gpu-id for element, please allocate input using unified memory, or use same gpu-ids OR, if same gpu-ids are used ensure appropriate Cuda memories are used

note: I have configured all the pipeline elements to use CUDA unified memory

2. Divide the pipeline in 2 each with a PGIE model. However, the NEW NVSTREAMMUX does not have a gpu-id parameter to select where to place this component.

In addition, I prefer the first of these two options, as it gives more flexibility to my solution, as I have it programmed.

Best regards

[miguel.taylor]

Hi @juan.isern95

We came across a different issue involving the transfer of PGIE detections from an RGB stream to an infrared stream. To address this challenge, we designed a custom element that effectively resolved the problem. While I’m uncertain if there’s an alternative solution to your issue that doesn’t involve crafting a custom element, I am confident that a custom element modeled after our solution could effectively resolve your concern.

The custom element, named metatransfer, is a GStreamer element based on GstAggregator. It operates by extracting all DeepStream metadata from a GStreamer buffer received through its metapad and transferring it to the buffers received through its buffpad. The resultant output buffer comprises the data from buffpad along with the metadata from metapad.

Below is a diagram illustrating how this element would solve your issue:

The core of the element is saving each buffer from the meta pad into self->last_meta_buffer and copying the meta into the buffpad buffer with the following code:

      gpointer state = NULL;
      GQuark nvdsmeta_quark = g_quark_from_static_string (NVDS_META_STRING);
      GstMeta *gst_meta;
      gboolean has_meta = FALSE;
      while ((gst_meta =
              gst_buffer_iterate_meta (self->last_meta_buffer, &state))) {
        /* DeepStream meta */
        if (gst_meta_api_type_has_tag (gst_meta->info->api, nvdsmeta_quark)) {
          GstMetaTransformCopy copy_data = { FALSE, 0, -1 };
          const GstMetaInfo *info = gst_meta->info;
          info->transform_func (buf, gst_meta, self->last_meta_buffer,
              _gst_meta_transform_copy, &copy_data);
          has_meta = TRUE;
        }
      }

I hope this helps with your issue. Let me know if you have any questions.

[yuweiw]

The 1st option, could you show all your pipeline to us? Or could you reproduce that with our demo? If we can reproduce that problem in our environment, we can analyze that quickly.
The 2nd option, the new nvstreammux will not proess the images, so there is no need to set gpu id.

[juan.isern95]

Hello again,

Regarding the first option, I share you privately the DsApp (Deepstream APP) class in python that implements my pipeline. I have checked this answer:

Multiple GPUs appear and GPUs are not used

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks There are two ways: Assign every DeepStream element in the same pipeline to the same GPU. E.G. nvv4l2decoder, nvstreammux, nvinfer, nvstreamdemux, nvvideoconvert,… If you want some elements to work in GPU0 while some other elements work in GPU1 in the same pipeline, the elements who allocates buffers should use “nvbuf-mem-cuda…

Supposedly the feature I am looking for should be implementable, according to the previous answer. Also, in other threads I have read that “surface-gpu-id just set by decoder” (Memory Compatibility Error:Input surface gpu-id doesn't match with configured gpu-id for element - #14 by junshengy). However, if unified memory is used it should be independent of which GPU the pipeline elements are assigned to, no?

Regarding the second option, I would like to know in which way I can modify an example code (deepstream_test_3.py) with the NEW NVSTREAMUX instead of the old one and have it used entirely on GPU 1? I understand that the new nvstreammux does not process the image, but the decoder does, and somehow the GPU on which it runs must be defined, am I wrong?

Best regards

[yuweiw]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Can you simplify your demo code or directly use our demo deepstream-test3 to analyze this problem?
1st option:You should set the gpu-id and unified memory type to every plugin with relevant parameters.
2nd option:Just set the env variable: export CUDA_VISIBLE_DEVICES=<gpu-id>