Failed to initialize nvstreammux with a MIG 10 pod under openshift

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GPU
• DeepStream Version 6.3
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only) 550.90.07
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hello,

I’m using deepstream docker image nvcr.io/nvidia/deepstream:6.3-gc-triton-devel (I’m unable to use 7.0 or 7.1 because of a problem with engine creation) inside an openshift pod using a 10GB MIG, when I try to run python sample test 2 the initialization of streammux fails:

root@devel-ds-6:/opt/nvidia/deepstream/deepstream/sources/deepstream_python_apps/apps/deepstream-test2# python ./deepstream_test_2.py /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
Creating Pipeline

Creating Source

Creating H264Parser

Creating Decoder

Unable to create Nvv4l2 Decoder
Unable to create NvStreamMux
Unable to create pgie
Unable to create tracker
Unable to make sgie1
Unable to make sgie2
Unable to make sgie3
Unable to create nvvidconv
Creating EGLSink

Playing file /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
Traceback (most recent call last):
File “./deepstream_test_2.py”, line 351, in
sys.exit(main(sys.argv))
File “./deepstream_test_2.py”, line 251, in main
streammux.set_property(‘width’, 1920)
AttributeError: ‘NoneType’ object has no attribute ‘set_property’

This is my nvidia-smi output:

root@devel-ds-6:/opt/nvidia/deepstream/deepstream/sources/deepstream_python_apps/apps/deepstream-test2# nvidia-smi

Fri Dec 13 10:30:56 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.90.07              Driver Version: 550.90.07      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA A100 80GB PCIe          On  |   00000000:98:00.0 Off |                   On |
| N/A   38C    P0             76W /  300W |                  N/A   |     N/A      Default |
|                                         |                        |              Enabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| MIG devices:                                                                            |
+------------------+----------------------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |                     Memory-Usage |        Vol|      Shared           |
|      ID  ID  Dev |                       BAR1-Usage | SM     Unc| CE ENC DEC OFA JPG    |
|                  |                                  |        ECC|                       |
|==================+==================================+===========+=======================|
|  0    9   0   0  |              13MiB /  9728MiB    | 14      0 |  1   0    0    0    0 |
|                  |                 0MiB / 16383MiB  |           |                       |
+------------------+----------------------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

test3 fails also:

root@devel-ds-6:/opt/nvidia/deepstream/deepstream/sources/deepstream_python_apps/apps/deepstream-test3# python deepstream_test_3.py -i file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
{‘input’: [‘file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264’], ‘configfile’: None, ‘pgie’: None, ‘no_display’: False, ‘file_loop’: False, ‘disable_probe’: False, ‘silent’: False}
Creating Pipeline

Creating streamux

Unable to create NvStreamMux
Traceback (most recent call last):
File “deepstream_test_3.py”, line 487, in
sys.exit(main(stream_paths, pgie, config, disable_probe))
File “deepstream_test_3.py”, line 226, in main
pipeline.add(streammux)
File “/usr/lib/python3/dist-packages/gi/overrides/Gst.py”, line 64, in add
if not Gst.Bin.add(self, arg):
TypeError: Argument 1 does not allow None as a value

thanks.

I saw a possible solution by removing ~/.cache/gstreamer-1.0/registry.x86_64.bin
after doing that I see the following error:

root@devel-ds-6:/opt/nvidia/deepstream/deepstream/sources/deepstream_python_apps/apps/deepstream-test2# python ./deepstream_test_2.py /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
Creating Pipeline

Creating Source

Creating H264Parser

Creating Decoder

max_fps_dur 8.33333e+06 min_fps_dur 2e+08
Creating EGLSink

Playing file /opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264
Traceback (most recent call last):
File “./deepstream_test_2.py”, line 351, in
sys.exit(main(sys.argv))
File “./deepstream_test_2.py”, line 251, in main
streammux.set_property(‘width’, 1920)
TypeError: object of type GstNvStreamMux' does not have property width’

Are you using the new nvstreammux? The new nvstreammux dose not have width and height properties.

Also if you are using DS6.3, please use the compatible gpu version 525.125.06.

And are you using the openshift on a Red Hat OS? DeepStream does not currently have a supported version oon the Red Hat.

Width and height is a just a symptom of the problem, the problem is that the nvstreammux was not created. Downgrading the driver is not an option right now, many services depend on that setup. We are using a docker based on nvcr.io/nvidia/deepstream:6.3-gc-triton-devel so it’s based on Ubuntu, not RedHat.

thanks

Because our DeepStream depends on a specific driver version, you can either lower your driver version or upgrade the DeepStream. Otherwise, some unusual issues may occur. Please refer to our dGPU model Platform and OS Compatibility.