I’ve been trying to containerize an application that uses GStreamer/DeepStream. Unfortunately, the behavior is radically different when running inside Docker versus on bare metal. The issue seems to have persisted across DeepStream 7.1 → 8.0, even though my colleagues have successfully used RTX 5060 with DeepStream 7.1.
A particularly notable message that appears in all Docker installations (but never on bare metal) is: Opening in BLOCKING MODE
This occurs whenever I use NVIDIA’s V4L2 encoder/decoder plugins. From my research, it seems Docker might be forcing synchronous access to the GPU/encoder hardware, which would explain why high-load pipelines stall.
I previously encountered similar issues, but I stopped debugging since the older version was said to be officially unsupported and as such it would be hard to seek support from the forums.
However i have now changed to 8.0 and the same overall issue remains: No video output alongside with a massive difference of resource consumption (somewhere in the pipeline it will cause the video frames to be dropped even though inference, etc runs of it)
I’ve narrowed it down to a generic RTSP stream example which i believe if solved or replaced could make me solve the other issues:
GST_DEBUG=2 gst-launch-1.0 rtspsrc location=rtsp://<generic_rtsp_server_simulator>:8554/cam1 user-id= user-pw= protocols=4 tcp-timeout=0 ! queue ! rtph264depay ! queue ! h264parse ! queue ! nvv4l2decoder ! queue ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! shmsink socket-path=/dev/shm/shm_0_joao wait-for-connection=false sync=false
Then to fetch the shm and visualize the result:
gst-launch-1.0 shmsrc socket-path=/dev/shm/shm_0_joao ! h264parse ! avdec_h264 ! videoconvert ! autovideosink
On Bare metal: Everything works as expected; autovideosink plays the video as desired
On Docker: The producer appears to stall. Running the client under sudo su(due to the container running under root) forces the producer to block; wait-for-connection=true is required to see output (not viable for my application) and worse its required for the client to connect in a short time of the producer launching for some reason.
This was tested in various setups(5060s on old drivers, 5060 on new drivers, 5060 on docker vs non docker, 2060 on old+ new drivers on docker and even 5060 ti).
But ill list only two as they are the most consistent(every single one displays the same issue when trying docker vs none docker but some have either some plugins broke requiring me to go around it and possibly making this forum more confusing than it already is) Ill list them in order as Machine 1 and Machine 2
• Hardware Platform (Jetson / GPU)
Machine 1: rtx 2060(12Gb), Machine 2: rtx 5060
• DeepStream Version
Machine 1: 8.0 from nvcr.io/nvidia/deepstream:8.0-gc-triton-devel, Machine 2: “Version: 7.1.0 DATE: Fri Oct 4 14:27:32 UTC 2024”.
• TensorRT Version
MACHINE 1: tensorrt-dev 10.9.0.34-1+cuda12.8, Machine 2: tensorrt 10.12.0.36-1+cuda12.9
• NVIDIA GPU Driver Version (valid for GPU only)
MACHINE 1: 580.95.05, Machine 2: 575.64.05
• Issue Type( questions, new requirements, bugs)
bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
GST_DEBUG=2 gst-launch-1.0 rtspsrc location=rtsp://<generic_rtsp_server_simulator>:8554/cam1 user-id= user-pw= protocols=4 tcp-timeout=0 ! queue ! rtph264depay ! queue ! h264parse ! queue ! h264parse ! queue ! nvv4l2decoder ! queue ! nvvideoconvert ! nvv4l2h264enc ! h264parse ! shmsink socket-path=/dev/shm/shm_0_joao wait-for-connection=false sync=false
—(then using the same machine):
gst-launch-1.0 shmsrc socket-path=/dev/shm/shm_0_joao ! h264parse ! avdec_h264 ! videoconvert ! autovideosink
When using docker, since user is root i run the second command outside under sudo su, yet the server, docker side seems frozen forcing me to do wait-for-connection=true to even see the output which apparently on our application cant be done(not to mention that resource wise the anomaly persists) when using a apt based installation of tensorrt+deepstream the video opens as normal even with wait-for-connection=false
Inside docker a strange message appears: “Opening in BLOCKING MODE”, apparently whenever nvv4l2decoder is used, other encoding related plugins also display this message
while on the baremetal installation:
”Failed to query video capabilities: Invalid argument”
these can be found when doing gst-inspect to the plugin on device-name
Example:
device-name : Name of the device
flags: Failed to query video capabilities: Invalid argument
readable
String. Default: “”
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
A generic rtsp h264 stream server with some kind of video(in this case the origin is the same ip)
For docker i used the following as to provide an easy troubleshooting without having to provide code, configs, etc…:
docker run --gpus all -it --rm --runtime=nvidia --network=host --privileged -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-8.0 nvcr.io/nvidia/deepstream:8.0-gc-triton-devel