Deepstream 6.4 - GLib 2.76.6 installation broke TLS support

• 3060
• DeepStream Version 6.4
• Bug

I would like to update my Deepstream application to 6.4

To ensure compatibility, I’ve had to upgrade GLib, as recommended in the release notes. Incidentally, I have a post associated with this upgrade that has been resolved: https://forums.developer.nvidia.com/t/glib-gthread-posix-c-unexpected-error-from-c-library-during-pthread-setspecific-invalid-argument-aborting/277705

Unfortunately, in resolving the error I was experiencing, the installation of GLib 2.76.6 “broke” support for TLS.

I need TLS to read rtsps streams (rtsp over tls).

I’ve prepared some files for you to replicate the error:

  • Dockerfile :
FROM nvcr.io/nvidia/deepstream:6.4-gc-triton-devel as base

WORKDIR /opt/nvidia/deepstream/deepstream

RUN apt update && apt install -y \
    python3-pip libcairo2-dev libcairo2 pkg-config python3-dev protobuf-compiler libyaml-cpp-dev libmpg123-0 libmpeg2-4 libavcodec58 \
    # Install necessary packages for building GLib:
    pkg-config python3-dev libffi-dev libmount-dev meson ninja-build wget

RUN python3.10 -m pip install --upgrade pip && pip3 install \
    https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/releases/download/v1.1.10/pyds-1.1.10-py3-none-linux_x86_64.whl

WORKDIR /opt/nvidia/deepstream/deepstream/lib
RUN wget https://github.com/GNOME/glib/archive/refs/tags/2.76.6.tar.gz
RUN tar -xzvf 2.76.6.tar.gz
RUN rm -r glib-2.76.6/subprojects/gvdb

# Building GLib 2.76.6 (required for DS 6.4)
# Change into the extracted directory
WORKDIR /opt/nvidia/deepstream/deepstream/lib/glib-2.76.6
# Configure the build
RUN meson _build
# Build GLib
RUN ninja -C _build
# Install GLib
RUN ninja -C _build install
# Copy library files to "standard path"
RUN rsync -av /usr/local/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/

WORKDIR /opt/nvidia/deepstream/deepstream

# Third stage: dev
FROM base as dev

ENTRYPOINT ["/bin/bash", "-c", "tail -f /dev/null"]
CMD []
  • Execute container :
docker run --gpus all -it --rm --net=host --privileged -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.4 built_container
  • Enter the container

  • Add and run this python script to replicate error (you need to change for your own rtsps stream) :

import gi
gi.require_version("Gst", "1.0")
from gi.repository import Gst, GObject
import threading
import time 
import sys


def cb_newpad(element, element_src_pad, data):
	print("IN NEW PAD")
	caps = element_src_pad.get_current_caps()
	structure = caps.get_structure(0)
	name = structure.get_name()
	media_type = structure.get_value("media")
	print(f"Pad added: {media_type} & {name}")
	
	if media_type == "video" and name == "application/x-rtp":
		print("Here we go")
		sinkpad = data.get_static_pad("sink") #where data is depay element
		print("Source pad capabilities:", element_src_pad.query_caps(None).to_string())
		print("Sink pad capabilities:", sinkpad.query_caps(None).to_string())
		if element_src_pad.link(sinkpad)  != Gst.PadLinkReturn.OK:
			print('Elements could not be linked: 1. Exiting.', flush=True)
			sys.exit(-1)      


def on_error(bus, message):
	err, debug = message.parse_error()
	print(f"Error: {err}, {debug}")

def main():
	GObject.threads_init()
	Gst.init(None)

	# Iterate over all plugins
	for plugin in Gst.Registry.get().get_plugin_list():
		# Print plugin name
		print("Plugin:", plugin.get_name(), flush=True)
		# Iterate over all features of each plugin
		for feature in Gst.Registry.get().get_feature_list_by_plugin(plugin.get_name()):
			# Print feature name (this includes elements)
			print("  Feature:", feature.get_name(), flush=True)


	# Set the default debug threshold
	Gst.debug_set_default_threshold(3)

	pipeline = Gst.Pipeline.new("dstest-sr-pipeline")
	
	source = Gst.ElementFactory.make("rtspsrc", "rtsp-source")
	source.set_property("location", "rtsps://modeluser:rKJAhFa1GbR9@drio.app:5541/tec/SSB_cam1_h264")
	source.set_property("protocols", 4)
	
	depay_pre_decode = Gst.ElementFactory.make("rtph264depay", "h264-depay")
	
	parse = Gst.ElementFactory.make("h264parse", "parse")
	
	source.connect("pad-added", cb_newpad, depay_pre_decode)
	
	decoder = Gst.ElementFactory.make("nvv4l2decoder", "nvv4l2-decoder")

	convert = Gst.ElementFactory.make("nvvideoconvert", "convert")

	caps = Gst.Caps.from_string("video/x-raw,width=1920,height=1080,formate=I420")
	filter = Gst.ElementFactory.make("capsfilter", "filter")
	filter.set_property("caps", caps)

	queue = Gst.ElementFactory.make("queue", "queue")

	sink = Gst.ElementFactory.make("nveglglessink", "video-sink")
	sink.set_property("window-x", 0)
	sink.set_property("window-y", 0)
	sink.set_property("window-width", 1280)
	sink.set_property("window-height", 720)

	pipeline.add(source)
	pipeline.add(depay_pre_decode)
	pipeline.add(parse)
	pipeline.add(decoder)
	pipeline.add(convert)
	pipeline.add(filter)
	pipeline.add(queue)
	pipeline.add(sink)
   
	if not depay_pre_decode.link(parse):
		print('Could not link depay_pre_decode to parse. Exiting.')
		sys.exit(-1)

	if not parse.link(decoder):
		print('Could not link parse to decoder. Exiting.')
		sys.exit(-1)

	if not decoder.link(convert):
		print('Could not link decoder to convert. Exiting.')
		sys.exit(-1)

	if not convert.link(filter):
		print('Could not link convert to filter. Exiting.')
		sys.exit(-1)

	if not filter.link(queue):
		print('Could not link filter to queue. Exiting.')
		sys.exit(-1)

	if not queue.link(sink):
		print('Could not link queue to sink. Exiting.')
		sys.exit(-1)



	bus = pipeline.get_bus()
	bus.add_signal_watch()
	bus.connect("message::error", on_error)



	# Mettre le pipeline en lecture
	ret = pipeline.set_state(Gst.State.PLAYING)
	if ret == Gst.StateChangeReturn.FAILURE:
		print("Unable to set the pipeline to the playing state.", flush=True)
		exit(-1)

	# Attendre que le pipeline rencontre une erreur ou EOS
	print("Running...")
	main_loop = GObject.MainLoop()
	try:
		main_loop.run()
	except KeyboardInterrupt:
		pass
	finally:
		pipeline.set_state(Gst.State.NULL)
		main_loop.quit()

if __name__ == "__main__":
	main()
  • Here is the error :
0:00:00.569020744    43 0x7f0720000e50 ERROR                default gstrtspconnection.c:1102:gst_rtsp_connection_connect_with_response_usec: failed to connect: TLS support is not available
0:00:00.569033619    43 0x7f0720000e50 ERROR                rtspsrc gstrtspsrc.c:5238:gst_rtsp_conninfo_connect:<rtsp-source> Could not connect to server. (Generic error)

Could you refer to the FAQ to update the glib and don’t use the rsync command in the docker file. You can just set the GI_TYPELIB_PATH env.

Thanks for you answer,
I followed the instructions provided at the link you provided.

After the installation, I have nothing installed at this path :

export GI_TYPELIB_PATH=/usr/local/lib/x86_64-linux-gnu/girepository-1.0

So this is the output I got from python3 glib_version.py :

2 71 3

The installations of girepository-1.0 I have are both 2.71.3 and located at :

/usr/lib/x86_64-linux-gnu/girepository-1.0
/usr/lib/girepository-1.0

This new version of GLib seems to be loaded by running ldconfig. But that still doesn’t solve my problem, I still get the error :

default gstrtspconnection.c:1102:gst_rtsp_connection_connect_with_response_usec: failed to connect: TLS support is not available

Voici le Dockerfile :

FROM nvcr.io/nvidia/deepstream:6.4-gc-triton-devel as base 

WORKDIR /opt/nvidia/deepstream/deepstream

RUN apt update && apt install -y \
    python3-pip libcairo2-dev libcairo2 pkg-config python3-dev protobuf-compiler libyaml-cpp-dev libmpg123-0 libmpeg2-4 libavcodec58 \
    # Install necessary packages for building GLib:
    pkg-config python3-dev libffi-dev libmount-dev meson ninja-build wget

RUN python3.10 -m pip install --upgrade pip && pip3 install packaging meson==1.2.0 \
    https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/releases/download/v1.1.10/pyds-1.1.10-py3-none-linux_x86_64.whl

WORKDIR /opt/nvidia/deepstream/deepstream/lib
RUN wget https://github.com/GNOME/glib/archive/refs/tags/2.79.1.tar.gz
RUN tar -xzvf 2.79.1.tar.gz

# Building GLib 2.79.1 (required for DS 6.4)
# Change into the extracted directory
WORKDIR /opt/nvidia/deepstream/deepstream/lib/glib-2.79.1/
RUN rm -r subprojects/gvdb
# Configure the build
RUN meson _build
# Build GLib
RUN ninja -C _build
# Install GLib
RUN ninja -C _build install
# Copy library files to "standard path"
#RUN rsync -av /usr/local/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/ 

WORKDIR /opt/nvidia/deepstream/deepstream

# Third stage: dev
FROM base as dev

ENTRYPOINT ["/bin/bash", "-c", "tail -f /dev/null"]
CMD []

This is werid. Could you try this docker: nvcr.io/nvidia/deepstream:6.4-triton-multiarch?

It’s seems like a glib-networking issue. Can you confirm that you installed that module?

Same issue with deepstream:6.4-triton-multiarch (nothing installed in the path)

Output for glib-networking (I did not install but seems to be installed by default) :

dpkg -l | grep glib-networking
ii  glib-networking:amd64                  2.72.0-1                                amd64        network-related giomodules for GLib
ii  glib-networking-common                 2.72.0-1                                all          network-related giomodules for GLib - data files
ii  glib-networking-services               2.72.0-1                                amd64        network-related giomodules for GLib - D-Bus services

Could you confirm that Gstreamer supports your rtsp source?

rtsps://modeluser:rKJAhFa1GbR9@drio.app:5541/tec/SSB_cam1_h264

You can run a simple piepline in our docker directly without glib upgrade first.

gst-launch-1.0 rtspsrc location=<your rtsp source> ! rtph264depay ! h264parse ! 'video/x-h264,stream-format=byte-stream' ! filesink location=test.h264

Yes it is supporting before update of glib

Using this image :

docker run --gpus all -it --rm --net=host --privileged -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-6.4 nvcr.io/nvidia/deepstream:6.4-gc-triton-devel

Could you try the FAQ in the docker directly without docker file? Or could you share this rtsp source to us?

I already tried without the dockerfile.

I can’t share you this rtsps source but as I already mentioned, it is working properly before the installation of Glib new version.
So this installation is breaking some functionalities of rtspsrc element.

By the way you can probably create your own rtsps with mediamtx. Or if you have some ip cameras, most of them have rtsps that can be enabled.

Could you please escalate this issue?

Thanks

I upgrade the glib according to the steps of the FAQ. Then create a rtsps server by referring the test-video.c.
I run your demo code with the rtsp source generated. It works normally without any error.
Did you refer to the FAQ and ungrade the glib step by step?

Well,
If I follow your steps, it is indeed working.
BUT : This is because of this command :

export GI_TYPELIB_PATH=/usr/local/lib/x86_64-linux-gnu/girepository-1.0

Which is doing NOTHING.
So you’re still using GLib 2.76.6.
That’s why it is working.
If you execute ldconfig , you won’t be able to run the code, because you’ll be using the new installed version.

If it is working after those steps,
Could you please provide me all your instructions in detail with logs included?
Could you please provide me your dockerfile.
Also could you please provide the steps on how you created your rtsps stream with testvideo.c

So I can replicate.

Thanks

No. I add the code below in your source code, and the printout is 2 79 1.

print(GLib.MAJOR_VERSION, GLib.MINOR_VERSION, GLib.MICRO_VERSION)

I use the dockerfile attached on the FAQ: nvcr.io/nvidia/deepstream:6.4-triton-multiarch.

Just download the source code, define the WITH_TLS macro in the test-video.c, make and run ./test-video.
Export GIO_MODULE_DIR env on your host.

export GIO_MODULE_DIR=/usr/lib/x86_64-linux-gnu/gio/modules/

Hi,

I still have the same error…

What is your output if you use this dockerfile ?

FROM nvcr.io/nvidia/deepstream:6.4-triton-multiarch

WORKDIR /opt/nvidia/deepstream/deepstream

RUN apt update && apt install -y \
    python3-pip libcairo2-dev libcairo2 pkg-config python3-dev protobuf-compiler libyaml-cpp-dev libmpg123-0 libmpeg2-4 libavcodec58 \
    pkg-config python3-dev libffi-dev libmount-dev meson ninja-build wget

RUN python3.10 -m pip install --upgrade pip && pip3 install packaging meson==1.2.0 \
    https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/releases/download/v1.1.10/pyds-1.1.10-py3-none-linux_x86_64.whl

WORKDIR /opt/nvidia/deepstream/deepstream/lib
RUN wget https://github.com/GNOME/glib/archive/refs/tags/2.79.1.tar.gz
RUN tar -xzvf 2.79.1.tar.gz

WORKDIR /opt/nvidia/deepstream/deepstream/lib/glib-2.79.1/
RUN rm -r subprojects/gvdb
RUN meson _build
RUN ninja -C _build
RUN ninja -C _build install

WORKDIR /opt/nvidia/deepstream/deepstream

# Set environment variable
ENV GI_TYPELIB_PATH=/usr/local/lib/x86_64-linux-gnu/girepository-1.0

# Run python script 
RUN printf "import gi\ngi.require_version('GLib', '2.0')\nfrom gi.repository import GLib\nprint(GLib.MAJOR_VERSION, GLib.MINOR_VERSION, GLib.MICRO_VERSION)" > glib_version.py

CMD [ "python3", "./glib_version.py"]

Could you add some codes in your Dockerfile like below and try?

...
+RUN apt update && apt install -y \
libgirepository1.0-dev

+ENV GIO_MODULE_DIR=/usr/lib/x86_64-linux-gnu/gio/modules/
...

Also please do not install only the whl files for python binding. You can just run the script in our docker: /opt/nvidia/deepstream/deepstream/user_deepstream_python_apps_install.sh.

./user_deepstream_python_apps_install.sh --version 1.1.10

Hello,

Thank you for your reply. It seems to have solved my problem.
I now have another error with my custom deepstream python application.

I think these imports are responsible:

ENV GI_TYPELIB_PATH=/usr/local/lib/x86_64-linux-gnu/girepository-1.0
ENV GIO_MODULE_DIR=/usr/lib/x86_64-linux-gnu/gio/modules/

Here is the gdb backtrace :

2024-02-07 05:37:43,097 - DEBUG - pipelineUtils - Decodebin child added: source
[New Thread 0x7fff217fe640 (LWP 124)]
[New Thread 0x7fff20ffd640 (LWP 125)]
[New Thread 0x7ffeffffd640 (LWP 126)]
[New Thread 0x7ffeff7fc640 (LWP 127)]
[Detaching after fork from child process 128]

Thread 44 "pool-python3" received signal SIGABRT, Aborted.
[Switching to Thread 0x7ffeff7fc640 (LWP 127)]
0x00007ffff7ce49fc in pthread_kill () from /usr/lib/x86_64-linux-gnu/libc.so.6
(gdb) backtrace 
#0  0x00007ffff7ce49fc in pthread_kill () at /usr/lib/x86_64-linux-gnu/libc.so.6
#1  0x00007ffff7c90476 in raise () at /usr/lib/x86_64-linux-gnu/libc.so.6
#2  0x00007ffff7c767f3 in abort () at /usr/lib/x86_64-linux-gnu/libc.so.6
#3  0x00007ffff2b53687 in  () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff2b5ea5b in __gxx_personality_v0 () at /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff5db9fe9 in __libunwind_Unwind_Resume () at /usr/lib/x86_64-linux-gnu/libunwind.so.8
#6  0x00007fffe2a7386d in  () at /usr/lib/x86_64-linux-gnu/libproxy.so.1
#7  0x00007fffe2a7c827 in px_proxy_factory_get_proxies () at /usr/lib/x86_64-linux-gnu/libproxy.so.1
#8  0x00007fffe7a06827 in  () at /usr/lib/x86_64-linux-gnu/gio/modules/libgiolibproxy.so
#9  0x00007ffff7491194 in  () at /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
#10 0x00007ffff77456b4 in  () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#11 0x00007ffff7742a51 in  () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#12 0x00007ffff7ce2ac3 in  () at /usr/lib/x86_64-linux-gnu/libc.so.6
#13 0x00007ffff7d74a40 in  () at /usr/lib/x86_64-linux-gnu/libc.so.6

Thanks!

Could you provide a simple pipeline including the source to reproduce your problem?
It may be a compatibility issue between the gio module and ubuntu versions. You can also try set the GIO_MODULE_DIR to /usr/local/lib/x86_64-linux-gnu/gio.

When using GIO_MODULE_DIR=/usr/local/lib/x86_64-linux-gnu/gio, my app is working fine but unfortunately, it breaks the TLS support again… Note that there is nothing in /usr/local/lib/x86_64-linux-gnu/gio
When using GIO_MODULE_DIR=/usr/local/lib/x86_64-linux-gnu/gio :

0:00:00.112493095   115 0x55caad55c0c0 ERROR                default gstrtspconnection.c:1102:gst_rtsp_connection_connect_with_response_usec: failed to connect: TLS support is not available
0:00:00.112508290   115 0x55caad55c0c0 ERROR                rtspsrc gstrtspsrc.c:5238:gst_rtsp_conninfo_connect:<rtsp-source> Could not connect to server. (Generic error)

UPDATE :
Using ENV GIO_MODULE_DIR=/usr/lib/x86_64-linux-gnu/gio/modules/ is working fine IF I remove the shared object libgiolibproxy.so located in /usr/lib/x86_64-linux-gnu/gio/modules/libgiolibproxy.so

So my questions are:
Do you have any idea why this shared object is causing problems?
Do you think removing it is risky, or should it not affect the rest? Is it a good/bad idea?

Thanks for your help, I appreciate it.

Hi,
Any news on that?
Thanks