GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting

• 3060
• DeepStream 6.4
• Bug

I’m currently trying to upgrade to Deepstream 6.4.
Unfortunately, I’m encountering an error which is causing my solution to crash.
The error is as follows:
GLib (gthread-posix.c): Unexpected error from C library during ‘pthread_setspecific’: Invalid argument. Aborting

This is listed in the Release notes. So I followed the instructions to upgrade to glib2.76.

When I am installing glib2.76 from inside the container, it works fine (I can run my Deepstream app).
But when I am installing glib from Dockerfile, I’m still getting this error.

Here’s the glib2.76 part of my Dockerfile:

# Change into the extracted directory
WORKDIR /opt/nvidia/deepstream/deepstream/lib/glib-2.76.6

RUN meson _build 

RUN ninja -C _build    

RUN ninja -C _build install

I think it is the command

# Install GLib
RUN ninja -C _build install

which is not able to install properly… Cause this is the only one command that I need to re-execute when I instantiate a container…

Am I forgetting anything?

Thank you for help

1 Like

Have you upgrade your Ubuntu to 22.04? You can refer to the link to upgrade the ralated software.
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html#id6

Why should I update my ubuntu since I’m running Deepstream with Docker container

Some components inside docker may have dependencies on the host.

What exactly do you mean here? Could you describe your use case and dockerfile in detail?

I am currently using Ubuntu 22.04.3 LTS

This is my Dockerfile :

FROM nvcr.io/nvidia/deepstream:6.4-gc-triton-devel as base

# THIS IS NON RELATED TO THE ISSUE FROM HERE 

WORKDIR /opt/nvidia/deepstream/deepstream

RUN apt update -y

RUN apt install python3-pip python3-gst-1.0 libcairo2-dev libcairo2 gir1.2-gst-rtsp-server-1.0 pkg-config python3-dev protobuf-compiler  libyaml-cpp-dev libmpg123-0 libmpeg2-4 libavcodec58 -y
RUN python3.10 -m pip install --upgrade pip
RUN pip install Cython
RUN python3.10 -m pip install python-dotenv pika opencv-python pydantic confluent-kafka
RUN python3.10 -m pip install --force-reinstall typing-extensions==4.5.0

COPY lib ./lib

RUN python3.10 -m pip install /opt/nvidia/deepstream/deepstream/lib/pyds-1.1.10-py3-none-linux_x86_64.whl \
    /opt/nvidia/deepstream/deepstream/lib/pipelineConfigParser-0.0.3-py3-none-any.whl

# THIS IS NON RELATED TO THE ISSUE UNTIL HERE 

# Install necessary packages for building GLib (According to release notes : https://github.com/GNOME/glib/blob/2.76.6/INSTALL.md)
RUN apt install pkg-config python3-dev libffi-dev libmount-dev meson ninja-build wget libxml2-utils -y

WORKDIR /opt/nvidia/deepstream/deepstream/lib/glib-2.76.6 

# INSTALL GLIB 2.76.6
RUN meson _build 
RUN ninja -C _build    
RUN ninja -C _build install

WORKDIR /opt/nvidia/deepstream/deepstream

# First stage: application
FROM base as application

COPY deepstreamInterface ./deepstreamInterface

WORKDIR /opt/nvidia/deepstream/deepstream/deepstreamInterface

ENTRYPOINT ["python3.10"]
CMD ["run.py"]

When I start a container, my deepstream app (started via entrypoint run.py) shows this error :
GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument

But if I enter the container and execute manually ninja -C _build install, I can successfully execute run.py (without the error crashing the pipeline)

It seems to be a PATH problem or something else…

Do you plan to provide a docker image for Deepstream 6.4 with Glib 2.76.6 to fix this error?

Thanks

No. We follow the software version of Ubuntu by default. It will increase docker’s size installing other version of software.

Where is this release note and what exactly does your run.py do?

Here is the release :

run.py is a custom pipeline builder depending on a schema that we define.

OK. We will not provide a docker image for Deepstream 6.4 with Glib 2.76.6. You need to install that yourself. But we can analyze your specific problem together.

Can you provide a simple demo that we can reproduce the problem in our environment?

Here’s a simplified version of a Dockerfile:

FROM nvcr.io/nvidia/deepstream:6.4-triton-multiarch

RUN apt update -y && \
    apt install -y --no-install-recommends \
        python3-pip \
        apt-transport-https \
        ca-certificates && \
        update-ca-certificates 

RUN pip install https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/releases/download/v1.1.10/pyds-1.1.10-py3-none-linux_x86_64.whl


# Install necessary packages for building GLib
RUN apt-get update && apt-get install -y \
    pkg-config \
    python3-dev \
    libffi-dev \
    libmount-dev \
    meson \
    ninja-build \
    wget
WORKDIR /opt/nvidia/deepstream/deepstream/lib
RUN wget https://github.com/GNOME/glib/archive/refs/tags/2.76.6.tar.gz
RUN tar -xzvf 2.76.6.tar.gz
RUN rm -r glib-2.76.6/subprojects/gvdb
WORKDIR /opt/nvidia/deepstream/deepstream/lib/glib-2.76.6

# Configure the build
RUN meson _build
# Build GLib
RUN ninja -C _build
# Install GLib
RUN ninja -C _build install

ENTRYPOINT ["/bin/sh", "-c" , "/opt/nvidia/deepstream/deepstream-6.4/entrypoint.sh && tail -f /dev/null"] 

It installs everything you need to run Deepstream Python examples.

Once the container is instantiated, if you’re attached to it, you can run this command :

pkg-config --modversion glib-2.0

This provides the version of Glib installed for Ubuntu: 2.76.6

Finally, to analyze the installed version of Glib for Python:

python3
import gi
gi.require_version('GLib', '2.0')
from gi.repository import GLib
print(GLib.glib_version)

This indicates that the version is 2.71.3

This version does not allow me to run my Deepstream application without the error :
GLib (gthread-posix.c): Unexpected error from C library during ‘pthread_setspecific’: Invalid argument. Aborting

This error is also apparent when running the deeptream-test1 python sample.

Although it doesn’t prevent this sample from being executed, it does prevent me from executing my own solution. I can’t provide you with my solution publicly because it’s the property of the company I work for.

As this is an error that you already know about from the release notes, I hope I can get a solution from you.

I suppose it’s better to build PyGObject manually, specifying the version of Glib you want, but there’s no documentation providing any instructions.

If you can’t help me, I’ll turn to some contacts we have internally at Nvidia by referring this post.

Thanks in advance

You can build that yourself. Please refer to the link deepstream_python_apps. You can install the glib-2.76.6 first, than build the whl yourself.

I have tried with your docker file. The glib-2.76.6 is installed in the /usr/local/lib/x86_64-linux-gnu/libglib-2.0.so. But the default glib library is still in the following directory /usr/lib/x86_64-linux-gnu/libglib-2.0.so. So I think the pyobject still loads the default lib.

Hi,

I tried what you suggested but it is still the wrong version of glib that is used by pyobject…

Any other suggestions ?
Thanks

Could you refer to the link below to rm the old version first?
https://www.linuxfromscratch.org/blfs/view/svn/general/glib2.html

It didn’t help me solve my problem.

But to get back to the original problem, what I don’t understand is that once my container is launched, if I execute ninja -C _build install again, the error GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument disappears and everything works perfectly…
Unfortunately, even by activating the logs, I can’t figure out what changes in the environment when this command is executed…

So even the print(GLib.glib_version) is 2.71.3, after you launch the container, it works well. But if you don’t launch the container, it will report error. Is that right?

It works well only if I execute the commande ninja -C _build install inside the container.
Yes the printed version is 2.71.3 inside the container for Python

1 Like

Same here for C++, if I have it in the dockerfile I can start my app for once and then either I have to clear the Gstreamer cache or run sudo ldconfig inside the docker container.

OK. Can you provide a detailed method step by step and relevant simple source code so that we can reproduce this problem? If you don’t want to make your code public, you can message it to me(click my icon).

Thanks to all the information you provided, @yuweiw and @ThomaSpyrou,
I managed to have the solution working now.
Here are my observations thanks to you guys :

The installation of the new version of GLib is located in a “non-standard” PATH. As a result, the library is not correctly loaded and referenced by the “ldconfig” tool (which is responsible for locating libraries and creating appropriate symbolic links). Initially, “ldconfig” is supposed to manage a library located in a “non-standard” PATH, but its use with Docker seems not to work as expected.

Observation:
Running “ldconfig” in a container where the library is located in a “non-standard” PATH enables the library to be loaded and our Deepstream application to be run. However, when the container has just started (docker run), the “ldconfig” cache indicates that the library has not been loaded correctly.

To solve this problem, the solution was to move the new version of Glib to the place of the old version, which is in a “standard” PATH.

Dockerfile new line :

RUN rsync -av /usr/local/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/ 

Thank you both

3 Likes

Hi Simon

I have tried the same but still facing some issues, would it be possible for you to post part of your docker file just for the glib issue?

Hey spyroutommy,

Here is the part for GLib, I hope it will help :

# Install necessary packages for building GLib
RUN apt-get update && apt-get install -y \
    pkg-config \
    python3-dev \
    libffi-dev \
    libmount-dev \
    meson \
    ninja-build \
    wget

# Change into the extracted directory
WORKDIR /opt/nvidia/deepstream/deepstream/lib/glib-2.76.6

# Configure the build
RUN meson _build
# Build GLib
RUN ninja -C _build
# Install GLib
RUN ninja -C _build install

RUN rsync -av /usr/local/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/ 
1 Like