Some question about test4 using kafka

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) GeForce GTX 1050
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 7.0
• NVIDIA GPU Driver Version (valid for GPU only) 440.100

Hi,

We are running test4 using kafka, we followed README in test4 but error occured, we use command:

deepstream-test4-app -i /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4 -p /opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_kafka_proto.so --conn-str= “add;port;test”

and it returned:

Now playing: /opt/nvidia/deepstream/deepstream-5.0/samples/streams/sample_1080p_h264.mp4
Running…
ERROR from element nvmsg-broker: Could not configure supporting library.
Error details: gstnvmsgbroker.c(332): gst_nvmsgbroker_start (): /GstPipeline:dstest4-pipeline/GstNvMsgBroker:nvmsg-broker:
unable to connect to broker library

Did you follow sources/libs/amqp_protocol_adaptor/README to setup amqp server?

Well, we are trying to use KAFKA instead of AMQP, shall we follow sources/libs/amqp_protocol_adaptor/README to setup amqp server?

Actually, we followed sources/libs/kafka_protocol_adaptor/README, and we run ./test_sample_proto_sync, it shows:

(base) xxx:/opt/nvidia/deepstream/deepstream-5.0/sources/libs/kafka_protocol_adaptor$ ./test_kafka_proto_sync
Refer to nvds log file for log output
send [0] completed
send [1] completed
send [2] completed
send [3] completed
send [4] completed
Disconnecting in 3 secs

And we run ./test_sample_proto_async, it shows:

(base) xxx:/opt/nvidia/deepstream/deepstream-5.0/sources/libs/kafka_protocol_adaptor$ ./test_kafka_proto_async
Refer to nvds log file for log output
/opt/nvidia/deepstream/deepstream-NVDS_VERSION/lib/libnvds_kafka_proto.so: cannot open shared object file: No such file or directory
unable to open shared library

We tried to make it again and found when run: ./configure , a lot of checking failed:
using cache file config.cache
checking for OS or distribution… ok (Ubuntu)
checking for C compiler from CC env… failed
checking for gcc (by command)… ok (cached)
checking for C++ compiler from CXX env… failed
checking for C++ compiler (g++)… ok (cached)
checking executable ld… ok (cached)
checking executable nm… ok (cached)
checking executable objdump… ok (cached)
checking executable strip… ok (cached)
checking for pkgconfig (by command)… ok (cached)
checking for install (by command)… ok (cached)
checking for PIC (by compile)… ok (cached)
checking for GNU-compatible linker options… ok (cached)
checking for GNU linker-script ld flag… ok (cached)
checking for __atomic_32 (by compile)… ok (cached)
checking for __atomic_64 (by compile)… ok (cached)
checking for socket (by compile)… ok (cached)
parsing version ‘0x000b0401’… ok (0.11.4)
checking for libpthread (by pkg-config)… failed
checking for libpthread (by compile)… ok (cached)
checking for zlib (by pkg-config)… ok
checking for zlib (by compile)… ok (cached)
checking for libcrypto (by pkg-config)… failed
checking for libcrypto (by compile)… failed (disable)
checking for liblz4 (by pkg-config)… failed
checking for liblz4 (by compile)… failed (disable)
checking for libssl (by pkg-config)… failed
checking for libssl (by compile)… failed (disable)
checking for libsasl2 (by pkg-config)… failed
checking for libsasl2 (by compile)… failed (disable)
checking for libsasl (by pkg-config)… failed
checking for libsasl (by compile)… failed (disable)
checking for crc32chw (by compile)… ok (cached)
checking for regex (by compile)… ok (cached)
checking for librt (by pkg-config)… failed
checking for librt (by compile)… ok (cached)
checking for strndup (by compile)… ok (cached)
checking for strerror_r (by compile)… ok (cached)
checking for libdl (by pkg-config)… failed
checking for libdl (by compile)… ok (cached)
checking for pthread_setname_gnu (by compile)… ok (cached)
checking for nm (by env NM)… ok (cached)
checking for python (by command)… ok (cached)
Generated Makefile.config
Generated config.h

Configuration summary:
prefix /usr/local
ARCH x86_64
CPU generic
GEN_PKG_CONFIG y
ENABLE_DEVEL n
ENABLE_VALGRIND n
ENABLE_REFCNT_DEBUG n
ENABLE_SHAREDPTR_DEBUG n
ENABLE_LZ4_EXT y
ENABLE_SSL y
ENABLE_SASL y
MKL_APP_NAME librdkafka
MKL_APP_DESC_ONELINE The Apache Kafka C/C++ library
MKL_DISTRO Ubuntu
SOLIB_EXT .so
CC gcc
CXX g++
LD ld
NM nm
OBJDUMP objdump
STRIP strip
CPPFLAGS -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wcast-align
PKG_CONFIG pkg-config
INSTALL install
LIB_LDFLAGS -shared -Wl,-soname,(LIBFILENAME) LDFLAG_LINKERSCRIPT -Wl,--version-script= RDKAFKA_VERSION_STR 0.11.4 MKL_APP_VERSION 0.11.4 LIBS -lpthread -lz -lrt -ldl CFLAGS CXXFLAGS -Wno-non-virtual-dtor SYMDUMPER (NM) -D
exec_prefix /usr/local
bindir /usr/local/bin
sbindir /usr/local/sbin
libexecdir /usr/local/libexec
datadir /usr/local/share
sysconfdir /usr/local/etc
sharedstatedir /usr/local/com
localstatedir /usr/local/var
libdir /usr/local/lib
includedir /usr/local/include
infodir /usr/local/info
mandir /usr/local/man
Generated config.cache

Now type ‘make’ to build

We don not know whether it matters. BTW, we did not download JAVA as it not mentioned in the README.

And when I run: gst-inspect-1.0 nvmsgbroker

(base) xxx:/opt/nvidia/deepstream/deepstream-5.0/sources/libs/kafka_protocol_adaptor/librdkafka$ gst-inspect-1.0 nvmsgbroker
no such component or plug-in‘nvmsgbroker’

Oh, sorry, seen as amqp, yes, you should refer to kafka readme, sources/libs/kafka_protocol_adaptor/README.
It’s strange you could run with sync test, but failed with async test. they connect to same broker.

Can you find /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_msgbroker.so

We are using a pc with x86_64-linux-gnu and we don not have a dir named aarch64-linux-gpu.

Oh, you run from x86, then need to change the dir accordingly.
/usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream

Hi, we found this dir /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream, and yes it has libnvdsgst_msgbroker.so. “need to change the dir accordingly”, does it mean we need to change the dir in some places? If so, where should we change?

Hi, we have solved this problem. Just re-build the librdkafka and it works now.
Thank you!

Thanks for let us knowing.