Hardware encoding for RTSP streaming

• NVIDIA Jetson Xavier NX Developer Kit
• DeepStream 6.2
• L4T 35.3.1

How to enable Hardware Encoding for RTSP streaming correctly? Here is an example configuration:

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=4
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=1
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
rtsp-port=8554
udp-port=5400

If I change enc-type from 1 (software) to 0 (hardware), then deepstream stops working:

# deepstream-app -c deepstream_app_source1_trafficcamnet.txt
** ERROR: <create_udpsink_bin:644>: Failed to create 'sink_sub_bin_encoder1'
** ERROR: <create_udpsink_bin:719>: create_udpsink_bin failed
** ERROR: <create_sink_bin:828>: create_sink_bin failed
** ERROR: <create_processing_instance:884>: create_processing_instance failed
** ERROR: <create_pipeline:1485>: create_pipeline failed
** ERROR: <main:697>: Failed to create pipeline
Quitting
nvstreammux: Successfully handled EOS for source_id=0
App run failed

I’m using official deepstream-lt4:6.2-samples docker image on Jetson Xavier NX Developer Kit with L4T 35.3.1

BTW, is there any documentation for Sinks? I’m unable to find any.

Can you ran the following command successfully on host ?

gst-launch-1.0 videotestsrc ! nvvidconv ! nvv4l2h264enc ! filesink location=out1.h264

And how you run the docker on jetson ? can you share the command ?

Sorry, there is no document for sinks. you can refer the source code deepstream_sink_bin.c
in /opt/nvidia/deepstream/deepstream/sources/apps/apps-common/src

# gst-launch-1.0 videotestsrc ! nvvidconv ! nvv4l2h264enc ! filesink location=out1.h264

(gst-plugin-scanner:513): GStreamer-WARNING **: 12:43:13.347: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:513): GStreamer-WARNING **: 12:43:13.419: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory

WARNING: erroneous pipeline: no element "nvvidconv"

Same for nvv4l2h264enc:

# gst-launch-1.0 videotestsrc ! nvv4l2h264enc 
WARNING: erroneous pipeline: no element "nvv4l2h264enc"

And how you run the docker on jetson ? can you share the command ?

Sure:

# docker pull nvcr.io/nvidia/deepstream-l4t:6.2-samples
# docker run -it --runtime=nvidia nvcr.io/nvidia/deepstream-l4t:6.2-samples

Then i run user_additional_install.sh in the /opt/nvidia/deepstream/deepstream-6.2 directory.

Maybe you mean nvvideoconvert instead of nvvidconv?

nvvideoconvert exists:

# gst-launch-1.0 videotestsrc ! nvvideoconvert ! nvv4l2h264enc ! filesink location=out1.h264
WARNING: erroneous pipeline: no element "nvv4l2h264enc"
gst-inspect-1.0
# gst-inspect-1.0 | grep '^nv'
nvdsgst_ofvisual:  nvofvisual: nvofvisual
nvvideoconvert:  nvvideoconvert: NvVidConv Plugin
nvdsgst_postprocess:  nvdspostprocess: NvDsPostProcess plugin for Transform/In-Place use-cases
nvdsgst_3dfilter:  nvds3dfilter: DS 3D filter custom plugin
nvdsgst_segvisual:  nvsegvisual: nvsegvisual
nvdsgst_msgbroker:  nvmsgbroker: Message Broker
nvdsgst_deepstream_bins:  nvinferbin: NvInfer Bin
nvdsgst_deepstream_bins:  nvinferserverbin: NvInferServer Bin
nvdsgst_deepstream_bins:  nvosdbin: NvOsd Bin
nvdsgst_deepstream_bins:  nvdewarperbin: NvDewarper Bin
nvdsgst_deepstream_bins:  nvtilerbin: NvTiler Bin
nvdsgst_deepstream_bins:  nvtrackerbin: NvTracker Bin
nvdsgst_deepstream_bins:  nvurisrcbin: NvUriSrc Bin
nvdsgst_deepstream_bins:  nvcamerasrcbin: NvCameraSrc Bin
nvdsgst_deepstream_bins:  nvanalyticsbin: NvAnalytics Bin
nvdsgst_deepstream_bins:  nvvideorenderersinkbin: NvVideoRenderer Bin
nvdsgst_deepstream_bins:  nvvideoencfilesinkbin: NvVideoEncFilesink Bin
nvdsgst_deepstream_bins:  nvrtspoutsinkbin: NvRtspOut Bin
nvdsgst_deepstream_bins:  nvmsgbrokersinkbin: NvMsgBroker Bin
nvdsgst_deepstream_bins:  nvdsbuffersyncbin: NvBufferSync Bin
nvdsgst_dsanalytics:  nvdsanalytics: DsAnalytics plugin
nvdsgst_osd:  nvdsosd: NvDsOsd plugin
nvdsgst_videotemplate:  nvdsvideotemplate: NvDsVideoTemplate plugin for Transform/In-Place use-cases
nvdsgst_of:  nvof: nvof
nvdsgst_speech:  nvdsasr: DS ASR Plugin for speech use-cases
nvdsgst_metamux:  nvdsmetamux: meta muxer
nvdsgst_dewarper:  nvdewarper: nvdewarper
nvdsgst_dsexample:  dsexample: DsExample plugin
nvdsgst_inferaudio:  nvinferaudio: NvInfer Audio plugin
nvdsgst_preprocess:  nvdspreprocess: gst-nvdspreprocess plugin
nvdsgst_logger:  nvdslogger: Nvdslogger
nvdsgst_multistreamtiler:  nvmultistreamtiler: Stream Tiler DS
nvdsgst_tracker:  nvtracker: NvTracker plugin
nvdsgst_multistream:  nvstreammux: Stream multiplexer
nvdsgst_multistream:  nvstreamdemux: Stream demultiplexer
nvdsgst_msgconv:  nvmsgconv: Message Converter
nvdsgst_audiotemplate:  nvdsaudiotemplate: DS AUDIO template Plugin for Transform IP use-cases
nvdsgst_text_to_speech:  nvds_text_to_speech: DS Text To Speech Plugin for Conversational AI use cases
nvdsgst_infer:  nvinfer: NvInfer plugin

Yes,I make a mistake.

It’s look like some soft link are broken, try run install.sh in /opt/nvidia/deepstream/deepstream-6.2

Still nothing

# /opt/nvidia/deepstream/deepstream-6.2/install.sh

# gst-launch-1.0 videotestsrc ! nvvideoconvert ! nvv4l2h264enc ! filesink location=out1.h264

(gst-plugin-scanner:1226): GStreamer-WARNING **: 10:20:55.911: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory

(gst-plugin-scanner:1226): GStreamer-WARNING **: 10:20:55.975: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
WARNING: erroneous pipeline: no element "nvv4l2h264enc"

Detailed ./install.sh output
# bash -x install.sh 
++ uname -m
+ TARGET_DEVICE=aarch64
++ cat /etc/os-release
++ awk -F= '$1=="ID"{print $2}'
++ sed 's/"//g'
+ OS=ubuntu
+ '[' aarch64 = x86_64 ']'
+ '[' aarch64 = aarch64 ']'
+ BASE_LIB_DIR=/usr/lib/aarch64-linux-gnu
+ '[' -d /usr/lib/aarch64-linux-gnu/nvidia ']'
+ NVIDIA_LIB_DIR=/usr/lib/aarch64-linux-gnu/tegra
+ getopts d: option
+ '[' -n aarch64 ']'
+ '[' aarch64 = x86_64 ']'
+ '[' aarch64 = aarch64 ']'
+ update-alternatives --install /usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream deepstream-plugins /opt/nvidia/deepstream/deepstream-6.2/lib/gst-plugins 62
+ ln -sf /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so /opt/nvidia/deepstream/deepstream-6.2/lib/libnvbufsurface.so
+ ln -sf /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so /opt/nvidia/deepstream/deepstream-6.2/lib/libnvbufsurftransform.so
+ echo /opt/nvidia/deepstream/deepstream-6.2/lib
+ echo /opt/nvidia/deepstream/deepstream-6.2/lib/gst-plugins
+ update-alternatives --install /usr/bin/deepstream-app deepstream-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-app 62
+ update-alternatives --install /usr/bin/deepstream-audio deepstream-audio /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-audio 62
+ update-alternatives --install /usr/bin/deepstream-asr-app deepstream-asr-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-asr-app 62
+ update-alternatives --install /usr/bin/deepstream-asr-tts-app deepstream-asr-tts-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-asr-tts-app 62
+ update-alternatives --install /usr/bin/deepstream-test1-app deepstream-test1-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-test1-app 62
+ update-alternatives --install /usr/bin/deepstream-test2-app deepstream-test2-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-test2-app 62
+ update-alternatives --install /usr/bin/deepstream-test3-app deepstream-test3-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-test3-app 62
+ update-alternatives --install /usr/bin/deepstream-test4-app deepstream-test4-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-test4-app 62
+ update-alternatives --install /usr/bin/deepstream-test5-app deepstream-test5-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-test5-app 62
+ update-alternatives --install /usr/bin/deepstream-testsr-app deepstream-testsr-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-testsr-app 62
+ update-alternatives --install /usr/bin/deepstream-transfer-learning-app deepstream-transfer-learning-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-transfer-learning-app 62
+ update-alternatives --install /usr/bin/deepstream-user-metadata-app deepstream-user-metadata-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-user-metadata-app 62
+ update-alternatives --install /usr/bin/deepstream-dewarper-app deepstream-dewarper-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-dewarper-app 62
+ update-alternatives --install /usr/bin/deepstream-nvof-app deepstream-nvof-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-nvof-app 62
+ update-alternatives --install /usr/bin/deepstream-image-decode-app deepstream-image-decode-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-image-decode-app 62
+ update-alternatives --install /usr/bin/deepstream-gst-metadata-app deepstream-gst-metadata-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-gst-metadata-app 62
+ update-alternatives --install /usr/bin/deepstream-opencv-test deepstream-opencv-test /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-opencv-test 62
+ update-alternatives --install /usr/bin/deepstream-preprocess-test deepstream-preprocess-test /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-preprocess-test 62
+ update-alternatives --install /usr/bin/deepstream-segmentation-app deepstream-segmentation-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-segmentation-app 62
+ update-alternatives --install /usr/bin/deepstream-image-meta-test deepstream-image-meta-test /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-image-meta-test 62
+ update-alternatives --install /usr/bin/deepstream-appsrc-test deepstream-appsrc-test /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-appsrc-test 62
+ update-alternatives --install /usr/bin/deepstream-can-orientation-app deepstream-can-orientation-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-can-orientation-app 62
+ update-alternatives --install /usr/bin/deepstream-nvdsanalytics-test deepstream-nvdsanalytics-test /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-nvdsanalytics-test 62
+ update-alternatives --install /usr/bin/deepstream-3d-action-recognition deepstream-3d-action-recognition /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-3d-action-recognition 62
+ update-alternatives --install /usr/bin/deepstream-3d-depth-camera deepstream-3d-depth-camera /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-3d-depth-camera 62
+ update-alternatives --install /usr/bin/deepstream-lidar-inference-app deepstream-lidar-inference-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-lidar-inference-app 62
+ update-alternatives --install /usr/bin/deepstream-nmos-app deepstream-nmos-app /opt/nvidia/deepstream/deepstream-6.2/bin/deepstream-nmos-app 62
+ ldconfig
+ rm -rf '/home/*/.cache/gstreamer-1.0/'
+ rm -rf /root/.cache/gstreamer-1.0/
find
# find / -name '*h264*'
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstuvch264.so
/opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_qHD.h264
/opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_720p.h264
/opt/nvidia/deepstream/deepstream-6.2/samples/streams/sample_1080p_h264.mp4
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvarguscamerasrc.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcompositor.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvdrmvideosink.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnveglglessink.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnveglstreamsrc.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvegltransform.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvivafilter.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvjpeg.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvtee.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvv4l2camerasrc.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvidconv.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideo4linux2.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideosink.so
/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideosinks.so

Do these files exist ?

nvv4l2h264enc element is implemented in libgstnvvideo4linux2.so

For DeepStream to run successfully on Jetson, these libraries must exist.
If no,maybe you need install the JetPack

Nothing at all.

ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvarguscamerasrc.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvcompositor.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvdrmvideosink.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnveglglessink.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnveglstreamsrc.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvegltransform.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvivafilter.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvjpeg.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvtee.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvv4l2camerasrc.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvidconv.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideo4linux2.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideosink.so': No such file or directory
ls: cannot access '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvvideosinks.so': No such file or directory

I tried to run find over every library name and found also nothing.

If no,maybe you need install the JetPack

Sorry, I don’t understand. I’m running the official unchanged docker image on a device that is already flashed with L4T using official instructions. Where should I install L4T? Inside the container?

I’m wondering how did you figure this out. Is there a source code somewhere?

JetPack SDK is some libraries for ai and multilmedia

Some libraries are shared between host and docker on Jetson, such as JetPack SDK.

If the host miss there libraries,docker can’t work normally

I think you missed the jetpack sdk when flashing the L4T

Oooooookay

The puzzle has come together. I found that the file libgstnvvideo4linux2.so is located in the package nvidia-l4t-gstreamer. And those package was not installed on the host for some reason.

So the fix is simple for me: apt install nvidia-l4t-gstreamer on host machine.

Why not create a more comprehensible error format and a file listing? This issue would have been resolved in ten minutes if I understood that the original error was related to the absence of nvv4l2h264enc that it is a component of the library libgstnvvideo4linux2.so.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.