Deepstream-app (latest JetPack 4.3) getting immediate NAL Unit Type 26/27 errors on RTSP

I am trying to get Deepstream-app to run for my 4 camera 5MP OnWote cameras. I am able to get the app working but only for the lower resolution RTSP sub streams. When I switch the source to the main stream (full 2592x1944 res) I get NAL errors.

Is there something I can change in my config file or camera setup to get this working at full resolution?

I am running this on a Jetson Xavier AGX …

Here is my config:

# Copyright (c) 2019 NVIDIA Corporation.  All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto.  Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=2
columns=2
width=1800
height=1040
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:XXXXXXXXXXXX@192.168.128.37:554/chID=1&streamType=sub&linkType=tcp
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:XXXXXXXXXXXX@192.168.128.37:554/chID=5&streamType=sub&linkType=tcp
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:XXXXXXXXXXXX@192.168.128.37:554/chID=3&streamType=sub&linkType=tcp
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source3]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:XXXXXXXXXXXX@192.168.128.37:554/chID=2&streamType=sub&linkType=tcp
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
sync=0
#iframeinterval=10
bitrate=2000000
output-file=out.mp4
source-id=0

[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=h264 2=h265
codec=1
sync=0
bitrate=4000000
# set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400

[osd]
enable=1
gpu-id=0
border-width=1
text-size=12
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=4
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=../../models/Primary_Detector_Nano/resnet10.caffemodel_b8_fp16.engine
batch-size=4
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_nano.txt

[tracker]
enable=1
tracker-width=480
tracker-height=272
#ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_mot_iou.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_mot_klt.so
#ll-config-file required for IOU only
#ll-config-file=iou_config.txt
gpu-id=0

[tests]
file-loop=0

Hi,
Please check if you can run pure video playback:

$ gst-launch-1.0 uridecodebin uri=rtsp://admin:XXXXXXXXXXXX@192.168.128.37:554/chID=3&streamType=sub&linkType=tcp ! nvvidconv ! nveglglessink

Yes. I can run pure video playback on both the sub and main RTSP streams.

So I was able to get this to somewhat work on the 5MP RTSP feeds by changing the width to 2592 and the height to 1944 (the resolution of the cameras) in the [streammux] section of the config file.

Unfortunately it crashes every few minutes with a “NAL Unit Type 50 not supported yet” error in gstrtph265depay.c(1553):gst_rtp_h265_depay_process(). Any thoughts on how to fix or ignore this NAL Unit error?

Hi,
Please try to set width,height to multiple of 2592x1944 in [tiled-display] for a try.

Also please run in max performance by executing ‘sudo nvpmodel -m 0’ and ‘sudo jetson_clocks’

Changing the setting outlined above did not stop the NAL Unit errors.

Strangely, once the camera’s switches over to night mode (grays scale) they seem to process far more stably, with only occasional NAL errors.

If the streams switch over to color or are already on color, deepstream-app crashes with the NAL Unit 50 error or never even connects.

Camera model (all 4) are OnWote MC400L-EN-Y_V0.

Issue happens whether I connect one camera or all 4 in full 5MP H.265 stream.

This is running on Jetson AGX Xavier Development Kit … Neither GPU, CPU, or Memory appear to be under heavy load (per: sudo jtop).

Hi,
Please run video playback and check if the issue can be reproduced:

$ gst-launch-1.0 rtspsrc location=rtsp://admin:XXXXXXXXXXXX@192.168.128.37:554/chID=3&streamType=sub&linkType=tcp ! rtph265depay ! h265parse ! nvv4l2decoder !

If it can be reproduced, please follow below link to dump the h265 stream and try software decoder avdec_h265:
https://devtalk.nvidia.com/default/topic/1069291/jetson-agx-xavier/rtsp-h264-corrupted-decoding-quot-reference-in-dpb-was-never-decoded-quot-/post/5416789/#5416789

Not sure but it looks like the h265 stream is corrupted in certain condition.

Just wondering if you ever figured this out. I too get mysterious “NAL Unit Type 26/27” errors in certain cases, and am trying to figure out what causes it…

Hi logidelic,
The definition in ffmpeg-3.3.1/libavcodec/h264.h:

/* NAL unit types */
enum {
    H264_NAL_SLICE           = 1,
    H264_NAL_DPA             = 2,
    H264_NAL_DPB             = 3,
    H264_NAL_DPC             = 4,
    H264_NAL_IDR_SLICE       = 5,
    H264_NAL_SEI             = 6,
    H264_NAL_SPS             = 7,
    H264_NAL_PPS             = 8,
    H264_NAL_AUD             = 9,
    H264_NAL_END_SEQUENCE    = 10,
    H264_NAL_END_STREAM      = 11,
    H264_NAL_FILLER_DATA     = 12,
    H264_NAL_SPS_EXT         = 13,
    H264_NAL_AUXILIARY_SLICE = 19,
};

Types 26/27 are not defined. It is most likely the h264 stream is corrupted. Please check if you can run TCP protocol in RTSP.
A user shares a patch of running test-launch/test-mp4 in TCP. FYR.
https://devtalk.nvidia.com/default/topic/1062748/deepstream-sdk/nvvideoconvert-crashes-on-rtsp-input-src-crop-x-y-w-h-pipeline/post/5387790/#5387790

Thanks. I ended up upgrading my NVR from the cheap one that came with the cameras to a commercial grade and I can now run deepstream at full resolution (5MP) on all four cameras and it is rock solid.

At 15W it barely pushes the GPU above 50%.

I don’t think the original NVR had the horsepower to handle what I was asking it to do.

Now I can start playing around with the models and tying it into my home automation solution!!

Thanks all!

Is upgrading the NVR the only solution? I too get mysterious “NAL Unit Type 26/27” errors in certain cases but I am not sure if its the h264 stream that is corrupted as I am able to capture the stream using OpenCV.
Moreover those errors appear and disappear randomly after a few hours of running successfully. Is there any other workaround for this problem?

Hi sharanssundar

Please open a new topic for your issue. Thanks

I did. Thank you.