Internal Data Stream Error when decoding filesrc streams with nvv4l2decoder plugin

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) - Jetson Xavier NX
• DeepStream Version Version 6
• JetPack Version (valid for Jetson only)- Jetpack 4.6 (32.6.1)
• TensorRT Version - 8.0.6.1
• Issue Type( questions, new requirements, bugs) - Question

Hello Everyone, I am working on Jetson Xavier NX with DeepStream6. I have a question regarding the nvv4l2decoder plugin and would like to know how to solve it. I am trying to decode 4 file sources with the filesrc plugin and nvv4l2decode plugin. My pipeline works when I add a videorate plugin and the pipeline looks like the following

filesrc(source=0, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
filesrc(source=1, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
filesrc(source=2, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
filesrc(source=3, .mp4) -> qtdemux -> h264parse -> nvv4l2decoder -> videorate -> capsfilter(caps=video/x-raw(memory:NVMM)) -> nvvideoconvert -> capsfilter(caps=video/x-raw(memory:NVMM), format=(string)RGBA) -> nvstreammux
nvstreammux -> nvinfer -> nvmultistreamtiler -> nvdsosd -> nvegltransform -> nveglglessink

But, when I remove the videorate plugin and the corresponding capsfilter, I get the following error

ERROR from element demuxer-0: Internal data stream error.
Error details: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:embedded-vision-pipeline/GstBin:source-bin-00/GstQTDemux:demuxer-0:
streaming stopped, reason not-linked (-1)
ERROR from element demuxer-3: Internal data stream error.
Error details: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:embedded-vision-pipeline/GstBin:source-bin-03/GstQTDemux:demuxer-3:
streaming stopped, reason not-linked (-1)
ERROR from element demuxer-1: Internal data stream error.
Error details: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:embedded-vision-pipeline/GstBin:source-bin-01/GstQTDemux:demuxer-1:
streaming stopped, reason not-linked (-1)
ERROR from element demuxer-2: Internal data stream error.
Error details: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:embedded-vision-pipeline/GstBin:source-bin-02/GstQTDemux:demuxer-2:
streaming stopped, reason not-linked (-1)
Returned, stopping playback
[NvMultiObjectTracker] De-initialized
Deleting pipeline

However, the pipeline works when I am decoding only one stream without the videorate plugin. Can anybody tell me how to make 4 streams work without the videorate plugin ?

Looking forward to your responses.

Hi,
Please check if you can launch 4 video file sources in deepstream-app. Please try this config file first:

/opt/nvidia/deepstream/deepstream-6.0/samples/configs/deepstream-app/source30_1080p_dec_infer-resnet_tiled_display_int8.txt

If it runs well, please change to 4 video sources and give it a try:

[tiled-display]
enable=1
rows=2
columns=2
...(skip)

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source3]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0
[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=4
...(skip)

[primary-gie]
enable=1
gpu-id=0
model-engine-file=../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=4
...(skip)

Hello @DaneLLL, Thank you for your reply.

  • On the first run I executed the config file as you suggested - source30_1080p_dec_infer-resnet_tiled_display_int8.txt. It executed perfectly and launched a window with 30 video streams on 2 video sources with each launching 15 video streams. Since I am executing it on Xavier NX, it ran very slow but that’s not an issue.

  • After this I made the changes as you suggested and it was able to launch 4 video sources and also the file looping worked perfectly.

Could you please tell me an overview of the pipeline that is being executed in the following way

element1 -> element2 -> element3 ... so on

If you are not using any video rate plugin, then why do I need to have a videorate plugin for my pipeline to execute and how can I make my pipeline work without the videorate plugin ?

Hi,
Not sure why it fails. Probably it needs function like queue plugin. You may replace videorate with queue for a try.

Besides, you may refer to deepstream-test3 and use uridecodebin:

 /opt/nvidia/deepstream/deepstream-6.0/sources/apps/sample_apps/deepstream-test3

deepstream_python_apps/apps/deepstream-test3 at master · NVIDIA-AI-IOT/deepstream_python_apps · GitHub

Hello @DaneLLL, Thanks for your swift response

  • I tried to replace the videorate plugin with the queue plugin but again it gave the same error.

  • After that I tried implementing the uridecodebin plugin but again the error is of a very similar nature

ERROR from element qtdemux0: Internal data stream error.
Error details: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:embedded-vision-pipeline/GstBin:source-bin-00/GstURIDecodeBin:uri-decode-bin/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated (-4)
ERROR from element qtdemux2: Internal data stream error.
Error details: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:embedded-vision-pipeline/GstBin:source-bin-03/GstURIDecodeBin:uri-decode-bin/GstDecodeBin:decodebin3/GstQTDemux:qtdemux2:
streaming stopped, reason not-negotiated (-4)
ERROR from element qtdemux3: Internal data stream error.
Error details: qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:embedded-vision-pipeline/GstBin:source-bin-02/GstURIDecodeBin:uri-decode-bin/GstDecodeBin:decodebin2/GstQTDemux:qtdemux3:
streaming stopped, reason not-negotiated (-4)
Returned, stopping playback
[NvMultiObjectTracker] De-initialized
Deleting pipeline

Is there any other way we could solve it ?

Hi,
deepstream-test3 should work well. Please give it a try. And then refer to the sample for the development.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.