Deepstream-fewshot-learning-app: filesink or rtspsink does not work

Please provide complete information as applicable to your setup.

• Hardware Platform dGPU
• DeepStream Version 6.4
• TensorRT Version
• NVIDIA GPU Driver Version 555.42.06
• Issue Type bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

I am testing deepstream-fewshot-learning-app as a standalone application.

So far, everything is fine, but I have tested with both filesink and rtspsink, but both do not work.

For filesink, the out.mp4 is not playable.
For rtspsink, the uri cannot be played in VLC.

Here is the main config.
mtmc_config.txt (7.0 KB)

Could you take a look at this? Thanks so much!

Are you want to deploy MTMC? Can you have a try based on the guide: Quickstart — mdx-docs 2.1 documentation (nvidia.com)?

Yes. However, I would like to custom components. For example, I already have VMS (Nx Witness), and I want to custom MCMT algorithm in the MCMT microservice.

deepstream-fewshot-learning-app is really good example/sample that I can follow, though it has been developed based on DS SDK 6.4 (not the latest, and there may be a bug: Nvstreammux consumes 100% CPU when drop-pipeline-eos=true and an EOS from source was processed).

I think many people would like to play with deepstream-fewshot-learning-app , and if your team could upgrade it to use DS SDK 7.0, it will be really helpful.

I checked the mtmc_config.txt. Seems you don’t enable sink in the config. Can you share the reproduce steps of the issue? So I can reproduce it in my side.

Here is the config I have tried (I enable file sink).
mtmc_config.txt (7.0 KB)

docker image: nvcr.io/nvidia/deepstream:6.4-triton-multiarch

$ ./deepstream-fewshot-learning-app -c mtmc_config.txt -m 1 -t 1 -l 5 --message-rate 1 

Logs

WARNING; playback mode used with URI [file:///opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-fewshot-learning-app/samples/Building_K_Cam7.mp4] not conforming to timestamp format; check README; using system-time

...

Active sources : 1
Tue Oct  8 10:19:00 2024
**PERF:  395.74 (427.20)
nvstreammux: Successfully handled EOS for source_id=0
Active sources : 1
Tue Oct  8 10:19:05 2024
**PERF:  377.70 (410.64)
Active sources : 0
Tue Oct  8 10:19:10 2024
**PERF:  0.00 (368.60)
Active sources : 0
Tue Oct  8 10:19:15 2024
**PERF:  0.00 (334.36)
Active sources : 0
Tue Oct  8 10:19:20 2024
**PERF:  0.00 (305.95)
qQuitting
[NvMultiObjectTracker] De-initialized
App run successful

with GST_DEBUG=3 I see

0:00:28.287465910  1144 0x55816f5e14c0 ERROR                  libav :0:: Invalid pts (107892391) <= last (107892391)
0:00:28.287480441  1144 0x55816f5e14c0 ERROR                  libav gstavvidenc.c:775:gst_ffmpegvidenc_handle_frame:<sink_sub_bin_encoder2> avenc_mpeg4: failed to encode buffer
0:00:28.289692902  1144 0x55816f5e14c0 ERROR                  libav :0:: Invalid pts (107892391) <= last (107892391)
0:00:28.289706096  1144 0x55816f5e14c0 ERROR                  libav gstavvidenc.c:775:gst_ffmpegvidenc_handle_frame:<sink_sub_bin_encoder2> avenc_mpeg4: failed to encode buffer

Yes, seems mpeg4 video encoder report error. Why you choose SW mpeg4 video encoder? Which GPU are you using?

I am testing the sample on NVIDIA GeForce RTX 4090 .

[sink2] group reads

[sink2]
enable=1
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
## only SW mpeg4 is supported right now.
codec=3
sync=1
bitrate=2000000
output-file=out.mp4
source-id=0

It has the lines “only SW mpeg4 is supported right now.” by default, so I just use codec=3.

You mean I can try codec=1 right?

NVIDIA GeForce RTX 4090 has HW video encoder. You can use h264/h265 video encoder.

Even I try to use h264 (codec=1), the out.mp4 file is still not playable in either QuickTime or VLC.

I observe the following logs with GST_DEBUG=3

0:00:09.239240241  2180 0x7fa7c003fd80 WARN                    v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat PM10
0:00:09.239261406  2180 0x7fa7c003fd80 WARN                    v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:09.239283776  2180 0x7fa7c003fd80 WARN                    v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat PM10
0:00:09.239321022  2180 0x7fa7c003fd80 WARN                    v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:09.239344984  2180 0x7fa7c003fd80 WARN                    v4l2 gstv4l2object.c:2985:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe minimum capture size for pixelformat NM12
0:00:09.239366588  2180 0x7fa7c003fd80 WARN                    v4l2 gstv4l2object.c:3100:gst_v4l2_object_get_nearest_size:<nvv4l2decoder0:src> Unable to try format: Unknown error -1
0:00:09.239389142  2180 0x7fa7c003fd80 WARN                    v4l2 gstv4l2object.c:2991:gst_v4l2_object_probe_caps_for_format:<nvv4l2decoder0:src> Could not probe maximum capture size for pixelformat NM12
0:00:09.360764135  2180 0x7fa7c003fd80 WARN            v4l2videodec gstv4l2videodec.c:2258:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
** INFO: <bus_callback:287>: Pipeline running

0:00:09.361247282  2180 0x7fa7c003fd80 WARN          v4l2bufferpool gstv4l2bufferpool.c:1114:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:09.362543089  2180 0x7fa7c4019460 WARN          v4l2bufferpool gstv4l2bufferpool.c:1565:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:09.363051433  2180 0x7fa7b000bd20 WARN         nvstreammux_ntp gstnvstreammux_ntp.cpp:121:check_if_sys_rtcp_time_is_ntp_sync:<src_bin_muxer> warning: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2024-10-09T07:33:34.833Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z
WARNING from src_bin_muxer: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2024-10-09T07:33:34.833Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z
Debug info: gstnvstreammux_ntp.cpp(121): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstNvStreamMux:src_bin_muxer
0:00:09.528373659  2180 0x558894d20cc0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1114:gst_v4l2_buffer_pool_start:<sink_sub_bin_encoder2:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:10.193770989  2180 0x7fa7c4019400 WARN          v4l2bufferpool gstv4l2bufferpool.c:1565:gst_v4l2_buffer_pool_dqbuf:<sink_sub_bin_encoder2:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:10.194859575  2180 0x558894d20de0 FIXME               basesink gstbasesink.c:3395:gst_base_sink_default_event:<sink_sub_bin_sink2> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:10.195050227  2180 0x558894d20de0 FIXME             aggregator gstaggregator.c:1410:gst_aggregator_aggregate_func:<sink_sub_bin_mux2> Subclass should call gst_aggregator_selected_samples() from its aggregate implementation.

and the following shows app does not exit after EOS.

**PERF:  420.62 (411.88)
nvstreammux: Successfully handled EOS for source_id=0
Active sources : 1
Wed Oct  9 07:34:19 2024
**PERF:  387.59 (408.89)
Active sources : 0
Wed Oct  9 07:34:24 2024
**PERF:  0.00 (367.19)

Hello,
Is there any update?

What is the size of out.mp4? MP4 output file will broken if the pipeline don’t send EOS to mp4mux. Why you need output mp4 file in this application? Can you use RTSP output?

It is 113M.

It is just for debug and visualization.

I have already tried but not successful yet (the stream rtsp://${HOST_IP}:8554/ds-test cannot be played in VLC app)

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=4
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
# set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400