Nvmpi decoder doesn't work

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Attempting to use nvidia flavor of ffmpeg with nvmpi, the decoder fails to produce frames generated by live555MediaServer. Regular h264 decoder on any platform, or CUDA-accelerated decoder on x86_64 have no trouble with that stream.

This is the command I’m attempting to run:
ffmpeg -c:v h264_nvmpi -i rtsp://127.0.0.1/short.mkv -f null -

Frame counter never goes past 0. Replacing h264_nvmpi with h264 makes it work.

Failing scenario:

root@86734c373772:/data/ffmpeg-3# ./bin/ffmpeg -c:v h264_nvv4l2dec -i rtsp://127.0.0.1/short.mkv  -f null -
ffmpeg version n4.2.2-15-g6878ea5a44 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --enable-nvv4l2dec --enable-libv4l2 --enable-shared --extra-libs='-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils' --extra-cflags='-I /usr/src/jetson_multimedia_api/include/'
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
Input #0, rtsp, from 'rtsp://127.0.0.1/short.mkv':
  Metadata:
    title           : Matroska video+audio+(optional)subtitles, streamed by the LIVE555 Media Server
    comment         : short.mkv
  Duration: 00:00:03.07, start: 0.200667, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
Failed to query video capabilities: Inappropriate ioctl for device
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_nvv4l2dec) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
frame=    0 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A
1 Like
2

I have similar issue here.

Setup:
R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64, DATE: Fri Jun 26 04:34:27 UTC 2020

Using ffmpeg with following:
ffmpeg -c:v h264_nvv4l2dec -i rtsp://127.0.0.1:8554/video.mkv -f null -

Gives the following output:

ffmpeg version n4.2.2-15-g6878ea5a44 Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 7 (Ubuntu/Linaro 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --enable-nvv4l2dec --enable-libv4l2 --enable-shared --extra-libs='-L/usr/lib/aarch64-linux-gnu/tegra -lnvbuf_utils' --extra-cflags='-I /usr/src/jetson_multimedia_api/include/'
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
Input #0, rtsp, from 'rtsp://127.0.0.1:8554/video.mkv':
  Metadata:
    title           : Matroska video+audio+(optional)subtitles, streamed by the LIVE555 Media Server
    comment         : video.mkv
  Duration: 00:00:03.07, start: 0.200667, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (h264_nvv4l2dec) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
frame=    0 fps=0.0 q=0.0 size=N/A time=-577014:32:22.77 bitrate=N/A speed=N/A
1 Like
4

Hi,
Could you try gstreamer command:

$ gst-launch-1.0 rtspsrc location=rtsp://127.0.0.1:8554/video.mkv ! rtph264depay ! h264parse ! nvv4l2decoder ! nvoverlaysink

Would like to know if it also fails in decoding through gstreamer command.

5

gstreamer command works

6

Hi,
We generally set up RTSP server by running test-launch. Don’t have experience in using live555. Please share how to run live555 to set up a server and we can try to replicate the issue.

7

live555-test.zip (4.6 MB)

You can use the attached test (you can also, of course, build live555 from publicly available source, if that is the preference).

Run ./live555MediaServer in the folder containing the video files. The server will make all the files in current working directory available. (Note: depending on local ports availability, the server may listen to on 554 or 8554; it will emit a log statement with the address).

The two attached files result in different behavior:

ffmpeg -c:v h264_nvv4l2dec -i rtsp://127.0.0.1:554/test_broken.mkv -f null - never generates a single frame.
ffmpeg -c:v h264_nvv4l2dec -i rtsp://127.0.0.1:554/test_warnings.mkv -f null - does generate frames, but also produces quite a few warnings.

Software decoder seems to like both streams.

8

Hi,
Please try to extract the h264 stream and use it for streaming:

$ gst-launch-1.0 filesrc location=test_broken.mkv ! matroskademux ! h264parse ! video/x-h264,stream-format=byte-stream ! filesink location=broken.264

Not sure but it seems like there is invalid packets when the source file is mkv. Please try h264 stream.

10

I would expect an invalid h264 bitstream to break both software and hardware decoders. However, using the same source only the nvmpi decoder breaks.

11

Hi,
It should be some additional NALs which are not required for decoding but are fed into decoder. So buffers are filled with non-bitstream data and decoder cannot continue to work. Since there is a working solution we would suggest take it. Also the code is public and you can download it by following the steps in
Jetson TX2 and FFmpeg - Can't initialize nvrm channel - #5 by DaneLLL

Would be great if you can help check and may work out a patch for the use-case.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.