Platform: Orin NX Developer Kit
JetPack 5.1.1 [L4T 35.3.1]
Hello, I am trying to decode a live h264 video stream from a drone using jetson_multimedia_api and the 00_video_decode sample as a base. The video is received as an RTSP stream which is h264 encoded with the following profile and encoding features.
H.264 Main Profile (profile_idc = 77)
The profile-level-id value is 0x4d 0x40 0x28 (i.e. profile=77, constraint_set1_flag=1, other constraint flags 0, level=4.0).
- CABAC
- No Interlacing
- No B-frames
- Slice Encoding
- Intra-Refresh (no I-frames)
- Non-ref frames
Currently I am using the video streaming API provided by the drone manufacturer to obtain the video. The API is capable of decoding the video using the CPU but this has an unacceptable performance cost so I would like to implement hardware-accelerated decoding.
I am able to get the individual NAL units of the encoded frames from the drone’s video API which I currently just write to a binary file. I’ve tried running 00_video_decode with the NALU file as the input both with and without the --input-nalu flag enabled but I get errors every time.
Command: ./video_decode H264 -o output --disable-rendering input_nalu.bin
With --input-nalu:
Set governor to performance before enabling profiler
Creating decoder in blocking mode
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Setting frame input mode to 0
Starting decoder capture loop thread
Could not read nal unit from file. EOF or file corrupted
Input file read complete
Got EoS at output plane
[ERROR] (NvV4l2Element.cpp:114) <dec0> Error while DQing event: Invalid argument
Error in dequeueing decoder event
Exiting decoder capture loop thread
App run failed
With --input-chunks:
Set governor to performance before enabling profiler
Creating decoder in blocking mode
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Setting frame input mode to 1
Input file read complete
Starting decoder capture loop thread
Got EoS at output plane
[ERROR] (NvV4l2Element.cpp:114) <dec0> Error while DQing event: Invalid argument
Error in dequeueing decoder event
Exiting decoder capture loop thread
App run failed
I also sometimes get this error: FMO streams not supported for T210 onwards
Due to the lack of IDR frames, other solutions such as GStreamer and FFmpeg do not work as the stream playback never starts without an IDR frame. I have also tried the Jetson-based fork of FFmpeg but it also did not work. I’m not very experienced with h264 and video encoding/decoding so it’s possible that I’m just making a basic mistake. Still, any help would be greatly appreciated.