Lossless data record and load with nvidia-cuda supported gstreamer

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
, , , ,
1

I am trying to record camera data without any data loss using nvidia hardware-accelerated gstreamer. However, I have encountered some issues.

The developement environments are as below.

  • NVIDIA Jetson AGX Orin Developer Kit
  • Linux ubuntu 5.10.104-tegra, aarch64 aarch64 aarch64 GNU/Linux

  • R35 (release), REVISION: 3.1

Question 1.
As I known, for lossless data recording, using YUV4:4:4 format is necessary, because the h264 codec only supports it. Therefore, I recorded and loaded without any data loss with pipelines shown below.

(Record)
appsrc ! video/x-raw, framerate=30/1, width=1280, height=720, format=Y444 ! videoconvert ! x264enc key-int-max=30 bitrate=65536 pass=qual quantizer=0 speed-preset=ultrafast tune=zerolatency ! filesink location=/mnt/recorded.h264

(Load)
filesrc location=/mnt/recorded.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! video/x-raw, framerate=30/1, width=1280, height=720, format=Y444 ! appsink emit-signals=true

However, I failed to do it with hardware-acceleration pipeline elements. The pipelines that I have tried are:

(Record)
appsrc ! video/x-raw, framerate=30/1, width=1280, height=720, format=NV24 ! videoconvert ! nvvidconv ! video/x-raw(memory:NVMM) ! nvv4l2h264enc bitrate=67108864 enable-lossless=true maxperf-enable=true ! filesink location=/mnt/recorded.h264

(Load)
filesrc location=/mnt/recorded.h264 ! video/x-h264 ! h264parse ! nvv4l2decoder enable-max-performance=true ! video/x-raw(memory:NVMM) ! nvvidconv ! videoconvert ! video/x-raw, framerate=30/1, width=1280, height=720, format=NV24 ! appsink emit-signals=true

When running it, data recording can be done without error messages. I could get somehow large-sized h264 file after recording. (but not sure if the recorded data has no problem, because it is not video container file). Afterwards, when I try loading the recorded data, I got error messages like:

nvbufsurftransform: Could not get EGL display connection
sh: 1: lsmod: not found
sh: 1: modprobe: not found
nvbuf_utils: Could not get EGL display connection
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NVMEDIA: NvMMLiteNVMEDIAProcessVES: 1824: NvMediaParserParse Unsupported Codec
Stream format not found, dropping the frame
Stream format not found, dropping the frame
NVMEDIA: NvMMLiteNVMEDIADecDoWork: 2228: NVMEDIA Video Dec Unsupported Stream
NVMEDIA: NvMMLiteNVMEDIAProcessVES: 1824: NvMediaParserParse Unsupported Codec
NVMEDIA: NvMMLiteNVMEDIAProcessVES: 1824: NvMediaParserParse Unsupported Codec

terminate called after throwing an instance of ‘std::runtime_error’
what(): error from: nvv4l2decoder0: No valid frames decoded before end of stream(gstvideodecoder.c(1140): gst_video_decoder_sink_event_default (): /GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0:
no valid frames found)

Could anyone help me to resolve it?

3

Hi,
Lossless encoding decoding is not supported in H264. Please refer to the topics and try H265:
Is H265 encoding via 01_video_encode really lossless?
Can tx2nx decode YUV444? - #3 by DaneLLL

4

Thanks for your reply.
However I have tested lossless data recording with this pipeline:

Pipeline used for recording

appsrc ! video/x-raw, framerate=30/1, width=1280, height=720, format=Y444 ! videoconvert ! x264enc key-int-max=30 bitrate=65536 pass=qual quantizer=0 speed-preset=ultrafast tune=zerolatency ! filesink location=/mnt/recorded.h264

Pipeline used for loading

filesrc location=/mnt/recorded.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! video/x-raw, framerate=30/1, width=1280, height=720, format=Y444 ! appsink emit-signals=true

And I found that the frame data completely match 100%, which means there was no data loss.
In addition, I discovered there is A property called enable-lossless={boolean} from nvv4l2h264enc encoder.

Could you explain about these things?

5

Hi,
Please try this and see if it works:

(Record)
appsrc ! video/x-raw, framerate=30/1, width=1280, height=720, format=NV24 ! videoconvert ! nvvidconv ! video/x-raw(memory:NVMM) ! nvv4l2h264enc bitrate=67108864 enable-lossless=true maxperf-enable=true ! filesink location=/mnt/recorded.h264

filesrc location=/mnt/recorded.h264 ! video/x-h264 ! h264parse ! avdec_h264 ! videoconvert ! video/x-raw, framerate=30/1, width=1280, height=720, format=NV24 ! appsink emit-signals=true

In H264, lossless encoding is supported but decoding is not supported. There was a typo in the comment and I have made correction.

7

I have tried recording and loading image data as you guided using nvv4l2h264enc encoder and avdec_h264 decoder, but it still fails. I wonder that if the encoder and decoders- nvv4l2h264enc, x264enc, nvv4l2decoder, avdec_h264 have use comparabilities. for example, if a video file recorded using x264enc encoder, then comparable decoder must be used, like avdec_h264, for decoding data.

8

Hi,
Could you try h265? Since we support lossless encoding/decoding in h265, it may be good to encode into h265 stream.

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.