Lossless or near lossless video compression on the TX2

I am trying to simultaneously compress 4 GigE Vision camera streams coming into the TX2 via ethernet.
The frame rate for each camera is around 5Hz. Each video stream is around 4 mega pixels at 10 bits per pixel (monochrome).
I need the compression to be lossless or very near lossless.
Each compressed video stream will need to be written to disk as new frames are processed.

Is this possible?

I have seen this NVEnc library, but I think this is only for x86 desktop PCs

I have also looked at the Jetson MultiMedia API. Another possibility is GStreamer.

Which is the correct API to do what I want?

Does any example code exist? I have not so far seen any examples with lossless or monochrome video compression.

Please refer to
https://devtalk.nvidia.com/default/topic/1026493/jetson-tx1/lossless-h-264-encoding/post/5220785/#5220785

In the link you sent, they are using gstreamer to do lossless encoding via the command:

$ gst-launch-1.0 nvcamerasrc num-buffers=150 ! omxh264enc control-rate=0 qp-range=0,1:0,1:0,1 quant-i-frames=0 quant-p-frames=0 ! 'video/x-h264,stream-format=byte-stream' ! filesink location=a.h264

I would like to accomplish something similar using one of the samples included with the Tegra Multimedia API.

When I run the sample program “01_video_encode” without any arguments, I get the following output:

video_encode <in-file> <in-width> <in-height> <encoder-type> <out-file> [OPTIONS]

Encoder Types:
	H264
	H265

OPTIONS:
	-h,--help             Prints this text
	--dbg-level <level>   Sets the debug level [Values 0-3]

	-br <bitrate>         Bitrate [Default = 4000000]
	-p <profile>          Encoding Profile [Default = baseline]
	-rc <rate-control>    Ratecontrol mode [Default = cbr]
	-ifi <interval>       I-frame Interval [Default = 30]
	-idri <interval>      IDR Interval [Default = 256]
	--insert-spspps-idr   Insert SPS PPS at every IDR [Default = disabled]
	-fps <num> <den>      Encoding fps in num/den [Default = 30/1]

	-tt <level>           Temporal Tradeoff level [Default = 0]
	-vbs <size>           Virtual buffer size [Default = 0]
	-nrf <num>            Number of reference frames [Default = 1]

	-slt <type>           Slice length type (1 = Number of MBs, 2 = Bytes) [Default = 1]
	-hpt <type>           HW preset type (1 = ultrafast, 2 = fast, 3 = medium,  4 = slow)
	-slen <length>        Slice length [Default = 0]
	-sir <interval>       Slice intrarefresh interval [Default = 0]

	-nbf <num>            Number of B frames [Default = 0]

	-rpc <string>         Change configurable parameters at runtime

	-goldcrc <string>     GOLD CRC

	--rcrc                Reconstructed surface CRC

	-rl <cordinate>       Reconstructed surface Left cordinate [Default = 0]

	-rt <cordinate>       Reconstructed surface Top cordinate [Default = 0

	-rw <val>             Reconstructed surface width

	-rh <val>             Reconstructed surface height

	-rcrcf <reconref_file_path> Specify recon crc reference param file

	--report-metadata     Print encoder output metadata
	--input-metadata      Enable encoder input metadata
	--mvdump              Dump encoded motion vectors

	--eroi                Enable ROI [Default = disabled]

	-roi <roi_file_path>  Specify roi param file

	--erps                Enable External RPS [Default = disabled]

	--egdr                Enable GDR [Default = disabled]

	--gif                 Enable Gaps in FrameNum [Default = disabled]

	-fnb <num_bits>       H264 FrameNum bits [Default = 0]

	-plb <num_bits>       H265 poc lsb bits [Default = 0]

	--ni                  No I-frames [Default = disabled]

	-rpsf <rps_file_path> Specify external rps param file

	--erh                 Enable External picture RC [Default = disabled]

	-gdrf <gdr_file_path> Specify GDR Parameters filename 

	-gdrof <gdr_out_file_path> Specify GDR Out filename 

	-smq <max_qp_value>   Max QP per session when external picture RC enabled

	-hf <hint_file_path>  Specify external rate control param file

	-MinQpI               Specify minimum Qp Value for I frame

	-MaxQpI               Specify maximum Qp Value for I frame

	-MinQpP               Specify minimum Qp Value for P frame

	-MaxQpP               Specify maximum Qp Value for P frame

	-MinQpB               Specify minimum Qp Value for B frame

	-MaxQpB               Specify maximum Qp Value for B frame

Do you know what the correct settings would be to get this example to compress in lossless mode?

Hi ceres.imaging,
This has not been supported in tegra_multimedia_pi. We will check and update.

Hi DaneLLL,

We’d like to compress 10-bit grayscale to lossless 10-bit HEVC (H.265) on a TX2.

I’d prefer to use ffmpeg rather than gstreamer since the documentation seems much easier to understand. Do I understand correctly that only gstreamer is supported with NVENC on a TX2 (as compared with desktop/laptop GPUs where there are more options)?

Could you suggest how we might setup a gstreamer pipeline to accomplish 10-bit Lossless HEVC encoding? We can’t find documentation on using 10-bit channels with gstreamer, and gst-inspect seems to suggest that only 8-bit YUV color planes are supported as inputs. I hope we’re misunderstanding!

Thanks,
Seth Nickell
VP of Engineering @ Ceres Imaging

Hi seth0dazx,
We support 10-bit HEVC decoding but not encoding.

DaneLLL,

I am able to modify the example code in Multimedia sample:
/samples/01_video_encode

and set the following parameters:

ctx->nMinQpI = 0;
ctx->nMaxQpI = 1;
ctx->nMinQpP = 0;
ctx->nMaxQpP = 1;
ctx->nMinQpB = 0;
ctx->nMaxQpB = 1;

which will result in close to lossless video compression. I am seeing maximum errors around 1%
(i.e. the gray values are off by 1 to 2 counts) after compression is performed on the TX2.

Are there any other parameters that I need to set on the context_t struct that will enable fully lossless compression using the TX2 compression hardware block?

typedef struct
{
    NvVideoEncoder *enc;
    uint32_t encoder_pixfmt;

    char *in_file_path;
    std::ifstream *in_file;

    uint32_t width;
    uint32_t height;

    char *out_file_path;
    std::ofstream *out_file;

    char *ROI_Param_file_path;
    char *Recon_Ref_file_path;
    char *RPS_Param_file_path;
    char *hints_Param_file_path;
    char *GDR_Param_file_path;
    char *GDR_out_file_path;
    std::ifstream *roi_Param_file;
    std::ifstream *recon_Ref_file;
    std::ifstream *rps_Param_file;
    std::ifstream *hints_Param_file;
    std::ifstream *gdr_Param_file;
    std::ofstream *gdr_out_file;

    uint32_t bitrate;
    uint32_t profile;
    enum v4l2_mpeg_video_bitrate_mode ratecontrol;
    uint32_t iframe_interval;
    uint32_t idr_interval;
    enum v4l2_mpeg_video_h264_level level;
    uint32_t fps_n;
    uint32_t fps_d;
    uint32_t gdr_start_frame_number; /* Frame number where GDR has to be started */
    uint32_t gdr_num_frames; /* Number of frames where GDR to be applied */
    uint32_t gdr_out_frame_number; /* Frames number from where encoded buffers are to be dumped */
    enum v4l2_enc_temporal_tradeoff_level_type temporal_tradeoff_level;
    enum v4l2_enc_hw_preset_type hw_preset_type;
    v4l2_enc_slice_length_type slice_length_type;
    uint32_t slice_length;
    uint32_t virtual_buffer_size;
    uint32_t num_reference_frames;
    uint32_t slice_intrarefresh_interval;
    uint32_t num_b_frames;
    uint32_t nMinQpI;              /* Minimum QP value to use for index frames */
    uint32_t nMaxQpI;              /* Maximum QP value to use for index frames */
    uint32_t nMinQpP;              /* Minimum QP value to use for P frames */
    uint32_t nMaxQpP;              /* Maximum QP value to use for P frames */
    uint32_t nMinQpB;              /* Minimum QP value to use for B frames */
    uint32_t nMaxQpB;              /* Maximum QP value to use for B frames */
    uint32_t sMaxQp;               /* Session Maximum QP value */
    bool insert_sps_pps_at_idr;

    bool report_metadata;
    bool input_metadata;
    bool dump_mv;
    bool externalRPS;
    bool enableGDR;
    bool bGapsInFrameNumAllowed;
    bool bnoIframe;
    uint32_t nH264FrameNumBits;
    uint32_t nH265PocLsbBits;
    bool externalRCHints;
    bool enableROI;

    bool use_gold_crc;
    char gold_crc[20];
    Crc *pBitStreamCrc;

    bool bReconCrc;
    uint32_t rl;                   /* Reconstructed surface Left cordinate */
    uint32_t rt;                   /* Reconstructed surface Top cordinate */
    uint32_t rw;                   /* Reconstructed surface width */
    uint32_t rh;                   /* Reconstructed surface height */

    std::stringstream *runtime_params_str;
    uint32_t next_param_change_frame;
    bool got_error;
} context_t;

Hi ceres.imaging,
It is what we can achieve on r28.1.

In next release, we will have the following controls implemented:

V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE
V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP
V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP
V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP

We can configure QP=0 with the controls.

More information about the controls:
https://01.org/linuxgraphics/gfx-docs/drm/media/uapi/v4l/extended-controls.html?highlight=v4l2_cid_mpeg_video_frame_rc_enable

DaneLLL,

when is the next release expected to be available?