lossless HP encoder producing serious artifacts

iko79 · February 7, 2017, 12:41pm

Hi,

I’m trying to encode kinect depth data with H.264 using lossless compression. As my code doesn’t produce valid output, I tried the NvEncoder sample and found that it produces the same output, with serios artifacts building up over only a few seconds.

When I run NvEncoder as follows

NvEncoder.exe -i depth.yuv -o depthlossless.mp4 -size 512 424 -codec 0 -preset lossless -devicetype 2 -inputFormat 0

I get this:
External Media

When I encode with lossy compression, like this

NvEncoder.exe -i depth.yuv -o depthlossy.mp4 -size 512 424 -codec 0 -preset lowLatencyHP -devicetype 2 -inputFormat 0

it works:
External Media

I tried around with different settings and devicetypes, but couldn’t get it to work. I’m running a GTX 980 with the latest driver (378.49).

You can find the raw yuv data I used here.

I’m stuck – any ideas what could be the issue here?

EDIT:
screenshots taken with MPC, which looks the same when played back with my OpenGL program, using NvDecode for decoding. VLC completely refuses to play the file; not sure however, what API it uses and if lossless decoding is implemented in the first place.

Vignesh_Ungrapalli · February 9, 2017, 9:16am

Hi,
Thanks for reporting the issue. We tried this at our end with the yuv you shared but could not reproduce the issue.

Could you try playback with ffplay in place of MPC/VLC?
What is the header version (SDK version) that you are using?

Thanks

iko79 · February 10, 2017, 11:43am

You are right, it works with ffplay. I’ll investigate and post the solution in case I find out what the problem was… Thanks for the reply.

iko79 · February 13, 2017, 3:27pm

What gives me some headache right now is that, while ffplay can indeed play back the file, there seems to be something wrong with it. In my application, I encode and stream via RTSP/RTP using live555. For debugging, I write the sent data into a “raw” .H264 file as well. This is the file that causes problems with MPC/VLC but can be played back with ffplay without any issues. On the client (running on the same machine for testing), I receive RTP-packages and write them into a debug file as well. What comes out there, even ffplay cannot deal with anymore. I get serious artifacts and many errors print to the console.

[h264 @ 0000000001c58c40] non-existing PPS 5 referenced  0B f=0/0
[h264 @ 0000000001c58c40] Truncating likely oversized PPS (59999 > 4096)
[h264 @ 0000000001c58c40] FMO not supported
[h264 @ 0000000001c58c40] non-existing SPS 8 referenced in buffering period
[h264 @ 0000000001c58c40] Truncating likely oversized SPS
[h264 @ 0000000001c58c40] illegal aspect ratio
[h264 @ 0000000001c58c40] Truncating likely oversized PPS (59998 > 4096)
[h264 @ 0000000001c58c40] sps_id 6 out of range
[h264 @ 0000000001c58c40] Truncating likely oversized PPS (59996 > 4096)
[h264 @ 0000000001c58c40] sps_id 2 out of range
[h264 @ 0000000001c58c40] Truncating likely oversized PPS (60000 > 4096)
Invalid UE golomb code
[h264 @ 0000000001c58c40] FMO not supported
[h264 @ 0000000001c58c40] reference overflow (pps)
[h264 @ 0000000001c473a0] Stream #0: not enough frames to estimate rate; conside
r increasing probesize
Input #0, h264, from 'depthRTPClient.mp4':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High 4:4:4 Predictive), yuv420p(progressive), 960x
540 [SAR 1:1 DAR 16:9], 29.92 fps, 30 tbr, 1200k tbn, 60 tbc
[h264 @ 00000000026c0540] corrupted macroblock 49 33 (total_coeff=-1)
[h264 @ 00000000026c0540] error while decoding MB 49 33
[h264 @ 00000000026c0540] concealing 60 DC, 60 AC, 60 MV errors in P frame
[h264 @ 0000000002cbe320] slice type 12 too large at 0   0B f=0/0
[h264 @ 0000000002cbe320] decode_slice_header error
[h264 @ 0000000002cbe320] no frame!
[h264 @ 0000000002cbe7c0] Invalid level prefix9KB sq=    0B f=0/0
[h264 @ 0000000002cbe7c0] error while decoding MB 53 32
[h264 @ 0000000002cbe7c0] concealing 116 DC, 116 AC, 116 MV errors in P frame
[h264 @ 0000000002b91e80] Invalid NAL unit 8, skipping.  0B f=0/0
[h264 @ 0000000002b91e80] Invalid level prefix
[h264 @ 0000000002b91e80] error while decoding MB 33 31
[h264 @ 0000000002b91e80] concealing 196 DC, 196 AC, 196 MV errors in P frame
[h264 @ 0000000002d26ba0] Invalid NAL unit 0, skipping.  0B f=0/0
[h264 @ 0000000002d26ba0] Invalid level prefix
[h264 @ 0000000002d26ba0] error while decoding MB 33 31
[h264 @ 0000000002d26ba0] concealing 196 DC, 196 AC, 196 MV errors in P frame
[h264 @ 00000000028c00a0] Invalid NAL unit 0, skipping.  0B f=0/0
[h264 @ 00000000028c00a0] Invalid level prefix
[h264 @ 00000000028c00a0] error while decoding MB 35 30
[h264 @ 00000000028c00a0] concealing 254 DC, 254 AC, 254 MV errors in P frame
[h264 @ 00000000026c0540] Invalid NAL unit 0, skipping.  0B f=0/0
Invalid UE golomb code
[h264 @ 00000000026c0540] cbp too large (3199971767) at 8 32
[h264 @ 00000000026c0540] error while decoding MB 8 32
[h264 @ 00000000026c0540] concealing 161 DC, 161 AC, 161 MV errors in P frame
[h264 @ 0000000002cbe320] Invalid level prefix
[h264 @ 0000000002cbe320] error while decoding MB 30 32
[h264 @ 0000000002cbe320] concealing 139 DC, 139 AC, 139 MV errors in P frame
[h264 @ 0000000002d26ba0] Invalid NAL unit 0, skipping.  0B f=0/0
[h264 @ 0000000002d26ba0] Invalid level prefix
[h264 @ 0000000002d26ba0] error while decoding MB 28 33
[h264 @ 0000000002d26ba0] concealing 81 DC, 81 AC, 81 MV errors in P frame
[h264 @ 00000000028c00a0] Invalid level prefix0KB sq=    0B f=0/0
[h264 @ 00000000028c00a0] error while decoding MB 35 32
[h264 @ 00000000028c00a0] concealing 134 DC, 134 AC, 134 MV errors in P frame
[h264 @ 00000000028c0540] Truncating likely oversized PPS (59999 > 4096)
[h264 @ 00000000028c0540] FMO not supported
[h264 @ 00000000028c0540] Invalid level prefix
[h264 @ 00000000028c0540] error while decoding MB 32 31
[h264 @ 00000000028c0540] concealing 197 DC, 197 AC, 197 MV errors in P frame
[h264 @ 00000000026c00a0] non-existing SPS 8 referenced in buffering period
[h264 @ 00000000026c00a0] out of range intra chroma pred mode
[h264 @ 00000000026c00a0] error while decoding MB 20 31
[h264 @ 00000000026c00a0] concealing 209 DC, 209 AC, 209 MV errors in P frame
[h264 @ 00000000026c0540] Invalid NAL unit 1, skipping.  0B f=0/0
[h264 @ 00000000026c0540] corrupted macroblock 55 30 (total_coeff=-1)
[h264 @ 00000000026c0540] error while decoding MB 55 30
[h264 @ 00000000026c0540] data partitioning is not implemented. Update your FFmp
eg version to the newest one from Git. If the problem still occurs, it means tha
t your file has a feature which has not been implemented.
[h264 @ 00000000026c0540] If you want to help, upload a sample of this file to f
tp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. (ffm
peg-devel@ffmpeg.org)
[h264 @ 00000000026c0540] concealing 234 DC, 234 AC, 234 MV errors in P frame
[h264 @ 0000000002cbe320] Invalid NAL unit 25, skipping. 0B f=0/0
[h264 @ 0000000002cbe320] concealing 213 DC, 213 AC, 213 MV errors in P frame
[h264 @ 0000000002cbe7c0] out of range intra chroma pred mode=0/0
[h264 @ 0000000002cbe7c0] error while decoding MB 13 32
[h264 @ 0000000002cbe7c0] concealing 156 DC, 156 AC, 156 MV errors in P frame
[h264 @ 0000000002b91e80] Invalid level prefix2KB sq=    0B f=0/0
[h264 @ 0000000002b91e80] error while decoding MB 33 32
[h264 @ 0000000002b91e80] concealing 136 DC, 136 AC, 136 MV errors in P frame
[h264 @ 0000000002d26ba0] Invalid NAL unit 1, skipping.  0B f=0/0
[h264 @ 0000000002d26ba0] Invalid level prefix
[h264 @ 0000000002d26ba0] error while decoding MB 58 32
[h264 @ 0000000002d26ba0] concealing 111 DC, 111 AC, 111 MV errors in P frame
[h264 @ 00000000028c00a0] Invalid NAL unit 1, skipping.  0B f=0/0
[h264 @ 00000000028c0540] Invalid level prefix0KB sq=    0B f=0/0
[h264 @ 00000000028c0540] error while decoding MB 29 33
[h264 @ 00000000028c0540] concealing 80 DC, 80 AC, 80 MV errors in P frame
[NULL @ 0000000001c58c40] Truncating likely oversized PPS (59997 > 4096)
[NULL @ 0000000001c58c40] sps_id 32 out of range
[h264 @ 00000000028c00a0] Invalid NAL unit 0, skipping.  0B f=0/0
[h264 @ 00000000028c00a0] corrupted macroblock 57 32 (total_coeff=-1)
[h264 @ 00000000028c00a0] error while decoding MB 57 32
[h264 @ 00000000028c00a0] concealing 112 DC, 112 AC, 112 MV errors in P frame
[h264 @ 00000000028c0540] Invalid NAL unit 30, skipping. 0B f=0/0
[h264 @ 00000000028c0540] out of range intra chroma pred mode
[h264 @ 00000000028c0540] error while decoding MB 59 32
[h264 @ 00000000028c0540] concealing 110 DC, 110 AC, 110 MV errors in P frame
[h264 @ 00000000026c00a0] corrupted macroblock 58 32 (total_coeff=-1)
[h264 @ 00000000026c00a0] error while decoding MB 58 32
[h264 @ 00000000026c00a0] concealing 111 DC, 111 AC, 111 MV errors in P frame
[h264 @ 00000000026c0540] Truncating likely oversized PPS (60000 > 4096)
Invalid UE golomb code
[h264 @ 00000000026c0540] FMO not supported
[h264 @ 00000000026c0540] reference overflow (pps)
[h264 @ 0000000002cbe320] corrupted macroblock 19 31 (total_coeff=-1)
[h264 @ 0000000002cbe320] error while decoding MB 19 31
[h264 @ 0000000002cbe320] concealing 210 DC, 210 AC, 210 MV errors in P frame
[h264 @ 0000000002cbe7c0] Invalid NAL unit 25, skipping. 0B f=0/0
[h264 @ 0000000002cbe7c0] Invalid level prefix
[h264 @ 0000000002cbe7c0] error while decoding MB 21 31
[h264 @ 0000000002cbe7c0] concealing 208 DC, 208 AC, 208 MV errors in P frame
[h264 @ 0000000001cb8e20] Invalid level prefix1KB sq=    0B f=0/0
[h264 @ 0000000001cb8e20] error while decoding MB 28 32
[h264 @ 0000000001cb8e20] concealing 141 DC, 141 AC, 141 MV errors in P frame
[h264 @ 0000000002d26ba0] out of range intra chroma pred mode=0/0
[h264 @ 0000000002d26ba0] error while decoding MB 55 32
[h264 @ 0000000002d26ba0] concealing 114 DC, 114 AC, 114 MV errors in P frame
[h264 @ 00000000028c00a0] Invalid NAL unit 0, skipping.  0B f=0/0
[h264 @ 00000000028c00a0] Invalid level prefix
[h264 @ 00000000028c00a0] error while decoding MB 16 33
[h264 @ 00000000028c00a0] concealing 93 DC, 93 AC, 93 MV errors in P frame
[h264 @ 00000000028c0540] Invalid NAL unit 30, skipping. 0B f=0/0
Invalid UE golomb code
[h264 @ 00000000028c0540] cbp too large (3199971767) at 56 33
[h264 @ 00000000028c0540] error while decoding MB 56 33
[h264 @ 00000000028c0540] concealing 53 DC, 53 AC, 53 MV errors in P frame
[h264 @ 00000000026c00a0] corrupted macroblock 21 33 (total_coeff=-1)
[h264 @ 00000000026c00a0] error while decoding MB 21 33
[h264 @ 00000000026c00a0] concealing 88 DC, 88 AC, 88 MV errors in P frame
[h264 @ 00000000026c0540] Invalid NAL unit 1, skipping.  0B f=0/0
    Last message repeated 1 times
[h264 @ 00000000026c0540] corrupted macroblock 27 31 (total_coeff=-1)
[h264 @ 00000000026c0540] error while decoding MB 27 31
[h264 @ 00000000026c0540] concealing 202 DC, 202 AC, 202 MV errors in P frame
[h264 @ 0000000002cbe320]     nan M-V:    nan fd=   1 aq=    0KB vq= 2326KB sq=
Invalid NAL unit 21, skipping.
[h264 @ 0000000002cbe320] Invalid level prefix
[h264 @ 0000000002cbe320] error while decoding MB 55 29
[h264 @ 0000000002cbe320] concealing 294 DC, 294 AC, 294 MV errors in P frame
[h264 @ 0000000002cbe7c0] Invalid level prefix6KB sq=    0B f=0/0
[h264 @ 0000000002cbe7c0] error while decoding MB 45 29
[h264 @ 0000000002cbe7c0] concealing 304 DC, 304 AC, 304 MV errors in P frame
[h264 @ 0000000001cb8e20] Invalid level prefix6KB sq=    0B f=0/0
[h264 @ 0000000001cb8e20] error while decoding MB 54 29
[h264 @ 0000000001cb8e20] concealing 295 DC, 295 AC, 295 MV errors in P frame
[h264 @ 0000000002d26ba0] Invalid NAL unit 25, skipping. 0B f=0/0
[h264 @ 0000000002d26ba0] Invalid level prefix
[h264 @ 0000000002d26ba0] error while decoding MB 24 29
[h264 @ 0000000002d26ba0] concealing 325 DC, 325 AC, 325 MV errors in P frame
[h264 @ 00000000028c00a0] negative number of zero coeffs at 47 29
[h264 @ 00000000028c00a0] error while decoding MB 47 29
[h264 @ 00000000028c00a0] concealing 302 DC, 302 AC, 302 MV errors in P frame
[h264 @ 00000000028c0540] Truncating likely oversized PPS (59997 > 4096)
[h264 @ 00000000028c0540] sps_id 32 out of range
[h264 @ 00000000028c0540] corrupted macroblock 39 30 (total_coeff=-1)
[h264 @ 00000000028c0540] error while decoding MB 39 30
[h264 @ 00000000028c0540] concealing 250 DC, 250 AC, 250 MV errors in P frame
[h264 @ 00000000026c0540] Invalid level prefix3KB sq=    0B f=0/0
[h264 @ 00000000026c0540] error while decoding MB 29 30
[h264 @ 00000000026c0540] concealing 260 DC, 260 AC, 260 MV errors in P frame
[h264 @ 0000000002cbe320] Invalid NAL unit 21, skipping. 0B f=0/0
[h264 @ 0000000002cbe320] Invalid level prefix
[h264 @ 0000000002cbe320] error while decoding MB 35 30
[h264 @ 0000000002cbe320] concealing 254 DC, 254 AC, 254 MV errors in P frame
[h264 @ 0000000002cbe7c0] Invalid level prefix3KB sq=    0B f=0/0
[h264 @ 0000000002cbe7c0] error while decoding MB 41 31
[h264 @ 0000000002cbe7c0] concealing 188 DC, 188 AC, 188 MV errors in P frame
[h264 @ 0000000002d26ba0] Invalid NAL unit 25, skipping. 0B f=0/0
[h264 @ 0000000002d26ba0] corrupted macroblock 40 33 (total_coeff=-1)
[h264 @ 0000000002d26ba0] error while decoding MB 40 33
[h264 @ 0000000002d26ba0] concealing 69 DC, 69 AC, 69 MV errors in P frame
    nan M-V:    nan fd=   1 aq=    0KB vq= 2500KB sq=    0B f=0/0

The same works without any hickups with lossy compression. Now it’s hard for me to spot the issue, there could out-of-order or lost packages on the client (unlikely when sending to loopback?), but it seems to me that the original video data is already faulty. While ffplay apparently can deal with that, MPC, VLC, and the live555 H264 payloader seem to have problems with it. I must say I’m neiter a specialist in H264 nor in RTP, I don’t really know where to start or how I can isolate the issue – does anybody have recommendations? Would you say when ffplay is able to play back the file, streaming via RTP should not introduce any additional problems? Is it more likely that my RTP code is erroneous? Or may the apparently flawed original video data be to blame?

Thanks for an suggestions.

EDIT:
Turns out that in my case the usage of live555 was to blame – the output buffer in the H264 fragmenter was too small and bytes were dropped. Why MPC/VLC are having problems however, I still don’t know.

Vignesh_Ungrapalli · February 14, 2017, 11:45am

1.We have confirmed that the lossless encoded bitstream generated by NVENC can be successfully decoded not only by FFplay, but also by JM decoder, which is a reference decoder for H.264 bitstreams. Based on this, the possibility of illegal bitstream generated by the encoder appears to be low.

2.We recommend that you save the binary payload at the input of the decoder (after RTP depacketization) to a file and send it to the decoder separately (FFplay or JM decoder). This will help you isolate whether the problem is in RTP packetization/depacketization.

Thanks