Clarification about h264 parsing with nvv4l2h264enc / nvv4l2decoder

Honey_Patouceul · June 10, 2020, 7:43pm

Hi everybody,

I have been facing a case with R32.4.2 where RTSP streaming with test-launch with nvv4l2h264enc gave me different results when trying to decode on client side with omxh264dec or nvv4l2decoder. Seems the received stream is already parsed, and with nvv4l2decoder using h264parse on receiver side leads to stall.

I’ve narrowed it to a simple local case. This works fine:

gst-launch-1.0 -v nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1' ! nvv4l2h264enc ! nvv4l2decoder ! videoconvert ! fpsdisplaysink video-sink= fakesink text-overlay=false
[...]
/GstPipeline:pipeline0/GstVideoConvert:videoconvert0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 16, dropped: 0, current: 31,87, average: 31,87
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 31, dropped: 0, current: 29,88, average: 30,88
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 47, dropped: 0, current: 30,04, average: 30,59
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 62, dropped: 0, current: 29,97, average: 30,44
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 78, dropped: 0, current: 30,01, average: 30,35
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0: last-message = rendered: 93, dropped: 0, current: 29,99, average: 30,29

But this doesn’t:

gst-launch-1.0 -v nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1' ! nvv4l2h264enc ! h264parse ! nvv4l2decoder ! videoconvert ! fpsdisplaysink video-sink= fakesink text-overlay=false

It stalls with this output:

gst-launch-1.0 -v nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1' ! nvv4l2h264enc ! h264parse ! nvv4l2decoder ! videoconvert ! fpsdisplaysink video-sink= fakesink text-overlay=false 
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:fpsdisplaysink0/GstFakeSink:fakesink0: sync = true
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/nvv4l2h264enc:nvv4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, level=(string)NULL, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, level=(string)NULL, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg, parsed=(boolean)true
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, level=(string)NULL, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg, parsed=(boolean)true
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, level=(string)NULL, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
/GstPipeline:pipeline0/nvv4l2h264enc:nvv4l2h264enc0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)NV12, framerate=(fraction)30/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2592 x 1944 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 16,000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 2592 x 1458 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 16,000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 1280 x 720 FR = 120,000005 fps Duration = 8333333 ; Analog Gain range min 1,000000, max 16,000000; Exposure Range min 22000, max 358733000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 120,000005 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
H264: Profile = 66, Level = 0 
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)constrained-baseline, level=(string)3, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt601, chroma-site=(string)jpeg, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:13.668327198
Setting pipeline to PAUSED ...
Setting pipeline to READY ...

After a first Ctrl-C, nothing happens. After a second Ctrl-C, it shuts down, but following messages are seen in syslog from Argus:

Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error InvalidState:  (propagating from src/services/gl/EGLStreamProducer.cpp, function ~EGLStreamProducer(), line 50)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: === gst-launch-1.0[12221]: WARNING: Cleaning up 1 outstanding stream settings...=== gst-launch-1.0[12221]: WARNING: Cleaning up 1 outstanding sessions...(NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:498) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter: Buffer is not pending (in src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 502)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBuffer(), line 487)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter:  (propagating from src/components/CaptureContainerImpl.cpp, function returnBuffer(), line 447)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter:  (propagating from src/components/stages/BufferReturnStage.h, function doExecute(), line 43)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 990)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:498) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:534) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 539)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:552) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:498) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:534) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 539)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:552) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:498) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:534) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 539)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:552) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:498) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:534) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: SCF: Error BadParameter:  (propagating from src/services/gl/EGLStreamProducer.cpp, function presentBufferInternal(), line 539)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex not initialized (/dvs/git/dirty/git-master_linux/camera/core_scf/src/services/gl/EGLStreamProducer.cpp:552) (in Mutex.cpp, function lock(), line 79)
Jun 10 21:39:28 xavier nvargus-daemon[11574]: (NvCameraUtils) Error InvalidState: Mutex has not been initialized (in Mutex.cpp, function unlock(), line 88)
Jun 10 21:39:33 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 655
Jun 10 21:39:33 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 654
Jun 10 21:39:33 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 653
Jun 10 21:39:33 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 652
Jun 10 21:39:33 xavier nvargus-daemon[11574]: SCF: Error Timeout: waitForIdle() timed out (in src/api/Session.cpp, function waitForIdleLocked(), line 920)
Jun 10 21:39:33 xavier nvargus-daemon[11574]: (Argus) Error Timeout:  (propagating from src/api/CaptureSessionImpl.cpp, function destroy(), line 166)
Jun 10 21:39:38 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 655
Jun 10 21:39:38 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 654
Jun 10 21:39:38 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 653
Jun 10 21:39:38 xavier nvargus-daemon[11574]: waitForIdleLocked remaining request 652
Jun 10 21:39:38 xavier nvargus-daemon[11574]: SCF: Error Timeout: waitForIdle() timed out (in src/api/Session.cpp, function waitForIdleLocked(), line 920)
Jun 10 21:39:38 xavier nvargus-daemon[11574]: SCF: Error Timeout:  (propagating from src/api/Session.cpp, function abortCaptures(), line 886)
Jun 10 21:39:38 xavier nvargus-daemon[11574]: SCF: Error Timeout:  (propagating from src/api/Session.cpp, function shutdown(), line 400)
Jun 10 21:39:38 xavier nvargus-daemon[11574]: PowerServiceCore:handleRequests: timePassed = 5031
Jun 10 21:39:38 xavier nvargus-daemon[11574]: SCF: Error Timeout:  (propagating from src/api/Session.cpp, function shutdown(), line 500)
Jun 10 21:39:38 xavier nvargus-daemon[11574]: SCF: Error Timeout:  (propagating from src/api/CameraDriver.cpp, function deleteSession(), line 619)
Jun 10 21:39:38 xavier nvargus-daemon[11574]: (Argus) Error Timeout:  (propagating from src/api/CaptureSessionImpl.cpp, function destroy(), line 191)
Jun 10 21:39:38 xavier nvargus-daemon[11574]: === gst-launch-1.0[12221]: NOTE: Destroy all libargus objects before destroying the CameraProvider to avoid these warnings.FiberScheduler: 4 fibers still exist!
Jun 10 21:39:38 xavier nvargus-daemon[11574]: === gst-launch-1.0[12221]: Connection cleaned up (7F8888D1D0)free(): invalid pointer
Jun 10 21:39:38 xavier systemd[1]: nvargus-daemon.service: Main process exited, code=killed, status=6/ABRT
Jun 10 21:39:38 xavier systemd[1]: nvargus-daemon.service: Failed with result 'signal'.
Jun 10 21:39:38 xavier systemd[1]: nvargus-daemon.service: Service hold-off time over, scheduling restart.
Jun 10 21:39:38 xavier systemd[1]: nvargus-daemon.service: Scheduled restart job, restart counter is at 9.
Jun 10 21:39:38 xavier systemd[1]: Stopped Argus daemon.
Jun 10 21:39:38 xavier systemd[1]: Started Argus daemon.

Might someone advise about this ?
Thanks

DaneLLL · June 11, 2020, 11:06pm

Hi,
Looks like if h264parse is present between nvv4l2h264enc and nvv4l2decoder, the issue happens. Will check and update. Thanks.

DaneLLL · June 18, 2020, 9:07am

Hi,
We don’t observe the issue if we save the h264 stream and then do video playback:

$ gst-launch-1.0 nvarguscamerasrc num-buffers=300 ! 'video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1' ! nvv4l2h264enc ! h264parse ! video/x-h264,stream-format=byte-stream ! filesink location=a.264
$ gst-launch-1.0 filesrc location=a.264 ! h264parse ! nvv4l2decoder ! nvoverlaysink

So the h264 stream should be valid. Need to investigate why it fails in linking in single pipeline.

Honey_Patouceul · June 18, 2020, 12:49pm

Hi @DaneLLL,

Thanks for your investigation.

I’ve noticed that double stream format conversion byte-stream->avc->byte-stream seems working:

gst-launch-1.0 videotestsrc ! nvvidconv ! 'video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1, format=NV12' ! nvv4l2h264enc ! video/x-h264, stream-format=byte-stream ! h264parse ! video/x-h264, stream-format=avc ! h264parse ! video/x-h264, stream-format=byte-stream ! nvv4l2decoder ! nvoverlaysink

Not sure it is related, but you may also try this:

# Works fine
gst-launch-1.0 videotestsrc ! nvvidconv ! 'video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1, format=NV12' ! nvv4l2h264enc ! nvv4l2decoder ! nvoverlaysink

# Does not work
gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1, format=NV12' ! nvv4l2h264enc ! nvv4l2decoder ! nvoverlaysink
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Redistribute latency...
NvMMLiteOpen : Block : BlockType = 4 
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2592 x 1944 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 16,000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 2592 x 1458 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 16,000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 1280 x 720 FR = 120,000005 fps Duration = 8333333 ; Analog Gain range min 1,000000, max 16,000000; Exposure Range min 22000, max 358733000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 120,000005 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
H264: Profile = 66, Level = 0 
WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
There may be a timestamping problem, or this computer is too slow.

Please fork to a new topic if you think it’s unrelated.

DaneLLL · July 9, 2020, 8:32am

Hi,
It works fine if we add muxer/demuxer:

$ gst-launch-1.0 videotestsrc ! nvvidconv ! nvv4l2h264enc! h264parse ! matroskamux ! matroskademux ! h264parse ! nvv4l2decoder ! nvoverlaysink

Could you try this?

Honey_Patouceul · July 10, 2020, 10:26am

Hi @DaneLLL,

Thanks for your reply, but I think this is just using the double conversion through avc workaround that I mentioned above, adding here extra overhead of container, so no improvement so far…

janardhan.reddi.g · October 10, 2020, 1:44pm

I too faced the same issue you described above with below pipeline. Were you able to resolve the issue. If yes, please share the pipeline.
gst-launch-1.0 nvarguscamerasrc ! ‘video/x-raw(memory:NVMM), width=640, height=480, framerate=30/1, format=NV12’ ! nvv4l2h264enc ! nvv4l2decoder ! nvoverlaysink

Honey_Patouceul · October 10, 2020, 2:56pm

You just need to avoid h264parse before nvv4l2decoder when the h264 stream has byte-stream format.
Or use conversion through avc format.

Honey_Patouceul · December 3, 2020, 11:48am

This bug has been fixed in R32.4.4.