VIDIOC_DQBUF blocks

shiretu · July 29, 2018, 4:38pm

Update: Here are some logs:

Failed to query video capabilities: Inappropriate ioctl for device
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1179: BeginSequence  1280x720, bVPR = 0
TVMR: LowCorner Frequency = 0 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1600: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1674 ColorSpace = NvColorSpace_YCbCr709_ER
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 13, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 3947
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 3947
TVMR: FrameRate = 30 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 3947
******* thread v4l2_out_buf_dq will be terminated by force
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
 decoder I stats: 1000/12532674
 decoder O stats: 1000/1572864000
rescaler I stats: 0/0
rescaler O stats: 0/0
 encoder I stats: 0/0
 encoder O stats: 0/0

shiretu · July 29, 2018, 4:40pm

Please notice that we get EAGAIN for the startup too, before queuing any buffers on any of the queues

shiretu · July 29, 2018, 4:42pm

This is same identical app ran again, but this time the NVIDIA native libs behind libv4l2 handled the situation properly. I did not modify anything, just re-run the app. Also, I get a final real decoded frame with V4L2_BUF_FLAG_LAST set on it.

Failed to query video capabilities: Inappropriate ioctl for device
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1179: BeginSequence  1280x720, bVPR = 0
TVMR: LowCorner Frequency = 0 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1600: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1674 ColorSpace = NvColorSpace_YCbCr709_ER
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 13, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4097
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4097
TVMR: FrameRate = 30 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
 decoder I stats: 1000/12532674
 decoder O stats: 1000/1572864000
rescaler I stats: 0/0
rescaler O stats: 0/0
 encoder I stats: 0/0
 encoder O stats: 0/0

shiretu · July 29, 2018, 5:56pm

This is a typical deadlock that happens, and we are forced to do pthread_cancel. It has thousands of frames in the stack. Did not show all, but is quite clear that something went wrong.

The output from app:

Failed to query video capabilities: Inappropriate ioctl for device
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1179: BeginSequence  1280x720, bVPR = 0
TVMR: LowCorner Frequency = 0 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1600: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1674 ColorSpace = NvColorSpace_YCbCr709_ER
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 13, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4664
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4664
TVMR: FrameRate = 30 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
Dec: 649; Res: 0; Enc: 0
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4664

The GDB attached:

shiretu@tegra-ubuntu:~$ gdb -p 4664
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 4664
[New LWP 4665]
[New LWP 4666]
[New LWP 4667]
[New LWP 4668]
[New LWP 4669]
[New LWP 4671]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
0x0000007f9c4a6d94 in nanosleep () at ../sysdeps/unix/syscall-template.S:86
86	../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) info threads 
  Id   Target Id         Frame 
* 1    Thread 0x7f9c867000 (LWP 4664) "ubnt_avcodec_te" 0x0000007f9c4a6d94 in nanosleep () at ../sysdeps/unix/syscall-template.S:86
  2    Thread 0x7f974fa1e0 (LWP 4665) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3cfd0, mutex=0xc3cfa0) at pthread_cond_wait.c:186
  3    Thread 0x7f96cfa1e0 (LWP 4666) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3d760, mutex=0xc3d730) at pthread_cond_wait.c:186
  4    Thread 0x7f964fa1e0 (LWP 4667) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3d7d0, mutex=0xc3d7a0) at pthread_cond_wait.c:186
  5    Thread 0x7f95cfa1e0 (LWP 4668) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3dae0, mutex=0xc3dab0) at pthread_cond_wait.c:186
  6    Thread 0x7f950fa1e0 (LWP 4669) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc461b0, mutex=0xc46180) at pthread_cond_wait.c:186
  7    Thread 0x7f8e5ff1e0 (LWP 4671) "v4l2_out_buf_dq" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc46140, mutex=0xc46110) at pthread_cond_wait.c:186
(gdb) bt
#0  0x0000007f9c4a6d94 in nanosleep () at ../sysdeps/unix/syscall-template.S:86
#1  0x0000007f9c4cc258 in usleep (useconds=<optimized out>) at ../sysdeps/posix/usleep.c:32
#2  0x00000000005b4718 in ubnt::avcodec::internal::v4l2::V4l2Device::WaitForEOS (this=0xc168a0) at /home/shiretu/work/evostream/sources/ubnt_avcodec/src/v4l2/v4l2device.cpp:263
#3  0x00000000005c2238 in ubnt::avcodec::internal::v4l2::V4l2Decoder::Flush (this=0xc14f80) at /home/shiretu/work/evostream/sources/ubnt_avcodec/src/v4l2/v4l2decoder.cpp:95
#4  0x00000000005dbdfc in TestLibavComponents::Run (this=0x7fc1322fa0) at /home/shiretu/work/evostream/sources/ubnt_avcodec_tests/src/testlibavcomponents.cpp:145
#5  0x00000000005d7c60 in Test::Execute<TestLibavComponents> (argc=2, ppArgv=0x7fc13231a8) at /home/shiretu/work/evostream/sources/ubnt_avcodec_tests/include/test.h:32
#6  0x00000000005d7bd4 in main (argc=2, ppArgv=0x7fc13231a8) at /home/shiretu/work/evostream/sources/ubnt_avcodec_tests/src/main.cpp:16
(gdb) thread 7
[Switching to thread 7 (Thread 0x7f8e5ff1e0 (LWP 4671))]
#0  0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc46140, mutex=0xc46110) at pthread_cond_wait.c:186
186	pthread_cond_wait.c: No such file or directory.
(gdb) bt
#0  0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc46140, mutex=0xc46110) at pthread_cond_wait.c:186
#1  0x0000007f9c299080 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#2  0x0000007f9be4d514 in TegraV4L2_Poll_CPlane () from /usr/lib/aarch64-linux-gnu/tegra/libtegrav4l2.so
#3  0x0000007f9bea0f80 in plugin_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l/plugins/libv4l2_nvvideocodec.so
#4  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#5  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#6  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#7  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#8  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#9  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#10 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#11 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#12 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#13 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#14 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#15 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#16 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#17 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#18 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#19 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#20 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#21 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#22 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#23 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#24 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#25 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#26 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#27 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#28 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#29 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#30 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#31 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#32 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#33 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#34 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#35 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#36 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#37 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#38 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#39 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#40 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#41 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#42 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#43 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#44 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#45 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#46 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#47 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#48 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#49 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#50 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#51 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#52 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#53 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#54 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#55 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#56 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#57 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#58 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#59 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#60 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#61 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#62 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#63 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#64 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#65 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#66 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#67 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#68 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#69 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#70 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#71 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#72 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#73 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#74 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
---Type <return> to continue, or q <return> to quit---

DaneLLL · July 30, 2018, 1:19am

Hi andrei2hqw6,
In general, we suggest users use NvVideoEncoder or NvVideoDecoder. For your case, it requires further support. Please contact your region salesperson so that we can have further cooperation.

cjwti · July 30, 2018, 5:25pm

Hi DaneLLL,

I am using NvVideoDecoder and I have seen similar inconsistent behavior and unreliability as andrei2hqw6 mentions. The NvVideoDecoder implementation is quite a thin wrapper over the V4L2 implementation and I don’t see how using NvVideoDecoder is supposed to solve any of the problems he mentions.

Thanks,

Chris Richardson

DaneLLL · July 31, 2018, 1:37am

Hi Chris,
Can the issue be reproduced with 00_video_decode? If yes, please help provide steps so that we can do further check.

shiretu · August 1, 2018, 7:39am

Hi DaneLLL,

Yes, the issue(s) can be replicated with a bit modified sample code. It is not happening with unmodified sources because of pure luck, and an almost-always-winning race between main thread and the capture loop thread. 99.9999% of cases, the main thread wins the race and everything seems fine, when in fact is not fine at all.

patch: https://gist.github.com/shiretu/221193b76f5f1f26a08975042e3000c2
running output: https://gist.github.com/shiretu/5f0572b9777df11c4ce3bb3328823d07
debugger attached: https://gist.github.com/shiretu/a26da0e596ad5d936ff9b44ef3f28d5c

DaneLLL · August 2, 2018, 9:16am

We will check and update.

DaneLLL · August 10, 2018, 8:51am

The sleep is not put in correct place. Please refer to below patch

@@ -1159,6 +1159,8 @@ main(int argc, char *argv[])
                 current_file = current_file % ctx.file_count;
                 continue;
             }
+            // delay EOS
+            sleep(30);
         }
         // It is necessary to queue an empty buffer to signal EOS to the decod
er
         // i.e. set v4l2_buf.m.planes[0].bytesused = 0 and queue the buffer

shiretu · August 10, 2018, 10:21am

Hi DaneLLL,

I’m very very confused.

What do you mean the sleep is not put in a correct place? What makes your version correct?

DaneLLL · August 10, 2018, 11:42am

Your sleep is after the EOS. Once you send EOS, decoder is flushing and you should not keep calling dqBuffer when all buffers are flushed out.

shiretu · August 10, 2018, 11:54am

That’s not what V4L2 specs say:
https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/vidioc-qbuf.html#return-value

EINVAL
The buffer type is not supported, or the index is out of bounds, or no buffers have been allocated yet, or the userptr or length are invalid.

The sample implementation is avoiding a bug with very complicated buffer machinery, when in turn it should simply error out.

This is very important. What you call “sample code” is no longer sample code. Is more like use this bag of work-arounds and treat the sample code as some sort of low quality SDK. Which is very far from being OK

DaneLLL · August 11, 2018, 6:06am

Hi shiretu,
We think the current implementation is good for most cases, although it may not precisely meet v4l2 specs. We would suggest you not to call dqBuffer after you send EOS, but if you do need the precise SW implementation, please contact your region salesperson. Let’s go over business review to have further cooperation.

shiretu · August 12, 2018, 12:27am

Hi DaneLLL,

On the short term, I will account the frames in and the frames out. However, please consider patching the v4l2 plugins to at least don’t enter pthread_cond_wait when EOS was encountered, or reliably signal the V4L2_BUF_FLAG_LAST. Either of those will literally eliminate the buffers counting complexity from app layers. Besides, it is crystal clear from the logs of the plugin itself that is fully aware of when EOS was delivered on input. It is probably boiling down to an if statement just before the pthread call.

Best regards,
Andrei

cjwti · August 16, 2018, 4:33pm

Hi DaneLLL,

I would beg to differ that the current behavior is good enough for most use cases. I spent a lot of time debugging hangs, no EOS output and inconsistent shutdown behavior related to NvDecoder, naively assuming its behavior was correct and race free and that I must have been doing something wrong. In addition to that, the fact that it doesn’t actually support O_NONBLOCK is frustrating. I’m still not sure I have a solution that is 100% reliable to shutdown and restart many times due to these races and inconsistent behavior.

My customers expect the systems I am working on to run 24/7 for a year or more without rebooting and I need all of this to be rock solid. Please consider making enhancements to NvDecoder to work a bit more properly and in-line with how V4L2 devices are supposed to work.

Thanks,

Chris Richardson

DaneLLL · August 17, 2018, 1:16am

Hi Chris,
Please get in contact with NVIDIA salesperson so that we can review and prioritize this request.

caruofc · January 9, 2019, 4:38pm

Did you find any solution to your problem. I am having similar issue.

This is what I am doing. I am grabbing h264 encoded frames from an IP camera, decomposing the encoded buffer by myself to get SPS, PPS and frame buffers. Then I am packing with each of these with starting prefix 0x00,0x00,0x00,0x01 and feeding this to the decoder as NAL units. I am initializing output_plane as

ctx[channel].dec->output_plane.setupPlane(V4L2_MEMORY_MMAP, 10, true, false);

But when all 10 output_plane buffer are en queued and the program starts to dequeue first before putting more frames into the queue, it stucks in

ret = v4l2_ioctl(fd, VIDIOC_DQBUF, &v4l2_buf);

of

int
NvV4l2ElementPlane::dqBuffer(struct v4l2_buffer &v4l2_buf, NvBuffer ** buffer,
NvBuffer ** shared_buffer, uint32_t num_retries)

function.

I tried using or not using
ret = ctx[channel].dec->disableCompleteFrameInputBuffer();

but no luck. It stuck always. difference of using disableCompleteFrameInputBuffer() is the v4l2_ioctl() function succeeds upto 3rd frame (i.e., SPS, PPS, iFrame), without disableCompleteFrameInputBuffer() it stuck after dequeuing only SPS and PPS.

Need help.

BTW, for your info, my program at the moment is only feeding h264 encoded frames into the decoder. There is no code yet to read and use the decoded buffer.

More info:
This is what I get in my qt application output window, if it helps:

NvMMLiteOpen : Block : BlockType = 261
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen
NvMMLiteBlockCreate : Block : BlockType = 261
TVMR: cbBeginSequence: 1179: BeginSequence 1280x720, bVPR = 0
TVMR: LowCorner Frequency = 0
TVMR: cbBeginSequence: 1529: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0
TVMR: cbBeginSequence: 1600: Display Resolution : (1280x720)
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1280x720)
TVMR: cbBeginSequence: 1669: ColorFormat : 5
TVMR: cbBeginSequence:1680 ColorSpace = NvColorSpace_YCbCr709
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 13, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1

shiretuxiv2i · January 9, 2019, 4:48pm

Hi caruofc,

Try not to separate the SPS/PPS from the actual payload. try to send them in a single payload like this:

0001SPS0001PPS0001<REAL_NALU…>

Also, make sure you count the enqueue and dequeue operations. Otherwise it will block.

Lastly, I do not use the sample from NVidia. I find it full of work-arounds. I’m better off investing time understanding every single work around rather than invest same time or more into that sample code which is very far from being a usable SDK-like sample.

caruofc · January 9, 2019, 5:20pm

Thanks shiretuxiv2i for you quick reply.
I did try sending SPS/PPS in one payload as you described at first. That did not work either.
Here is my function which basically fill the output_plane of the decoder and once the output buffers are all queued it starts to dequeue and enqueue new encoded nal payload as it comes.
There is no eos whatsoever as its a continuous stream coming from IP camera once connected.

What happens is, first dqBuffer() operation succeeds following by qBuffer() but next time when next payload comes the dqBuffer() blocks when it tries to execute v4l2_ioctl() after experiencing EAGAIN.

bool VideoDecoder_Tegra::decode(const char* data, int len)
{
context_t* aCtx = &ctx[channel];

if (aCtx->got_error || aCtx->dec->isInError())
    return false;

int ret;
struct v4l2_buffer v4l2_buf;
struct v4l2_plane planes[MAX_PLANES];
NvBuffer *buffer;

memset(&v4l2_buf, 0, sizeof(v4l2_buf));
memset(planes, 0, sizeof(planes));

if (frame_counter<(int)aCtx->dec->output_plane.getNumBuffers()){
    buffer = aCtx->dec->output_plane.getNthBuffer(frame_counter);
    v4l2_buf.index = frame_counter;
    v4l2_buf.m.planes = planes;
    frame_counter++;
}else{
    v4l2_buf.m.planes = planes;
    ret = aCtx->dec->output_plane.dqBuffer(v4l2_buf, &buffer, NULL, -1);
    if (ret < 0)
    {
        qDebug() << "Error DQing buffer at output plane for channel " << channel;
        aCtx->got_error = true;
        return false;
    }
}

// Length is the size of the buffer in bytes
memcpy(buffer->planes[0].data, data, len);
buffer->planes[0].bytesused = len;

v4l2_buf.m.planes[0].bytesused = buffer->planes[0].bytesused;
// It is necessary to queue an empty buffer to signal EOS to the decoder
// i.e. set v4l2_buf.m.planes[0].bytesused = 0 and queue the buffer
ret = aCtx->dec->output_plane.qBuffer(v4l2_buf, NULL);
if (ret < 0)
{
    qDebug() << "Error Qing buffer at output plane for channel " << channel;
    aCtx->got_error = true;
    return false;
}

return true;

}