Nvidia-desktop kernel: [407343.357549] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600

Hi,
Could you share the sample with header files? We have seen errors while trying to compile it:

test.cpp: In function ‘void* pthread_Frame(void*)’:
test.cpp:3:5: error: ‘cv’ has not been declared
     cv::VideoCapture *cvCap = new cv::VideoCapture;
     ^~
test.cpp:3:23: error: ‘cvCap’ was not declared in this scope
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                       ^~~~~
test.cpp:3:35: error: ‘cv’ does not name a type
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                                   ^~
test.cpp:4:5: error: ‘cv’ has not been declared
     cv::Mat *frame = nullptr;
     ^~
test.cpp:4:14: error: ‘frame’ was not declared in this scope
     cv::Mat *frame = nullptr;
              ^~~~~
test.cpp:19:21: error: ‘cv’ does not name a type
         frame = new cv::Mat;
                     ^~
test.cpp:21:18: error: ‘cout’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                  ^~~~
test.cpp:21:43: error: ‘endl’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                                           ^~~~
test.cpp:27:17: error: ‘sleep’ was not declared in this scope
                 sleep(2);
                 ^~~~~
test.cpp:33:18: error: ‘string’ is not a member of ‘std’
             std::string pipeName = "rtspsrc location=rtsp://admin:admin@192.168
.1.10:554/h264/ch1/main/av_stream ! rtph264depay ! h264parse ! nvv4l2decoder ena
ble-max-performance=1 drop-frame-interval=" + std::to_string(dropFrameCount) + "
 ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! a
ppsink sync=true max-buffers=1 drop=true"; //get newest frame
                  ^~~~~~
test.cpp:35:25: error: ‘pipeName’ was not declared in this scope
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
test.cpp:35:25: note: suggested alternative: ‘typename’
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
                         typename
test.cpp:35:35: error: ‘cv’ has not been declared
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                                   ^~
test.cpp:37:17: error: ‘sleep’ was not declared in this scope
                 sleep(1);
                 ^~~~~
test.cpp:39:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Failed to open camera ! ex = " + pipeName<<std::end
l;
                      ^~~~
test.cpp:39:77: error: ‘endl’ is not a member of ‘std’
              std::cout<<"Failed to open camera ! ex = " + pipeName<<std::endl;
                                                                          ^~~~
test.cpp:41:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                      ^~~~
test.cpp:41:55: error: ‘endl’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                                                       ^~~~
test.cpp:46:16: error: type ‘<type error>’ argument given to ‘delete’, expected
pointer
         delete frame;
                ^~~~~
test.cpp: In function ‘int main()’:
test.cpp:53:10: error: ‘vector’ is not a member of ‘std’
     std::vector<pthread_t> pthreadIdArr;
          ^~~~~~
test.cpp:53:17: error: ‘pthread_t’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
test.cpp:53:17: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
                 pthread_Frame
test.cpp:53:28: error: ‘pthreadIdArr’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
test.cpp:53:28: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
                            pthread_Frame
test.cpp:55:19: error: expected ‘;’ before ‘pthreadHandle’
         pthread_t pthreadHandle;
                   ^~~~~~~~~~~~~
test.cpp:56:25: error: ‘pthreadHandle’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
test.cpp:56:25: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
                         pthread_Frame
test.cpp:56:9: error: ‘pthread_create’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
test.cpp:56:9: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
         pthread_Frame
test.cpp:61:10: error: ‘size_t’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
          ^~~~~~
test.cpp:61:24: error: ‘i’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
                        ^
test.cpp:62:9: error: ‘pthread_join’ was not declared in this scope
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
test.cpp:62:9: note: suggested alternative: ‘pthread_Frame’
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
         pthread_Frame
nvidia@dhcp-10-19-107-227:~$ g++ -o testcv -Wall -std=c++11 test.cpp -I/usr/loca
l/include/opencv4/ -L/usr/local/lib/ -lopencv_core -lopencv_highgui -lopencv_img
proc -lopencv_videoio
test.cpp: In function ‘void* pthread_Frame(void*)’:
test.cpp:3:5: error: ‘cv’ has not been declared
     cv::VideoCapture *cvCap = new cv::VideoCapture;
     ^~
test.cpp:3:23: error: ‘cvCap’ was not declared in this scope
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                       ^~~~~
test.cpp:3:35: error: ‘cv’ does not name a type
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                                   ^~
test.cpp:4:5: error: ‘cv’ has not been declared
     cv::Mat *frame = nullptr;
     ^~
test.cpp:4:14: error: ‘frame’ was not declared in this scope
     cv::Mat *frame = nullptr;
              ^~~~~
test.cpp:19:21: error: ‘cv’ does not name a type
         frame = new cv::Mat;
                     ^~
test.cpp:21:18: error: ‘cout’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                  ^~~~
test.cpp:21:43: error: ‘endl’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                                           ^~~~
test.cpp:27:17: error: ‘sleep’ was not declared in this scope
                 sleep(2);
                 ^~~~~
test.cpp:33:18: error: ‘string’ is not a member of ‘std’
             std::string pipeName = "rtspsrc location=rtsp://admin:admin@192.168
.1.10:554/h264/ch1/main/av_stream ! rtph264depay ! h264parse ! nvv4l2decoder ena
ble-max-performance=1 drop-frame-interval=" + std::to_string(dropFrameCount) + "
 ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! a
ppsink sync=true max-buffers=1 drop=true"; //get newest frame
                  ^~~~~~
test.cpp:35:25: error: ‘pipeName’ was not declared in this scope
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
test.cpp:35:25: note: suggested alternative: ‘typename’
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
                         typename
test.cpp:35:35: error: ‘cv’ has not been declared
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                                   ^~
test.cpp:37:17: error: ‘sleep’ was not declared in this scope
                 sleep(1);
                 ^~~~~
test.cpp:39:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Failed to open camera ! ex = " + pipeName<<std::end
l;
                      ^~~~
test.cpp:39:77: error: ‘endl’ is not a member of ‘std’
              std::cout<<"Failed to open camera ! ex = " + pipeName<<std::endl;
                                                                          ^~~~
test.cpp:41:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                      ^~~~
test.cpp:41:55: error: ‘endl’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                                                       ^~~~
test.cpp:46:16: error: type ‘<type error>’ argument given to ‘delete’, expected
pointer
         delete frame;
                ^~~~~
test.cpp: In function ‘int main()’:
test.cpp:53:10: error: ‘vector’ is not a member of ‘std’
     std::vector<pthread_t> pthreadIdArr;
          ^~~~~~
test.cpp:53:17: error: ‘pthread_t’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
test.cpp:53:17: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
                 pthread_Frame
test.cpp:53:28: error: ‘pthreadIdArr’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
test.cpp:53:28: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
                            pthread_Frame
test.cpp:55:19: error: expected ‘;’ before ‘pthreadHandle’
         pthread_t pthreadHandle;
                   ^~~~~~~~~~~~~
test.cpp:56:25: error: ‘pthreadHandle’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
test.cpp:56:25: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
                         pthread_Frame
test.cpp:56:9: error: ‘pthread_create’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
test.cpp:56:9: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
         pthread_Frame
test.cpp:61:10: error: ‘size_t’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
          ^~~~~~
test.cpp:61:24: error: ‘i’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
                        ^
test.cpp:62:9: error: ‘pthread_join’ was not declared in this scope
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
test.cpp:62:9: note: suggested alternative: ‘pthread_Frame’
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
         pthread_Frame

Would be great if you can share the sample which can be built and run directly.

Do you have one conclusion? But I still offten get error message, “Creating pipes for GWakeup: Too many open files”. @DaneLLL

Hi,
We are not able to compile the test sample and then set up the test. Would need your help to share a complete sample so that we can set up and run.

I have found that the number of command “lsof” get more and more as FD leak, when do cvCap->open rtsp stream and cvCap->release().

Then it printed following infos,

Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877093] falcon 15340000.vic: nvhost_ioctl_channel_submit: failed with err -24
Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877359] falcon 15340000.vic: submit_add_gathers: failed to copy user inputs: class_ids=00000000e40d1b88 num_cmdbufs=2
Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877559] falcon 15340000.vic: nvhost_ioctl_channel_submit: failed with err -22
Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877790] (NULL device *): nvhost_sync_fence_set_name: failed to get fence
Feb 10 15:29:10 nvidia-desktop EdgeApp-local[25036]: Creating pipes for GWakeup: Too many open files
Feb 10 16:03:18 nvidia-desktop kernel: [ 9107.764728] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:03:18 nvidia-desktop kernel: [ 9107.764937] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:08:42 nvidia-desktop EdgeApp-local[11032]: Creating pipes for GWakeup: Too many open files
Feb 10 16:12:03 nvidia-desktop kernel: [ 9632.936356] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:03 nvidia-desktop kernel: [ 9632.936601] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:03 nvidia-desktop kernel: [ 9633.177397] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:03 nvidia-desktop kernel: [ 9633.177594] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9635.766714] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9635.766934] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9636.063738] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9636.063965] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9652.950569] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9652.950777] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9653.192869] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9653.193070] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9655.782836] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9655.783033] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9656.079299] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9656.079492] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:15:46 nvidia-desktop EdgeApp-local[4174]: Creating pipes for GWakeup: Too many open files
Feb 10 16:19:10 nvidia-desktop EdgeApp-local[9795]: Creating pipes for GWakeup: Too many open files

Hi,
Please share us a test sample so that we can run and replicate the issue. The code you shared in previous comment is not complete and we are not able to compile it and run.

testCap.zip (1.5 KB)

Hi @DaneLLL ,

I have attached my sample, and I got memory increased as follows. I tested it in AGX, and nv release is “# R32 (release), REVISION: 6.1, GCID: 27863751, BOARD: t186ref, EABI: aarch64, DATE: Mon Jul 26 19:36:31 UTC 2021”.

Sincerely expect you could fix the memory increasing problem when gst-launch release by cv::VideoCapture::release.

.5C PMIC@50C AUX@32.5C CPU@36C thermal@33.25C
RAM 5420/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [48%@1190,51%@1190,48%@1190,44%@1190,52%@1190,43%@1190] EMC_FREQ 0% GR3D_FREQ 0% AO@32.5C GPU@32.5C PMIC@50C AUX@33C CPU@35.5C thermal@33.9C
RAM 5431/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [54%@1420,43%@1420,37%@1420,47%@1420,43%@1420,41%@1420] EMC_FREQ 0% GR3D_FREQ 0% AO@32.5C GPU@33C PMIC@50C AUX@33C CPU@36.5C thermal@33.9C


RAM 5609/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [32%@1420,40%@1420,35%@1420,37%@1420,35%@1420,36%@1420] EMC_FREQ 0% GR3D_FREQ 22% AO@33C GPU@33.5C PMIC@50C AUX@33C CPU@36.5C thermal@34.55C
RAM 5610/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [31%@1420,41%@1420,30%@1420,27%@1420,38%@1420,39%@1420] EMC_FREQ 0% GR3D_FREQ 44% AO@33.5C GPU@33.5C PMIC@50C AUX@33.5C CPU@36.5C thermal@34.55C
RAM 5612/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [44%@1420,53%@1420,39%@1420,43%@1420,41%@1420,41%@1420] EMC_FREQ 0% GR3D_FREQ 45% AO@33.5C GPU@33.5C PMIC@50C AUX@33.5C CPU@36C thermal@34.55C
RAM 5613/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [46%@1190,46%@1190,42%@1190,42%@1190,44%@1190,33%@1190] EMC_FREQ 0% GR3D_FREQ 0% AO@33C GPU@33.5C PMIC@50C AUX@33C CPU@36C thermal@34.25C

Hi, do you have some questions to compile the sample?

Hi,
Thanks for the sample. We would need some time to set up and test. Will update when there is further progress.

Hi, @DaneLLL , we have a project now, and it need this function to reduce the cpu occupancy of pulling rtsp network camera stream. It is urgently for us, and could you fix the problem more fast?
And we’ll be gratefully!

Hi,
We have tired but do not observe significant memory increase. How long would it take to observe the issue in your test? And it looks like OpenCV does not send EoS in cvCap.release(). Have seen some discussion in
writing file from Gstreamer pipleine in a VideoCapture on the Tx2 - OpenCV Q&A Forum

We would suggest run like this python sample:
Nvv4l2decoder sometimes fails to negotiate with downstream after several pipeline re-launches - #16 by DaneLLL
Sens EoS and wait for it before switching to NULL state. This seems impossible while running gstreamer pipeline in cv2.VideoCapture().

Hi,
There are two known issues in r32.6.1. Please apply the two steps:

  1. Apply the patch and rebuild/replace libgstnvvideo4linux2.so:
    Syncpts and threads leak using gstreamer plugin nvv4l2decoder - #11 by DaneLLL
  2. Rename libv4l2_nvargus.so as suggested in
    Memory Leak (Alloc/free mismatch) in Tegra multimedia API (encoder) - #6 by DaneLLL

Hi @DaneLLL ,
I tried again, and I still got memory increased about 80M every reopen 16 camera streams. Maybe you cound check if all the gst plugins were closed when reopen the pipeline. In fact, there existed memory leaks when reopen pipoline.

How long did you run the sample ? I just run about 20 mins.

Hi,
We have tried some cases and check RAM usage in sudo tegrastats. Tried 4 decoding threads and do not observe memory increase:

int main()
{
    std::vector<pthread_t> pthreadIdArr;
-    for (int k = 0; k < 16; k++) {
+    for (int k = 0; k < 4; k++) {

For 8 encoding threads we can see the increase. And it also happens when using software decoder like:

 std::string pipeName = "rtspsrc location=rtsp://10.19.107.227:8554/test ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! appsink ";

From the tests it looks like if the multiple decoding thread number exceeds a certain value, will see memory increase. It is not specific to using nvv4l2decoder. Not sure what the threshold value is, but in our tests, 4 threads are good and can pass overnight test.

Hi @DaneLLL ,

I tested rtsp stream by gst-launch, and got following debug information. It printed " /GstPipeline:pipeline0/GstXImageSink:ximagesink0: A lot of buffers are being dropped". Could you check if there existed some gst plugins or some buffers unreleased ?

0:00:00.072562774 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspsrc”
0:00:00.077268423 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspreal”
0:00:00.078568771 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtspwms”
0:00:00.080856827 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtph264depay”
0:00:00.082398678 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “h264parse”
0:00:00.145178700 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvv4l2decoder”
0:00:00.148040930 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “nvvidconv”
0:00:00.150499482 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “videoconvert”
0:00:00.153325585 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “ximagesink”
0:00:00.154023151 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “pipeline”
0:00:00.157126821 26452 0x558a855410 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “capsfilter”
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.8.103/test1
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
0:00:00.390685682 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:359:gst_element_factory_create: creating element “rtpbin” named “manager”
0:00:00.392078478 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpsession”
0:00:00.393338922 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpssrcdemux”
0:00:00.393740713 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpstorage”
0:00:00.394015720 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
0:00:00.394167591 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “funnel”
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
0:00:00.401354288 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpjitterbuffer”
0:00:00.402383405 26452 0x558a8f30a0 INFO GST_ELEMENT_FACTORY gstelementfactory.c:361:gst_element_factory_create: creating element “rtpptdemux”
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
WARNING: from element /GstPipeline:pipeline0/GstXImageSink:ximagesink0: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstXImageSink:ximagesink0:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstXImageSink:ximagesink0: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2902): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstXImageSink:ximagesink0:
There may be a timestamping problem, or this computer is too slow.

Hi,
Please set sync=0 to xvimagesink. For optimal perfomance in rendering, please use nv3dsink or nveglglessink

Hi @DaneLLL ,

Using gst-launch for rtsp, I just told you about some debug information, but it was necessary to set sync=true to reduce CPU consumption.

And it was really too high for CPU occupancy to pull 16 network camera streams !

Hi,
It is expected to have certain CPU usage whiling using OpenCV. Please check discussion in
[Gstreamer] nvvidconv, BGR as INPUT

Since BGR is not supported by hardware converter, need CPU to convert frame data to the format. For 16 cameras, the CPU usage is more significant.

Hi @DaneLLL ,

Do you have any idea about memory leak of 8 decoding threads ?

My app printed following debug information, and can you please tell us what did it do or why printed the info.

Mar 14 16:37:23 nvidia-desktop /usr/lib/gdm3/gdm-x-session[7944]: (–) NVIDIA(GPU-0): AOC 2270W (DFP-0): connected
Mar 14 16:37:23 nvidia-desktop /usr/lib/gdm3/gdm-x-session[7944]: (–) NVIDIA(GPU-0): AOC 2270W (DFP-0): External TMDS

Hi,
It seems to be an issue in gstreamer or OpenCV frameworks. By default is is gstreamer 1.14.5 and OpenCV 4.1.1. Since the L4T release is verified with the release versions, we would suggest keep the default versions and run less threads, However, if you must run more threads in single process, may consider to upgrade to later versions and give it a try.

For upgrading gstreamer, please refer to
https://docs.nvidia.com/jetson/archives/l4t-archived/l4t-3261/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/accelerated_gstreamer.html#wwpID0E06E0HA

Our default plugins are built with 1.14.5 and you shall need to rebuild the plugins. The source code is in
https://developer.nvidia.com/embedded/l4t/r32_release_v6.1/sources/t186/public_sources.tbz2

Please remember to clean cache after the upgrade:

$ rm .cache/gstreamer-1.0/registry.aarch64.bin

For upgrading OpenCV, may try this script from community contribution:
GitHub - mdegans/nano_build_opencv: Build OpenCV on Nvidia Jetson Nano

Hi @DaneLLL ,

Thanks a lot, and I’ll try it to upgrade.

I have a question, whether or not all the packages in [public_sources.tbz2] need be re-complied ?