Nvidia-desktop kernel: [407343.357549] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600

I pulled rtsp network camera stream by cv::VideoCapture using following url,

std::string url = rtspsrc location=rtsp:** latency=0 ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! appsink drop=true sync=false

I offen reopen the url by cv::VideoCapture::open, but my app crashed with the following debug information:

Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: Opening in BLOCKING MODE
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: [ WARN:337768] global /opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1063) open OpenCV | GStreamer warning: unable to query duration of stream
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: [ WARN:337768] global /opencv-4.5.4/modules/videoio/src/cap_gstreamer.cpp (1100) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=8, duration=-1
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: NvMMLiteBlockCreate : Block : BlockType = 261
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: SYNC_IOC_FENCE_INFO ioctl failed with 9
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: NvDdkVicExecute Failed
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: nvbuffer_transform Failed
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: gst_nvvconv_transform: NvBufferTransform Failed
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: NvDdkVicExecute Failed
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: nvbuffer_transform Failed
Jan 19 07:00:36 nvidia-desktop bootapp.sh[23107]: gst_nvvconv_transform: NvBufferTransform Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: NvDdkVicExecute Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: nvbuffer_transform Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: gst_nvvconv_transform: NvBufferTransform Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: (EdgeApp-local:23168): GStreamer-CRITICAL **: 07:00:37.049: gst_poll_write_control: assertion ‘set != NULL’ failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: message repeated 3 times: [ (EdgeApp-local:23168): GStreamer-CRITICAL **: 07:00:37.049: gst_poll_write_control: assertion ‘set != NULL’ failed]
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: (EdgeApp-local:23168): GStreamer-CRITICAL **: 07:00:37.068: gst_poll_read_control: assertion ‘set != NULL’ failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: NvDdkVicExecute Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: nvbuffer_transform Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: gst_nvvconv_transform: NvBufferTransform Failed
Jan 19 07:00:37 nvidia-desktop EdgeApp-local[23168]: gst_poll_read_control: assertion ‘set != NULL’ failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: NvDdkVicExecute Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: nvbuffer_transform Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: gst_nvvconv_transform: NvBufferTransform Failed
Jan 19 07:00:37 nvidia-desktop EdgeApp-local[23168]: gst_poll_write_control: assertion ‘set != NULL’ failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: NvDdkVicExecute Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: nvbuffer_transform Failed
Jan 19 07:00:37 nvidia-desktop bootapp.sh[23107]: gst_nvvconv_transform: NvBufferTransform Failed
Jan 19 07:00:51 nvidia-desktop EdgeApp-local[23168]: gst_poll_get_read_gpollfd: assertion ‘set != NULL’ failed
Jan 19 07:00:51 nvidia-desktop bootapp.sh[23107]: libv4l2: error getting capabilities: Inappropriate ioctl for device
Jan 19 07:00:51 nvidia-desktop kernel: [407343.357549] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Jan 19 07:00:51 nvidia-desktop kernel: [407343.357764] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Jan 19 07:00:56 nvidia-desktop EdgeApp-local[23168]: gst_poll_get_read_gpollfd: assertion ‘set != NULL’ failed
Jan 19 07:00:56 nvidia-desktop bootapp.sh[23107]: libv4l2: error getting capabilities: Inappropriate ioctl for device
Jan 19 07:00:56 nvidia-desktop kernel: [407348.375410] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Jan 19 07:00:56 nvidia-desktop kernel: [407348.375598] (NULL device ): nvhost_channelctl: invalid cmd 0x80685600
Jan 19 07:01:01 nvidia-desktop kernel: [407353.393717] (NULL device ): nvhost_channelctl: invalid cmd 0x80685600
Jan 19 07:01:01 nvidia-desktop kernel: [407353.393917] (NULL device ): nvhost_channelctl: invalid cmd 0x80685600
Jan 19 07:01:02 nvidia-desktop bootapp.sh[23107]: “2022-01-19T07:01:02” “ReadFrame_15” ERROR: "Failed to open camera ! ex = rtsp://
"
Jan 19 07:01:02 nvidia-desktop EdgeApp-local[23168]: Creating pipes for GWakeup: Too many open files
Jan 19 07:01:03 nvidia-desktop bootapp.sh[23107]: /home/nvidia/EdgeApp/bootapp.sh: line 9: 23168 Trace/breakpoint trap (core dumped) ./EdgeApp-local -d
Jan 19 07:01:03 nvidia-desktop systemd[1]: App.service: Main process exited, code=exited, status=133/n/a
Jan 19 07:01:03 nvidia-desktop systemd[1]: App.service: Failed with result ‘exit-code’.

Hi,
We have to reproduce the issue and check. Would need your help to share a test sample so that we can set up and test on Xavier.

We created 16 threads to start reopen rtsp stream, as follows.


void* pthread_Frame(void *arg)
{
    cv::VideoCapture *cvCap = new cv::VideoCapture;
    cv::Mat *frame = nullptr;

    int dropFrameCount = 25;
    bool isCamRestart = false;
    int reopenCount = 0;

    while(true){

        if(reopenCount++%500 == 0){
            dropFrameCount = dropFrameCount - 2;
            if(dropFrameCount < 1){
                dropFrameCount = 25;
            }
        }

        frame = new cv::Mat;
        if(!isCamRestart && cvCap->read(*frame)){
            std::cout<<"Read Done."<<std::endl;
        } else {
            isCamRestart = false;

            if(cvCap->isOpened()){
                cvCap->release();
                sleep(2);
                //uInfo->D(TAG, "end of video ");
                //break;
            }

            // export GST_DEBUG=*FACTORY*:4
            std::string pipeName = "rtspsrc location=rtsp://admin:admin@192.168.1.10:554/h264/ch1/main/av_stream ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 drop-frame-interval=" + std::to_string(dropFrameCount) + " ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! appsink sync=true max-buffers=1 drop=true"; //get newest frame

            cvCap->open(pipeName, cv::CAP_GSTREAMER);
            if(!cvCap->isOpened()){
                sleep(1);

                std::cout<<"Failed to open camera ! ex = " + pipeName<<std::endl;
            } else {
                std::cout<<"Camera is openned."<<std::endl;
            }
        }

        frame->release();
        delete frame;
        frame = nullptr;
    }
}

int main()
{
    std::vector<pthread_t> pthreadIdArr;
    for (int k = 0; k < 16; k++) {
        pthread_t pthreadHandle;
        pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
        pthreadIdArr.push_back(pthreadHandle);
    }

    // wait all done
    for (size_t i = 0; i < pthreadIdArr.size(); i++) {
        pthread_join(pthreadIdArr[i], nullptr);
    }
}

The example code is here above @DaneLLL .

Hi,
Could you also share build command? So that we can save some time in working out the command. Thanks.

I think it is OK to compile it by gcc directly as one cpp file.

@DaneLLL Generally, the code above has one abnormal situation that program would be blocked in open function in some thread.

Could you parse and give one conclusion?

Hi,
We would need some time to set up and test. Will update once there’s further finding

Hi,
Could you share the sample with header files? We have seen errors while trying to compile it:

test.cpp: In function ‘void* pthread_Frame(void*)’:
test.cpp:3:5: error: ‘cv’ has not been declared
     cv::VideoCapture *cvCap = new cv::VideoCapture;
     ^~
test.cpp:3:23: error: ‘cvCap’ was not declared in this scope
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                       ^~~~~
test.cpp:3:35: error: ‘cv’ does not name a type
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                                   ^~
test.cpp:4:5: error: ‘cv’ has not been declared
     cv::Mat *frame = nullptr;
     ^~
test.cpp:4:14: error: ‘frame’ was not declared in this scope
     cv::Mat *frame = nullptr;
              ^~~~~
test.cpp:19:21: error: ‘cv’ does not name a type
         frame = new cv::Mat;
                     ^~
test.cpp:21:18: error: ‘cout’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                  ^~~~
test.cpp:21:43: error: ‘endl’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                                           ^~~~
test.cpp:27:17: error: ‘sleep’ was not declared in this scope
                 sleep(2);
                 ^~~~~
test.cpp:33:18: error: ‘string’ is not a member of ‘std’
             std::string pipeName = "rtspsrc location=rtsp://admin:admin@192.168
.1.10:554/h264/ch1/main/av_stream ! rtph264depay ! h264parse ! nvv4l2decoder ena
ble-max-performance=1 drop-frame-interval=" + std::to_string(dropFrameCount) + "
 ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! a
ppsink sync=true max-buffers=1 drop=true"; //get newest frame
                  ^~~~~~
test.cpp:35:25: error: ‘pipeName’ was not declared in this scope
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
test.cpp:35:25: note: suggested alternative: ‘typename’
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
                         typename
test.cpp:35:35: error: ‘cv’ has not been declared
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                                   ^~
test.cpp:37:17: error: ‘sleep’ was not declared in this scope
                 sleep(1);
                 ^~~~~
test.cpp:39:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Failed to open camera ! ex = " + pipeName<<std::end
l;
                      ^~~~
test.cpp:39:77: error: ‘endl’ is not a member of ‘std’
              std::cout<<"Failed to open camera ! ex = " + pipeName<<std::endl;
                                                                          ^~~~
test.cpp:41:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                      ^~~~
test.cpp:41:55: error: ‘endl’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                                                       ^~~~
test.cpp:46:16: error: type ‘<type error>’ argument given to ‘delete’, expected
pointer
         delete frame;
                ^~~~~
test.cpp: In function ‘int main()’:
test.cpp:53:10: error: ‘vector’ is not a member of ‘std’
     std::vector<pthread_t> pthreadIdArr;
          ^~~~~~
test.cpp:53:17: error: ‘pthread_t’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
test.cpp:53:17: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
                 pthread_Frame
test.cpp:53:28: error: ‘pthreadIdArr’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
test.cpp:53:28: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
                            pthread_Frame
test.cpp:55:19: error: expected ‘;’ before ‘pthreadHandle’
         pthread_t pthreadHandle;
                   ^~~~~~~~~~~~~
test.cpp:56:25: error: ‘pthreadHandle’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
test.cpp:56:25: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
                         pthread_Frame
test.cpp:56:9: error: ‘pthread_create’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
test.cpp:56:9: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
         pthread_Frame
test.cpp:61:10: error: ‘size_t’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
          ^~~~~~
test.cpp:61:24: error: ‘i’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
                        ^
test.cpp:62:9: error: ‘pthread_join’ was not declared in this scope
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
test.cpp:62:9: note: suggested alternative: ‘pthread_Frame’
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
         pthread_Frame
nvidia@dhcp-10-19-107-227:~$ g++ -o testcv -Wall -std=c++11 test.cpp -I/usr/loca
l/include/opencv4/ -L/usr/local/lib/ -lopencv_core -lopencv_highgui -lopencv_img
proc -lopencv_videoio
test.cpp: In function ‘void* pthread_Frame(void*)’:
test.cpp:3:5: error: ‘cv’ has not been declared
     cv::VideoCapture *cvCap = new cv::VideoCapture;
     ^~
test.cpp:3:23: error: ‘cvCap’ was not declared in this scope
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                       ^~~~~
test.cpp:3:35: error: ‘cv’ does not name a type
     cv::VideoCapture *cvCap = new cv::VideoCapture;
                                   ^~
test.cpp:4:5: error: ‘cv’ has not been declared
     cv::Mat *frame = nullptr;
     ^~
test.cpp:4:14: error: ‘frame’ was not declared in this scope
     cv::Mat *frame = nullptr;
              ^~~~~
test.cpp:19:21: error: ‘cv’ does not name a type
         frame = new cv::Mat;
                     ^~
test.cpp:21:18: error: ‘cout’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                  ^~~~
test.cpp:21:43: error: ‘endl’ is not a member of ‘std’
             std::cout<<"Read Done."<<std::endl;
                                           ^~~~
test.cpp:27:17: error: ‘sleep’ was not declared in this scope
                 sleep(2);
                 ^~~~~
test.cpp:33:18: error: ‘string’ is not a member of ‘std’
             std::string pipeName = "rtspsrc location=rtsp://admin:admin@192.168
.1.10:554/h264/ch1/main/av_stream ! rtph264depay ! h264parse ! nvv4l2decoder ena
ble-max-performance=1 drop-frame-interval=" + std::to_string(dropFrameCount) + "
 ! nvvidconv ! video/x-raw,width=1920,height=1080,format=BGRx ! videoconvert ! a
ppsink sync=true max-buffers=1 drop=true"; //get newest frame
                  ^~~~~~
test.cpp:35:25: error: ‘pipeName’ was not declared in this scope
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
test.cpp:35:25: note: suggested alternative: ‘typename’
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                         ^~~~~~~~
                         typename
test.cpp:35:35: error: ‘cv’ has not been declared
             cvCap->open(pipeName, cv::CAP_GSTREAMER);
                                   ^~
test.cpp:37:17: error: ‘sleep’ was not declared in this scope
                 sleep(1);
                 ^~~~~
test.cpp:39:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Failed to open camera ! ex = " + pipeName<<std::end
l;
                      ^~~~
test.cpp:39:77: error: ‘endl’ is not a member of ‘std’
              std::cout<<"Failed to open camera ! ex = " + pipeName<<std::endl;
                                                                          ^~~~
test.cpp:41:22: error: ‘cout’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                      ^~~~
test.cpp:41:55: error: ‘endl’ is not a member of ‘std’
                 std::cout<<"Camera is openned."<<std::endl;
                                                       ^~~~
test.cpp:46:16: error: type ‘<type error>’ argument given to ‘delete’, expected
pointer
         delete frame;
                ^~~~~
test.cpp: In function ‘int main()’:
test.cpp:53:10: error: ‘vector’ is not a member of ‘std’
     std::vector<pthread_t> pthreadIdArr;
          ^~~~~~
test.cpp:53:17: error: ‘pthread_t’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
test.cpp:53:17: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                 ^~~~~~~~~
                 pthread_Frame
test.cpp:53:28: error: ‘pthreadIdArr’ was not declared in this scope
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
test.cpp:53:28: note: suggested alternative: ‘pthread_Frame’
     std::vector<pthread_t> pthreadIdArr;
                            ^~~~~~~~~~~~
                            pthread_Frame
test.cpp:55:19: error: expected ‘;’ before ‘pthreadHandle’
         pthread_t pthreadHandle;
                   ^~~~~~~~~~~~~
test.cpp:56:25: error: ‘pthreadHandle’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
test.cpp:56:25: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
                         ^~~~~~~~~~~~~
                         pthread_Frame
test.cpp:56:9: error: ‘pthread_create’ was not declared in this scope
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
test.cpp:56:9: note: suggested alternative: ‘pthread_Frame’
         pthread_create(&pthreadHandle, nullptr, pthread_Frame, nullptr);
         ^~~~~~~~~~~~~~
         pthread_Frame
test.cpp:61:10: error: ‘size_t’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
          ^~~~~~
test.cpp:61:24: error: ‘i’ was not declared in this scope
     for (size_t i = 0; i < pthreadIdArr.size(); i++) {
                        ^
test.cpp:62:9: error: ‘pthread_join’ was not declared in this scope
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
test.cpp:62:9: note: suggested alternative: ‘pthread_Frame’
         pthread_join(pthreadIdArr[i], nullptr);
         ^~~~~~~~~~~~
         pthread_Frame

Would be great if you can share the sample which can be built and run directly.

Do you have one conclusion? But I still offten get error message, “Creating pipes for GWakeup: Too many open files”. @DaneLLL

Hi,
We are not able to compile the test sample and then set up the test. Would need your help to share a complete sample so that we can set up and run.

I have found that the number of command “lsof” get more and more as FD leak, when do cvCap->open rtsp stream and cvCap->release().

Then it printed following infos,

Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877093] falcon 15340000.vic: nvhost_ioctl_channel_submit: failed with err -24
Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877359] falcon 15340000.vic: submit_add_gathers: failed to copy user inputs: class_ids=00000000e40d1b88 num_cmdbufs=2
Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877559] falcon 15340000.vic: nvhost_ioctl_channel_submit: failed with err -22
Feb 10 15:23:13 nvidia-desktop kernel: [ 6702.877790] (NULL device *): nvhost_sync_fence_set_name: failed to get fence
Feb 10 15:29:10 nvidia-desktop EdgeApp-local[25036]: Creating pipes for GWakeup: Too many open files
Feb 10 16:03:18 nvidia-desktop kernel: [ 9107.764728] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:03:18 nvidia-desktop kernel: [ 9107.764937] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:08:42 nvidia-desktop EdgeApp-local[11032]: Creating pipes for GWakeup: Too many open files
Feb 10 16:12:03 nvidia-desktop kernel: [ 9632.936356] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:03 nvidia-desktop kernel: [ 9632.936601] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:03 nvidia-desktop kernel: [ 9633.177397] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:03 nvidia-desktop kernel: [ 9633.177594] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9635.766714] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9635.766934] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9636.063738] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:06 nvidia-desktop kernel: [ 9636.063965] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9652.950569] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9652.950777] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9653.192869] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:23 nvidia-desktop kernel: [ 9653.193070] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9655.782836] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9655.783033] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9656.079299] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:12:26 nvidia-desktop kernel: [ 9656.079492] (NULL device *): nvhost_channelctl: invalid cmd 0x80685600
Feb 10 16:15:46 nvidia-desktop EdgeApp-local[4174]: Creating pipes for GWakeup: Too many open files
Feb 10 16:19:10 nvidia-desktop EdgeApp-local[9795]: Creating pipes for GWakeup: Too many open files

Hi,
Please share us a test sample so that we can run and replicate the issue. The code you shared in previous comment is not complete and we are not able to compile it and run.

testCap.zip (1.5 KB)

Hi @DaneLLL ,

I have attached my sample, and I got memory increased as follows. I tested it in AGX, and nv release is “# R32 (release), REVISION: 6.1, GCID: 27863751, BOARD: t186ref, EABI: aarch64, DATE: Mon Jul 26 19:36:31 UTC 2021”.

Sincerely expect you could fix the memory increasing problem when gst-launch release by cv::VideoCapture::release.

.5C PMIC@50C AUX@32.5C CPU@36C thermal@33.25C
RAM 5420/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [48%@1190,51%@1190,48%@1190,44%@1190,52%@1190,43%@1190] EMC_FREQ 0% GR3D_FREQ 0% AO@32.5C GPU@32.5C PMIC@50C AUX@33C CPU@35.5C thermal@33.9C
RAM 5431/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [54%@1420,43%@1420,37%@1420,47%@1420,43%@1420,41%@1420] EMC_FREQ 0% GR3D_FREQ 0% AO@32.5C GPU@33C PMIC@50C AUX@33C CPU@36.5C thermal@33.9C


RAM 5609/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [32%@1420,40%@1420,35%@1420,37%@1420,35%@1420,36%@1420] EMC_FREQ 0% GR3D_FREQ 22% AO@33C GPU@33.5C PMIC@50C AUX@33C CPU@36.5C thermal@34.55C
RAM 5610/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [31%@1420,41%@1420,30%@1420,27%@1420,38%@1420,39%@1420] EMC_FREQ 0% GR3D_FREQ 44% AO@33.5C GPU@33.5C PMIC@50C AUX@33.5C CPU@36.5C thermal@34.55C
RAM 5612/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [44%@1420,53%@1420,39%@1420,43%@1420,41%@1420,41%@1420] EMC_FREQ 0% GR3D_FREQ 45% AO@33.5C GPU@33.5C PMIC@50C AUX@33.5C CPU@36C thermal@34.55C
RAM 5613/7765MB (lfb 117x4MB) SWAP 2851/3883MB (cached 6MB) CPU [46%@1190,46%@1190,42%@1190,42%@1190,44%@1190,33%@1190] EMC_FREQ 0% GR3D_FREQ 0% AO@33C GPU@33.5C PMIC@50C AUX@33C CPU@36C thermal@34.25C

Hi, do you have some questions to compile the sample?

Hi,
Thanks for the sample. We would need some time to set up and test. Will update when there is further progress.

Hi, @DaneLLL , we have a project now, and it need this function to reduce the cpu occupancy of pulling rtsp network camera stream. It is urgently for us, and could you fix the problem more fast?
And we’ll be gratefully!

Hi,
We have tired but do not observe significant memory increase. How long would it take to observe the issue in your test? And it looks like OpenCV does not send EoS in cvCap.release(). Have seen some discussion in
writing file from Gstreamer pipleine in a VideoCapture on the Tx2 - OpenCV Q&A Forum

We would suggest run like this python sample:
Nvv4l2decoder sometimes fails to negotiate with downstream after several pipeline re-launches - #16 by DaneLLL
Sens EoS and wait for it before switching to NULL state. This seems impossible while running gstreamer pipeline in cv2.VideoCapture().

Hi,
There are two known issues in r32.6.1. Please apply the two steps:

  1. Apply the patch and rebuild/replace libgstnvvideo4linux2.so:
    Syncpts and threads leak using gstreamer plugin nvv4l2decoder - #11 by DaneLLL
  2. Rename libv4l2_nvargus.so as suggested in
    Memory Leak (Alloc/free mismatch) in Tegra multimedia API (encoder) - #6 by DaneLLL

Hi @DaneLLL ,
I tried again, and I still got memory increased about 80M every reopen 16 camera streams. Maybe you cound check if all the gst plugins were closed when reopen the pipeline. In fact, there existed memory leaks when reopen pipoline.

How long did you run the sample ? I just run about 20 mins.