VPI-2.3-samples/15-image_view error: "VPI_ERROR_INTERNAL: Failed to create the NvVic handle"

[jcli]

I hit this error after modifying the 15-image_view sample of VPI-2.3 to accept usb camera input via opencv.

My system:

Changes to CMakeLists.txt:

xxx@ubuntu:~/NVIDIA_VPI-2.3-samples/15-image_view$ diff CMakeLists.txt /opt/nvidia/vpi2/samples/15-image_view/CMakeLists.txt 
35,48c35
< #target_link_libraries(${PROJECT_NAME} vpi opencv_core opencv_imgproc)
< 
< # Link your application with OpenCV libraries
< target_link_libraries(${PROJECT_NAME} PRIVATE vpi ${OpenCV_LIBS})
< 
< # If the package has been found, several variables will
< # be set, you can find the full list with descriptions
< # in the OpenCVConfig.cmake file.
< # Print some message showing some of them
< message(STATUS "OpenCV library status:")
< message(STATUS "    config: ${OpenCV_DIR}")
< message(STATUS "    version: ${OpenCV_VERSION}")
< message(STATUS "    libraries: ${OpenCV_LIBS}")
< message(STATUS "    include path: ${OpenCV_INCLUDE_DIRS}")
---
> target_link_libraries(${PROJECT_NAME} vpi opencv_core opencv_imgproc)
49a37
> target_link_libraries(${PROJECT_NAME} opencv_imgcodecs)

Changes to main.cpp:

xxx@ubuntu:~/NVIDIA_VPI-2.3-samples/15-image_view$ diff main.cpp  /opt/nvidia/vpi2/samples/15-image_view/main.cpp
32,33d31
< #include <opencv2/videoio.hpp>
< #include <opencv2/highgui.hpp>
84,100d81
< 
<     // Initialize camera
<     cv::VideoCapture cap;
<     cap.open(0);
< 
<     if(!cap.isOpened()) {
<         std::cout << "ERROR opening camera..." << std::endl;
<         return -1;
<     }
< 
< 
<     int w      = cap.get(cv::CAP_PROP_FRAME_WIDTH);
<     int h      = cap.get(cv::CAP_PROP_FRAME_HEIGHT);
< 
<     while (cap.read(cvImage)) {
< 
<         /*
114d94
<         */
121,122d100
<         //const int parentWidth = w; 
<         //const int parentHeight = h;
141c119
<         for (int i = 0; i < 2; ++i)
---
>         for (int i = 0; i < 4; ++i)
173,174c151,152
<         //CHECK_STATUS(vpiStreamWaitEvent(streams[2], parentEvent));
<         //CHECK_STATUS(vpiStreamWaitEvent(streams[3], parentEvent));
---
>         CHECK_STATUS(vpiStreamWaitEvent(streams[2], parentEvent));
>         CHECK_STATUS(vpiStreamWaitEvent(streams[3], parentEvent));
184c162
<         //CHECK_STATUS(vpiSubmitImageFlip(streams[2], VPI_BACKEND_CUDA, inViews[2], outViews[2], VPI_FLIP_VERT));
---
>         CHECK_STATUS(vpiSubmitImageFlip(streams[2], VPI_BACKEND_CUDA, inViews[2], outViews[2], VPI_FLIP_VERT));
187c165
<         //CHECK_STATUS(vpiSubmitImageFlip(streams[3], VPI_BACKEND_CUDA, inViews[3], outViews[3], VPI_FLIP_BOTH));
---
>         CHECK_STATUS(vpiSubmitImageFlip(streams[3], VPI_BACKEND_CUDA, inViews[3], outViews[3], VPI_FLIP_BOTH));
192,193c170,171
<         //CHECK_STATUS(vpiStreamSync(streams[2]));
<         //CHECK_STATUS(vpiStreamSync(streams[3]));
---
>         CHECK_STATUS(vpiStreamSync(streams[2]));
>         CHECK_STATUS(vpiStreamSync(streams[3]));
204,208c182,183
<         CHECK_STATUS(vpiImageDataExportOpenCVMat(outData, &cvOut));
<         //imwrite("output_views.png", cvOut);
< 
<         cv::imshow("output", cvOut);
<         cv::waitKey(30);
---
>         vpiImageDataExportOpenCVMat(outData, &cvOut);
>         imwrite("output_views.png", cvOut);
213d187
<     }
223c197
<     for (int i = 0; i < 2; ++i)
---
>     for (int i = 0; i < 4; ++i)
232c206
<     for (int i = 0; i < 2; ++i)
---
>     for (int i = 0; i < 4; ++i)

Error symptom:

xxx@ubuntu:~/NVIDIA_VPI-2.3-samples/15-image_view/build$ ./vpi_sample_15_image_view 
[ WARN:0] global /home/ubuntu/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (1100) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
Gtk-Message: 14:33:30.453: Failed to load module "canberra-gtk-module"
NVMAP_IOC_GET_FD failed: Bad address
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
[WARN ] 2023-08-17 14:33:36 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
NvRmStream: Buffer mapping failed (err=196611)
[WARN ] 2023-08-17 14:33:36 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
NvRmStream: Buffer mapping failed (err=196611) 

(repeat the last three lines...)

[WARN ] 2023-08-17 14:33:40 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
NvRmStream: Buffer mapping failed (err=196611)
[WARN ] 2023-08-17 14:33:40 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
[WARN ] 2023-08-17 14:33:40 PvaError_PVAKMDBuffPinSlotsExceeded
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
NvRmStream: Buffer mapping failed (err=196611)
[WARN ] 2023-08-17 14:33:40 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
VPI_ERROR_OUT_OF_MEMORY: Requested size to allocate PVA memory backend is beyond the maximum value

I’ve masked off all CUDA backend used in this sample main.cpp so only CPU backend is invovled besides the added opencv camera. It is weired to see the complaints of VIC and PVA related memory errors.

This also errors on AGX Orin with Jetpack 5.1.1, BTW. It seems the latest version of VPI 2.3 in Jetpack 5.1.2 I’m using here doesn’t resolve this issue.

[AastaLLL]

Hi,

Could you check if there is any error in dmesg?

$ sudo dmesg

Thanks.

[jcli]

xxx@ubuntu:/work$ sudo dmesg |egrep -i ‘err|warn’
[ 0.000000] CPU features: detected: NVIDIA Carmel CNP erratum
[ 0.222973] CPU features: detected: User Access Override
[ 0.428446] mc-err: mcerr ops are set to t19x
[ 0.928900] kvm [1]: vgic interrupt IRQ9
[ 2.621922] THERMAL EST: end of probe, return err: 0
[ 2.907743] sdhci: Copyright(c) Pierre Ossman
[ 5.665683] irq: IRQ284: trimming hierarchy from :interrupt-controller@3881000
[ 5.721478] tegradc 15210000.display: dc.1 probe not in device tree order, defering
[ 6.123314] tegradc 15220000.display: dc.2 probe not in device tree order, defering
[ 7.282748] r8152 2-4.4:1.0: Direct firmware load for rtl_nic/rtl8153a-3.fw failed with error -2
[ 7.365626] irq: IRQ288: trimming hierarchy from :interrupt-controller@3881000
[ 8.552353] systemd[1]: (This warning is only shown for the first unit using IP firewalling.)
[ 10.043682] tegra-hda 3510000.hda: Override SDO lines to 4
[ 10.977202] irq: IRQ305: trimming hierarchy from :interrupt-controller@3881000

[jcli]

I’ve tried to remove camera as well and fed input from the assets/pedestrian.mp4 video, same symptom.

But I did forget to mention, no matter input is camera (/dev/video0) or mp4 video, the program actually ran for a while before the error messages were print out. I suspect memory leak somewhere internal to VPI.

xxx@ubuntu:/work/NVIDIA_VPI-2.3-samples/15-image_view/build$ ./vpi_sample_15_image_view  ../../assets/pedestrians.mp4 
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
[WARN ] 2023-08-18 10:23:47 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
[WARN ] 2023-08-18 10:23:47 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
[WARN ] 2023-08-18 10:23:48 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
[WARN ] 2023-08-18 10:23:48 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
NVMAP_IOC_GET_FD failed: Bad address
PosixMemMap:74 FD from Handle failed : Bad address
[WARN ] 2023-08-18 10:23:48 VPI_ERROR_INTERNAL: Failed to create the NvVic handle
...

[AastaLLL]

Hi,

Sorry for the late update.
Would you mind attaching the the CMakefile and main.cpp directly?

Thanks.

[jcli]

Here you go. Small changes from NVIDIA_VPI-2.3-samples/15-image_view.
main.cpp (9.1 KB)
CMakeLists.txt (2.3 KB)

[AastaLLL]

Hi,

It looks like you created the VPI resources for every frame but did not destroy them.
These resources can be reused and just need to be created at the initial time.

For example: main.cpp (9.2 KB)

diff --git a/main.cpp b/main.cpp
index e4f7012..33a3cf2 100644
--- a/main.cpp
+++ b/main.cpp
@@ -75,6 +75,7 @@ int main(int argc, char *argv[])
     VPIStream streams[4] = {};

     int retval = 0;
+    bool init  = false;

     // Views on Tegra devices need parent images to have only CPU or CUDA backends enabled.
     int imgFlags = VPI_BACKEND_CPU | VPI_BACKEND_CUDA;
@@ -120,6 +121,7 @@ int main(int argc, char *argv[])
             throw std::runtime_error(std::string("Input image too small, it must be at least 2x2"));
         }

+       if( !init ) {
         // Create the input parent image as a single unsigned 8-bit channel
         CHECK_STATUS(vpiImageCreate(parentWidth, parentHeight, VPI_IMAGE_FORMAT_U8, imgFlags, &inParent));

@@ -149,6 +151,9 @@ int main(int argc, char *argv[])
             CHECK_STATUS(vpiImageCreateView(outParent, &clipBounds, imgFlags, &outViews[i]));
         }

+        init = true;
+        }
+
         // Run one algorithm on each view:
         // Each algorithm runs on a different stream, potentially in parallel, all streams working together
         // consuming the same input to produce a single output, using CPU and CUDA backends

We didn’t meet the VIC error after applying the above change.

Thanks.

[jcli]

Thank you for fixing my stupid mistake of memory re-alloction, :)