Cannot import VPI from inside a container

[natanel]

I am running a ROS container on a jetson nano. I took the container from here:

GitHub - dusty-nv/jetson-containers: Machine Learning Containers for NVIDIA...

I’m trying to use NVIDIA VPI inside the container, and I have two problem I was hoping to get some help with:

1. I’m getting a No module named 'vpi' when trying to import it in Python. I presume it’s because it’s not installed inside the container, so is there a way to import VPI into the container from outside? I tried the run command from here:
   VPI does not work inside of docker container - #5 by AastaLLL
   but it didn’t work.

2. Not sure this is the place to ask this, but if for example I have this (part of) code in C++:

int main(void){
    cv::Mat image;
    vector<Point2f> points;
    VideoCapture cap("video.avi");
    cap >> frame;  //reading frame from video, size = 1936x1216
    cvtColor(cap, image, cv::COLOR_RGB2GRAY);  // to grayscale
    cv::resize(image, image, m_ImageSizeFinal);  //resize to 968x608
    goodFeaturesToTrackMask = createDetectionMaskImage();  //Create mask of type CV_8UC1
    cv::goodFeaturesToTrack(image, points, 500, 0.001, 15, goodFeaturesToTrackMask, 3, true);
}

What would be the correct way to convert it to VPI? This is my attempt:

import vpi
import cv2
import numpy as np
import csv

cap = cv2.VideoCapture('2021_01_25__14_05_01.avi')
backend = vpi.Backend.CUDA

image1, image2 = None, None
with backend:
    writer = csv.writer(f)
    start = time.time():
        while cap.isOpened():
          now = time.time()
          ret, frame = cap.read()
          if ret == True:
              image = vpi.asimage(frame, vpi.Format.BGR8).convert(vpi.Format.U8)
              image = image.rescale((968, 608), interp=vpi.Interp.LINEAR, border=vpi.Border.ZERO)
              corners, scores = image.harriscorners(sensitivity=0.001)    

But it doesn’t detect any points. Also the parameters aren’t the same between the functions (no parameter for mask for example), so I’m not sure how to go about this. What am I doing wrong?

Thank you very much for the help.

[AastaLLL]

Hi,

1. Please run the following command to install VPI:

$ sudo apt install nvidia-vpi

2. You can find both C++/python tracker sample below:
https://docs.nvidia.com/vpi/algo_klt_tracker.html
Or in /opt/nvidia/vpi2/samples/06-klt_tracker.

Is the bounding box listed below the mask you want?
https://docs.nvidia.com/vpi/group__VPI__KLTFeatureTracker.html#structVPIKLTTrackedBoundingBox

Thanks.

[natanel]

Hey, thank you for the response.

1. I tried running the command in the container and I get the error:
   E: Unable to locate package nvidia-vpi
   maybe there’s something else I need to install before?

2. Does the KLT Feature tracker do the same thing as OpenCV’s cv::goodFeaturesToTrack function? From what I understand the equivalent in VPI is Harris Corner Detector, but the functions don’t have the same parameters and produce different results (the VPI returns 0 points).

Thanks again for the help.

[AastaLLL]

Hi,

1. Please update the source list and try it again.

Over-the-Air Update

To update to a new minor release

2. KLT tracker is used for tracking objects with the given bounding boxes.
Harris is the detector and below is its parameter:

https://docs.nvidia.com/vpi/group__VPI__HarrisCorners.html#structVPIHarrisCornerDetectorParams

Thanks.