[Problem] I cannot create HAAR-based classifier for cv::cuda::CascadeClassifier.

I have a classifier problem.
Could you please help me ?

[Problem]
I cannot create HAAR-based classifier for cv::cuda::CascadeClassifier.
Accoring to the manual, the classifier should be the old HAAR-based one.
But I cannot find the way to create the old one.

[Background]
I want to let nVIDIA’s Jetson TX1 GPU detect whatever I want by using original contents.
So, I create HAAR-based classifier and LBP-based one by using opencv_createsample and opencv_traincascade.

As a result, cv::CascadeClassifier seems to work by using HAAR-based and LBP-based.
But cv::cuda::CascadeClassifier which had been optimized for GPU, works only by using LBP-based one.
When using HAAR-based one, it reports the following error messages.

According to the reference manual(OpencCV 3.2), cv::cuda::CascadeClassifier::create() supports only the old haar classifier(trained by the haar training application) and
NVIDIA’s nvbin.
So, I need to know how to use the haar training application or how to use NVIDIA’s nvbin.
But I cannot find the usage of them.

http://docs.opencv.org/trunk/d9/d80/classcv_1_1cuda_1_1CascadeClassifier.html#a2cc2a7ffa981a55ef1aa33cbf8583f22

[Result by using My HAAR ]

ubuntu@tegra-ubuntu:~/hehe/hoge/DetectVehicle/build$ ./detectVehicle HAAR gpu
Built with OpenCV 3.2.0
Try to create CascadeClassifier instance with file(/home/ubuntu/Pictures/makotoqnb/cascade/HAAR/cascade.xml).
OpenCV Error: Unspecified error (The node does not represent a user object (unknown type?)) in cvRead, file /home/ubuntu/hoge/tools/opencv-3.2.0/modules/core/src/persistence.cpp, line 6628
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/ubuntu/hoge/tools/opencv-3.2.0/modules/core/src/persistence.cpp:6628: error: (-2) The node does not represent a user object (unknown type?) in function cvRead

Aborted
ubuntu@tegra-ubuntu:~/hehe/hoge/DetectVehicle/build$

[Result by using HAAR which OpenCV 3.2 provided ]

ubuntu@tegra-ubuntu:~/hehe/hoge/DetectVehicle/build$ ./detectVehicle OLDHAAR gpu
Built with OpenCV 3.2.0
Try to create CascadeClassifier instance with file(/home/ubuntu/hoge/tools/opencv-3.2.0/data/haarcascades_cuda/haarcascade_eye.xml).
Try to open video file(/home/ubuntu/Videos/sample01.mp4).
Capture is opened
ubuntu@tegra-ubuntu:~/hehe/hoge/DetectVehicle/build$

[Condition]
Target board : nVIDIA JETSON TX1
JetPack : 2.3 for L4T
CUDA : 8.0
Target OS : Ubuntu 16.04 LTS(L4T)
OpenCV : ver 3.2 Local build(WITH_CUDA=ON)

[My test code (detectVehicle.cpp)]

#include <iostream>
#include <vector>

#include <opencv2/core.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/opencv.hpp>
#include <opencv2/opencv_modules.hpp>
#include <opencv2/cudaobjdetect.hpp>
#include <opencv2/cudaimgproc.hpp>

using namespace cv;
using namespace std;

#define __DEBPRINT__    (false)
#define __SHOWFRAMENO__ (false)
#define __SHOWRECTSIZE__ (false)
#define STRLEN_FOR_INFORMATION (1024)

/* Global variables */
String vehicle_cascade_OpenCV_ORG_OLDHAAR = "/home/ubuntu/hoge/tools/opencv-3.2.0/data/haarcascades_cuda/haarcascade_eye.xml";
String vehicle_cascade_HAAR = "/home/ubuntu/Pictures/makotoqnb/cascade/HAAR/cascade.xml";
String vehicle_cascade_LBP = "/home/ubuntu/Pictures/makotoqnb/cascade/LBP/cascade.xml";
String vehicle_cascade_name1 = vehicle_cascade_HAAR;

String window_name = "Capture - vehicle detection";

void drawText(Mat & frame)
{
    putText(frame, "/nHello OpenCV/n",
            Point(20, 50),
            FONT_HERSHEY_COMPLEX, 1, // font face and scale
            Scalar(255, 255, 255), // white
            1, LINE_AA); // line thickness and type
}

int main(
    int argc,
    char* argv[]
)
{
    //------------------------
    // Should we run with GPU mode ?
    // Should we use HAAR/LBP/OLDHAAR trained data ?
    //------------------------
    bool isGPU = false;
    for (int i = 1; i < argc; i++)
    {
        if (strcmp(argv[i], "gpu") == 0)
        {
             isGPU = true;
        }
        else if (strcmp(argv[i], "cpu") == 0)
        {
             isGPU = false;
        }
        else if (strcmp(argv[i], "HAAR") == 0)
        {
            vehicle_cascade_name1 = vehicle_cascade_HAAR;
        }
        else if (strcmp(argv[i], "LBP") == 0)
        {
            vehicle_cascade_name1 = vehicle_cascade_LBP;
        }
        else if (strcmp(argv[i], "OLDHAAR") == 0)
        {
            vehicle_cascade_name1 = vehicle_cascade_OpenCV_ORG_OLDHAAR;
        }
    }

    //------------------------
    // Show OpenCV version.
    //------------------------
    cout << "Built with OpenCV " << CV_VERSION << endl;

//------------------------
    // Create cascade object and load the cascades for vehicle detection.
    //------------------------
    cout << "Try to create CascadeClassifier instance with file(" << vehicle_cascade_name1 << ")." << endl;
    Ptr<CascadeClassifier> h_vehicle_cascade;
    Ptr<cuda::CascadeClassifier> d_vehicle_cascade;
    if (isGPU)
    {
        d_vehicle_cascade = cv::cuda::CascadeClassifier::create(vehicle_cascade_name1);
        if (NULL == d_vehicle_cascade)
        {
            cout << "Error creating vehicle cascade." << endl;
            return -1;
        }
    }
    else
    {
        h_vehicle_cascade = new CascadeClassifier();
        if (NULL == h_vehicle_cascade)
        {
            cout << "Error creating vehicle cascade." << endl;
            return -1;
        }

        bool bResult = h_vehicle_cascade->load(vehicle_cascade_name1);
        if (!bResult)
        {
            cout << "Error loading vehicle cascade." << endl;
            return -1;
        }
    }

//------------------------
    // Open our target video.
    //------------------------
    Mat h_colorframe;
     
    String videoFilename = "/home/ubuntu/Videos/sample01.mp4";
    cout << "Try to open video file(" << videoFilename << ")." << endl;
    VideoCapture capture;
    capture.open(videoFilename);
    if(capture.isOpened())
    {
        cout << "Capture is opened" << endl;
         
        int frameNum = capture.get(CAP_PROP_FRAME_COUNT);
        int currentFrame;
        //String dispCurrentFrame = new String(STRLEN_FOR_INFORMATION, ); 
        char* dispDetectedRect = (char*)malloc(STRLEN_FOR_INFORMATION * sizeof(char));
        char* dispCurrentFrame = (char*)malloc(STRLEN_FOR_INFORMATION * sizeof(char));
        if (NULL == dispCurrentFrame)
        {
            cout << "Out of heap!!" << endl;
        }

//------------------------
        // Setup window
        //------------------------
        namedWindow(window_name, WINDOW_AUTOSIZE);
        setWindowProperty(window_name, WINDOW_GUI_EXPANDED, WINDOW_AUTOSIZE);
        int initialPos = 0;

//------------------------
        // data for CPU
        //------------------------
        Mat h_colorframe;
        Mat h_grayframe;
        std::vector<Rect> h_vehicles;

//------------------------
        // data for GPU
        //------------------------
        cuda::GpuMat d_colorframe;
        cuda::GpuMat d_grayframe;
        cuda::GpuMat d_vehicles;

//------------------------
        // Continue detecting while frame existing.
        //------------------------
        for(;;)
        {
            capture >> h_colorframe;

            if(h_colorframe.empty())
            {
                break;
            }

//------------------------
            // Convert frame to grayscale and equalizing by Histogram.
            //------------------------
            if (isGPU)
            {
                d_colorframe.upload(h_colorframe); 
                if (false)
                {
                    cuda::cvtColor(d_colorframe, d_grayframe, CV_RGB2GRAY);
                }
                else
                {
                    cuda::cvtColor(d_colorframe, d_grayframe, CV_BGR2GRAY);
                }
                cuda::equalizeHist(d_grayframe, d_grayframe);

            }
            else
            {
                if (false)
                {
                    cvtColor(h_colorframe, h_grayframe, CV_RGB2GRAY);
                }
                else
                {
                    cvtColor(h_colorframe, h_grayframe, CV_BGR2GRAY);
                }
                equalizeHist(h_grayframe, h_grayframe);
            }
#if __DEBPRINT__
            cout << "Finished converting frame to grayscale and equalizing." << endl;
#endif

            //------------------------
            // Detect vehicles.
            //------------------------
            try
            {
                if (isGPU)
                {
                    d_vehicle_cascade->detectMultiScale(d_grayframe, d_vehicles);
                    d_vehicle_cascade->convert(d_vehicles, h_vehicles);
                }
                else
                {

                    h_vehicle_cascade->detectMultiScale(h_grayframe, h_vehicles);
                    h_vehicle_cascade->detectMultiScale(h_grayframe, h_vehicles, 1.1, 3, 0, Size(30, 30), Size(500, 500));
                }

#if __DEBPRINT__
                cout << "Finished detectMultiScale" << endl;
#endif

                // Draw detected rectangle.             
                for (size_t i = 0; i < h_vehicles.size(); i++)
                {
#if __SHOWRECTSIZE__
                    sprintf(dispDetectedRect, "%4d, %4d", h_vehicles[i].width, h_vehicles[i].height);
                    putText(h_colorframe, dispDetectedRect,
                            Point(h_vehicles[i].x, h_vehicles[i].y - 10),
                            FONT_ITALIC , 0.5, // font face and scale
                            Scalar(255, 0, 255), // white
                            1, LINE_AA); // line thickness and type
#endif
                    rectangle(h_colorframe, h_vehicles[i], Scalar( 255, 0, 255 ), 4, 8, 0 );
                }

//------------------------
                // Show results.
                //------------------------
                Mat result = h_colorframe;
#if __SHOWFRAMENO__
                currentFrame = capture.get(CV_CAP_PROP_POS_FRAMES);
                sprintf(dispCurrentFrame, "Frame No. : %08d", currentFrame);
                if (isGPU)
                {
                    strcat(dispCurrentFrame, "  Mode : GPU");
                }
                else
                {
                    strcat(dispCurrentFrame, "  Mode : CPU");
                }
                putText(result, dispCurrentFrame,
                        Point(20, 460),
                        FONT_ITALIC , 1, // font face and scale
                        Scalar(255, 255, 255), // white
                        1, LINE_AA); // line thickness and type

#endif
                imshow(window_name, result);
                int keyCode = waitKey(1);
                if (keyCode != -1)
                {
                    if ((char)keyCode == 'p')
                    {
                        putText(result, "PAUSED",
                                Point(200, 300),
                                FONT_ITALIC , 2, // font face and scale
                                Scalar(0, 0, 255), // white
                                3, LINE_AA); // line thickness and type
                        imshow(window_name, result);
                        waitKey(0);
                    }
                    else if ((char)keyCode == 'q') 
                    {
                        break;
                    }
                    else
                    {
                        char dispKeyCode[64];
                        sprintf(dispKeyCode, "Key Code : %04d", keyCode);
                        putText(result, dispKeyCode,
                                Point(50, 50),
                                FONT_ITALIC , 1, // font face and scale
                                Scalar(0, 0, 255), // white
                                3, LINE_AA); // line thickness and type
                        imshow(window_name, result);
                    }
                }

            }
            catch (Exception& e)
            {
                cout << e.msg << endl;
            }

#if __DEBPRINT__
            cout << "stage 9\n";
#endif
        }
    }
    else
    {
        cout << "No capture" << endl;
        h_colorframe = Mat::zeros(480, 640, CV_8UC1);
        drawText(h_colorframe);
        imshow("Sample", h_colorframe);
        waitKey(0);
    }

    return 0;
}

[My trained data]

cascade.xml.zip (8.05 KB)

Hi,

We are investigating this issue now. Will update to you later.

Hi,

I have tested your code with the model file found in opencv github:

It works well and I use haarcascade_frontalface_alt.xml.
Could you modify your xml into the supported format by referring to the above files?

Hi AstaLLL,

Thank you for your having investigated this issue.
I will try to check your solution soon.

After that, I will update.

BRs,
makotoqnb

Hi AstaLLL,
Does the "gpu::CascadeClassifier_GPU::load

Loads the classifier from a file. The previous content is destroyed.

C++: bool gpu::CascadeClassifier_GPU::load(const string& filename)
Parameters:

    filename – Name of the file from which the classifier is loaded. Only the old haar classifiers (trained by the haar training application) and NVIDIA’s nvbin are supported for HAAR and only new type of OpenCV XML cascade supported for LBP. The working haar classifiers can be found under data\haarcascades_GPU\." have the same problem?  Does it mean that the TX1 GPU do not support the format of the train data makotoqnb gave?

gpu::CascadeClassifier_GPU::load supports two format for harr

  1. nvbin
  2. Old haar classifier

Please make sure your format is supported.
Old haar classifier can be found in:

Hi AstaLLL,
thanks,and i got it