Program frozen after initialization of PathNet....

Autonomous Vehicles DRIVE PX 2 DriveWorks
1

Hi! all,

I try to combine MapNet and PathNet in one program, and I make my program by coping code from demo landmarkdetection and pathperception and then doing some modified. But I found that my programm will be frozened when it finish iniliaze MapNet and PathNet. There were no error, the program is still running, but not go to next step.

Here is some snippt of my code:

int main(int argc, char* argv[]){
  ....
  DetectorApp app;
  app.onInitialize();
  app.onProcess();
  app.onRelease();
  ....
}

//------------------------------------------------------------------------------
    // initialize MapNet detector
    //------------------------------------------------------------------------------
    {
      // Initialize MapNet network
      dwMapNetParams mapNetParams{};
      CHECK_DW_ERROR(dwMapNet_initDefaultParams(&mapNetParams));

      mapNetParams.networkPrecision = DW_PRECISION_FP32;
      mapNetParams.networkModel = DW_MAPNET_MODEL_FRONT_MULTI_CLASS;

      CHECK_DW_ERROR(dwMapNet_initialize(&m_mapNet, &mapNetParams, m_sdk));

      // Initialize LandmarkDetector from MapNet
      dwLandmarkDetectorParams landmarkParams{};
      dwLandmarkDetector_initializeDefaultParams(&landmarkParams);
      CHECK_DW_ERROR(dwLandmarkDetector_initializeFromMapNet(
          &m_landmarkDetector, m_mapNet, landmarkParams, m_imageWidth,
          m_imageHeight, m_sdk));

      CHECK_DW_ERROR(dwLandmarkDetector_setCUDAStream(
          m_cudaStream, DW_LANDMARK_TYPE_LANE_MARKINGS, m_landmarkDetector));

      // Get threshold from command line
      float32_t threshold;
      try {
        spdlog::info("The landmark network threadshold is" +
                     std::to_string(FLAGS_landmark_network_threadshold));
        threshold = FLAGS_landmark_network_threadshold;
      } catch (...) {
        std::cerr << "Given threshold cannot be parsed." << std::endl;
        return false;
      }

      CHECK_DW_ERROR(dwLandmarkDetector_setDetectionThreshold(
          threshold, DW_LANDMARK_TYPE_LANE_MARKINGS, m_landmarkDetector));
      CHECK_DW_ERROR(dwLandmarkDetector_setDetectionThreshold(
          threshold, DW_LANDMARK_TYPE_POLES, m_landmarkDetector));
      spdlog::info("MapNet Initialized!");
    }
    {
      // Initialize PathNet.
      dwPathNetParams pathNetParams{};
      CHECK_DW_ERROR(dwPathNet_initDefaultParams(&pathNetParams));
      CHECK_DW_ERROR(dwPathNet_initialize(&m_pathNet, &pathNetParams, m_sdk));

      // Initialize PathDetector.
      //   dwImageProperties imageProperties = camera->getOutputProperties();
      CHECK_DW_ERROR(dwPathDetector_initializeFromPathNetWithCameraRig(
          &m_pathDetector, m_pathNet, m_imageWidth, m_imageHeight,
          m_calibratedCam, m_transformation, m_cudaStream, m_sdk));

      // Set detection threshold.
      float32_t detectionThreshold = FLAGS_pathnet_threadhold;
      if (detectionThreshold < 0.0f || detectionThreshold > 1.0f) {
        spdlog::warn("[PathDetectorApp]: Wrong detection threshold value. The "
                     "threshold is expected to be between 0 and 1. "
                     "Detection threshold is set to 0.5.");
        detectionThreshold = 0.5f;
      }

      dwPathDetector_setDetectionThreshold(detectionThreshold, m_pathDetector);

      // Set temporal smoothing factor.
      float32_t temporalSmoothingFactor = FLAGS_temporal_smoothing_factor;
      if (temporalSmoothingFactor < 0.0f || temporalSmoothingFactor > 1.0f) {
        spdlog::warn("[PathDetectorApp]: Wrong temporal smoothing factor "
                     "value. The factor is expected to be between 0 and 1. "
                     "Temporal smoothing threshold is set to 0.1.");
        temporalSmoothingFactor = 0.1f;
      }

      dwPathDetector_setTemporalSmoothingFactor(temporalSmoothingFactor,
                                                m_pathDetector);

      // Set look ahead distance.
      float32_t lookAheadDistance = FLAGS_look_ahead_distance;
      dwPathDetector_setMaxLookAheadDistance(lookAheadDistance, m_pathDetector);

      // Set detection ROI.
      int32_t roiX = FLAGS_roi_x;
      int32_t roiY = FLAGS_roi_y;
      int32_t roiWidth = FLAGS_roi_width;
      int32_t roiHeight = FLAGS_roi_height;
      int32_t imageWidth = static_cast<int32_t>(FLAGS_width);
      int32_t imageHeight = static_cast<int32_t>(FLAGS_height);

      if (roiX < 0 || roiX > imageWidth) {
        spdlog::warn("[PathDetectorApp]: Incorrect values for roi.x. It should "
                     "be between 0 and imageWidth."
                     "roi.x is set to 0.");
        roiX = 0;
      }

      if (roiY < 0 || roiY > imageHeight) {
        spdlog::warn("[PathDetectorApp]: Incorrect values for roi.y. It should "
                     "be between 0 and imageHeight."
                     "roi.y is set to 0.");
        roiY = 0;
      }

      if (roiX + roiWidth < 0 || roiX + roiWidth > imageWidth) {
        spdlog::warn("[PathDetectorApp]: Incorrect values for roi.width."
                     "roi.width is set to imageWidth - roi.x.");
        roiWidth = imageWidth - roiX;
      }

      if (roiY + roiHeight < 0 || roiY + roiHeight > imageHeight) {
        spdlog::warn("[PathDetectorApp]: Incorrect values for roi.height."
                     "roi.height is set to imageHeight - roi.y.");
        roiHeight = imageHeight - roiY;
      }

      dwRect detectionROI{roiX, roiY, roiWidth, roiHeight};
      CHECK_DW_ERROR(dwPathDetector_setDetectionROI(&detectionROI, m_pathDetector));
      spdlog::info("PathNet initialized!");
    }
    spdlog::info("Init successful!");
    return true;
  }
2

Dear dengqi,
There were no error, the program is still running, but not go to next step.

From the code snippet it is not clear. Could you attach all necessary files here to compile it on my machine

3

Dear SivaRamaKrishna,

Thanks for reply. I found the reason is because that the socket connection was blocked.

4

Hello, I wanted to know where I can find information about the Mapnet network. I understand that it is a software owned by nvidia but there are any papers that explains how it detects the lines or some academic explanation (no marketing). What it needs for input and what you get output? I did not find anything about it, so I wanted to know how it was trainned, what type of images, maybe nvidia dataset or free dataset? Can this model be retrained with my own images? How can this network be quantitatively evaluated? @SivaRamaKrishnaNV