ISP fails depending on Log output level

Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.6.1.8175
1.6.0.8170
other 1.4.1.7402

Host Machine Version
native Ubuntu 18.04
other

Dear Support,

We use NvSIPL to control the Camera on Drive OS 5.2.0.
Pipeline consists of Camera output and ISP0 output.

When I set the Log Level of INvSIPLTrace to the Info level, it works normally.
When set to Warning, the ISP will output an error.

NOTIF_INFO_ICP_PROCESSING_DONE. Pipeline: 8
[SiplControlSatCscProductCheck:1525] [SiplControlSatCscProductCheck:1525] Bad input parameter
[SiplControlIspRangeCheck:1600] [SiplControlIspRangeCheck:1600] Bad input parameter
[SiplControlIspProcess:614] [SiplControlIspProcess:614] Bad input parameter
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 503: ProgramISP: NvSIPLCtrlProcess failed. status:127
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 382: ProcessISP: ProgramISP failed
SIPL_ICP_ISP_8: CNvMSensorPipeline.cpp: 735: DoISPProcessing: ISP block process isp failed
NOTIF_ERROR_ISP_PROCESSING_FAILURE. Pipeline: 8
Pipeline status = NVSIPL_UNKNOWN_ERROR

You can see from the Log of NOTIF_INFO_ICP_PROCESSING_DONE. That the data can be read normally up to ICP.

Why is the Bad input parameter output by the ISP at the Log output level?
Please let me know if there are any APIs or parameters that should be set.

Dear @N.Miya,
Could you share camera model and used command to reproduce the issue. I am assuming you are using nvsipl_camera sample.

Dear @SivaRamaKrishnaNV

We are using IMX390.
I want to do raw to rgba conversion from raw data using SIPL simulator.

Therefore, since the implementation is different from the actual sample, the code of the construction part of the pipeline is shared instead of the command.

Please let me know if there are any missing settings.

  // INvSIPLQuery
  auto query = INvSIPLQuery::GetInstance();
  if (query == nullptr) {
    DLOG_DEBUG(0, "INvSIPLQuery::GetInstance() failed.");
    return kFailure;
  }

  // Parse database
  auto status = query->ParseDatabase();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("INvSIPLQuery::ParseDatabase() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  // Set verbosity level
  INvSIPLQueryTrace::GetInstance()->SetLevel(INvSIPLQueryTrace::TraceLevel::LevelCritical);
  INvSIPLTrace::GetInstance()->SetLevel(INvSIPLTrace::TraceLevel::LevelWarning);
  CLogger::GetInstance().SetLogLevel(CLogger::LogLevel::LEVEL_WARNING);
  // INvSIPLTrace::GetInstance()->SetLevel(INvSIPLTrace::TraceLevel::LevelDebug);
  // CLogger::GetInstance().SetLogLevel(CLogger::LogLevel::LEVEL_INFORMATION);
  
  SigSetup();

  // Get PlatformCfg
  PlatformCfg platform_cfg;
  status = query->GetPlatformCfg("IMX390_RGGB_CPHY_x4", platform_cfg);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("INvSIPLQuery::GetPlatformCfg() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  // Set SIPL mask

  status = query->ApplyMask(platform_cfg, { 0, 0, 1, 0});
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("INvSIPLQuery::ApplyMask() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }


  if (platform_cfg.numDeviceBlocks != 1) {
      DLOG_DEBUG(0, "Only one device block is supported in simulator mode. Please correct mask.");
      return kFailure;
  }
  if (platform_cfg.deviceBlockList[0].numCameraModules != 1) {
      DLOG_DEBUG(0, "Only one camera module is supported in simulator mode. Please correct mask.");
      return kFailure;
  }

  platform_cfg.deviceBlockList[0].isSimulatorModeEnabled = true;

  sipl_inputter_.reset(new SIPLDataInputter());
  const auto& vcinfo = platform_cfg.deviceBlockList[0].cameraModuleInfoList[0].sensorInfo.vcInfo;
  status = sipl_inputter_->Init(vcinfo, &inputter_quit_);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("SIPLDataInputter::Init() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  sipl_notify_handler_.reset(new CPipelineNotificationHandler());
  sipl_master_.reset(new CNvSIPLMaster());

  NvSciBufModule buf_module = nullptr;
  NvSciSyncModule sync_module = nullptr;
  status = sipl_master_->Setup(&platform_cfg, &buf_module, &sync_module);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::Setup() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  sipl_consumer_.reset(new CNvSIPLConsumer());
  status = sipl_consumer_->Init();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLConsumer::Init() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  INvSIPLClient::ConsumerDesc client;
  client.eOutputType = INvSIPLClient::ConsumerDesc::OutputType::ISP0;
  client.callback = sipl_consumer_.get();

  NvSIPLPipelineCfg pipeline_cfg;
  pipeline_cfg.clients.push_back(client);
  pipeline_cfg.notifier = sipl_notify_handler_.get();
  pipeline_cfg.imageGroupWriter = sipl_inputter_.get();

  uint32_t sensor_id = platform_cfg.deviceBlockList[0].cameraModuleInfoList[0].sensorInfo.id;
  status = sipl_master_->SetPipelineConfig(sensor_id, pipeline_cfg);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::SetPipelineConfig() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  status = sipl_master_->Init();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::Init() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  status = sipl_master_->AllocateAndRegisterBuffers(sensor_id, true, false);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::AllocateAndRegisterBuffers() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  std::string module_name = platform_cfg.deviceBlockList[0].cameraModuleInfoList[0].name;
  std::vector<std::uint8_t> blob;
  status = LoadNITOFile("/opt/nvidia/nvmedia/nit/", module_name, blob);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("LoadNITOFile() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  status = sipl_master_->RegisterAutoControl(sensor_id, NV_PLUGIN, nullptr, blob);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::RegisterAutoControl() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }
  
  auto status = sipl_master_->Start();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::Start() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

Thank you.

Do you have any update on this issue?

Dear @N.Miya,
When I set the Log Level of INvSIPLTrace to the Info level, it works normally

Does that mean you could get RGBA without any issue in this case and failed to get only when set to Warning? Or it is different issue than the original query?

Thank you for your reply.

However, we are having trouble at the Warning level.
In the case of Info, it is working normally.

Does that mean you could get RGBA without any issue in this case and failed to get only when set to Warning?

Yes. I can’t get RGBA when we set it to Warning as below.
INvSIPLTrace::GetInstance()->SetLevel(INvSIPLTrace::TraceLevel::LevelWarning);

Dear @SivaRamaKrishnaNV ,

When I set the output to VUYX, it worked without the Bad input Parameter even at the Warning level.

The RGBA output is set as shown in the commented out part below. Is the setting wrong?

    SIPLStatus AllocateAndRegisterBuffers(std::uint32_t pip,
                                          bool isp0Enabled,
                                          bool isp1Enabled)
    {
        // Get attributes
        auto pDevice = m_upDevice.get();
        NvSIPLImageAttr attr;
        auto status = m_pCamera->GetImageAttributes(pip, INvSIPLClient::ConsumerDesc::OutputType::ICP, attr);
        if (status != NVSIPL_STATUS_OK) {
            LOG_ERR("GetImageAttributes failed\n");
            return status;
        }
        std::vector<NvMediaImageGroup*> imageGroups;
        status = AllocateImageGroups(pDevice, attr.surfaceType, attr.surfaceAllocAttr, 1U, imageGroups);
        if (status != NVSIPL_STATUS_OK) {
            LOG_ERR("AllocateImageGroups failed\n");
            return status;
        }
        status = m_pCamera->RegisterImageGroups(pip, imageGroups);
        if (status != NVSIPL_STATUS_OK) {
            LOG_WARN("RegisterImageGroups failed\n");
            DeleteImageGroups(imageGroups);
            return status;
        }
        m_imageGroupList.push_back(imageGroups);

        if (isp0Enabled) {
            // Get attributes
            NvSIPLImageAttr attr;
            status = m_pCamera->GetImageAttributes(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP0, attr);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("GetImageAttributes failed\n");
                return status;
            }
            /* surf set rgba*/
            NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
            NVM_SURF_FMT_SET_ATTR_YUV (surfFormatAttrs, VUYX,NONE,PACKED,UINT,8,PL);
            // NVM_SURF_FMT_SET_ATTR_RGBA (surfFormatAttrs, RGBA, UINT, 8, PL);

            attr.surfaceType =NvMediaSurfaceFormatGetType(surfFormatAttrs, NVM_SURF_FMT_ATTR_MAX);
            attr.surfaceAllocAttr[3].value = NVM_SURF_ATTR_COLOR_STD_REC709_ER;
            // attr.surfaceAllocAttr[3].value = NVM_SURF_ATTR_COLOR_STD_SRGB;
            /* end */

            std::vector<NvMediaImage*> images;
            status = AllocateImages(pDevice, attr.surfaceType, attr.surfaceAllocAttr, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("AllocateImages failed\n");
                return status;
            }
            status = m_pCamera->RegisterImages(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP0, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_WARN("RegisterImageGroups failed\n");
                DeleteImages(images);
                return status;
            }
            m_imagesList.push_back(images);
        }

        if (isp1Enabled) {
            // Get attributes
            NvSIPLImageAttr attr;
            status = m_pCamera->GetImageAttributes(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP1, attr);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("GetImageAttributes failed\n");
                return status;
            }
            std::vector<NvMediaImage*> images;
            status = AllocateImages(pDevice, attr.surfaceType, attr.surfaceAllocAttr, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("AllocateImageGroups failed\n");
                return status;
            }
            status = m_pCamera->RegisterImages(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP1, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_WARN("RegisterImageGroups failed\n");
                DeleteImages(images);
                return status;
            }
            m_imagesList.push_back(images);
        }

        return NVSIPL_STATUS_OK;
    }

Dear @N.Miya,
As for as I know, there should not be any dependency on INvSIPLTrace::TraceLevel parameters. I will check internally and confirm.

Hi, @N.Miya

Why do you overwrite the value of index 3 attribute? Please make sure you overwrite the value of NVM_SURF_ATTR_COLOR_STD_TYPE attribute.

Dear @VickNV ,

Why do you overwrite the value of index 3 attribute?

When I checked the type of index 3 attribute, the value was 8.

According to the following, 8 is NVM_SURF_ATTR_COLOR_STD_TYPE, so the place to overwrite should be correct.
https://docs.nvidia.com/drive/drive-os-5.2.0.0L/drive-os/DRIVE_OS_Linux_SDK_Development_Guide/baggage/group__surface__handling__api.html#ggab71504a12ce4af12741d26b723c21878a6f21929ba1a11263bf644e65d0d885a6

Please try to overwrite it with the below snippet and see if it helps.

            for (std::vector<int>::size_type i = 0U; i < attr.surfaceAllocAttr.size(); i++) {                                                                                          
                if (attr.surfaceAllocAttr[i].type == NVM_SURF_ATTR_COLOR_STD_TYPE) {
                    attr.surfaceAllocAttr[i].value = NVM_SURF_ATTR_COLOR_STD_SRGB;
                    break;                                                      
                }                                                               
            }

I changed the code for below and confirmed that NVM_SURF_ATTR_COLOR_STD_TYPE was set as intended.
However, the situation remains the same and I get an ISP error at the Warning level.

        NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
        NVM_SURF_FMT_SET_ATTR_RGBA (surfFormatAttrs, RGBA, UINT, 8, PL);

        attr.surfaceType =NvMediaSurfaceFormatGetType(surfFormatAttrs, NVM_SURF_FMT_ATTR_MAX);

        for (std::vector<int>::size_type i = 0U; i < attr.surfaceAllocAttr.size(); i++) {                                                                                          
            if (attr.surfaceAllocAttr[i].type == NVM_SURF_ATTR_COLOR_STD_TYPE) {
                attr.surfaceAllocAttr[i].value = NVM_SURF_ATTR_COLOR_STD_SRGB;
                cout << "attr.surfaceAllocAttr i = " << i << "\n";
                break;                                                      
            }                                                               
        }

Console output

attr.surfaceAllocAttr i = 3
[SiplControlSatCscProductCheck:1525] [SiplControlSatCscProductCheck:1525] Bad input parameter
[SiplControlIspRangeCheck:1600] [SiplControlIspRangeCheck:1600] Bad input parameter
[SiplControlIspProcess:614] [SiplControlIspProcess:614] Bad input parameter
NOTIF_INFO_ICP_PROCESSING_DONE. Pipeline: 8
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 503: ProgramISP: NvSIPLCtrlProcess failed. status:127
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 382: ProcessISP: ProgramISP failed
SIPL_ICP_ISP_8: CNvMSensorPipeline.cpp: 735: DoISPProcessing: ISP block process isp failed
NOTIF_ERROR_ISP_PROCESSING_FAILURE. Pipeline: 8

I haven’t tried on DRIVE OS 5.2.0. It works on DRIVE OS 5.2.6. Please try on 5.2.6. Thanks.

Does it work on 5.2.6 on your side?