ISP fails depending on Log output level

Software Version
DRIVE OS Linux 5.2.6
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.6.1.8175
1.6.0.8170
other 1.4.1.7402

Host Machine Version
native Ubuntu 18.04
other

Dear Support,

We use NvSIPL to control the Camera on Drive OS 5.2.0.
Pipeline consists of Camera output and ISP0 output.

When I set the Log Level of INvSIPLTrace to the Info level, it works normally.
When set to Warning, the ISP will output an error.

NOTIF_INFO_ICP_PROCESSING_DONE. Pipeline: 8
[SiplControlSatCscProductCheck:1525] [SiplControlSatCscProductCheck:1525] Bad input parameter
[SiplControlIspRangeCheck:1600] [SiplControlIspRangeCheck:1600] Bad input parameter
[SiplControlIspProcess:614] [SiplControlIspProcess:614] Bad input parameter
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 503: ProgramISP: NvSIPLCtrlProcess failed. status:127
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 382: ProcessISP: ProgramISP failed
SIPL_ICP_ISP_8: CNvMSensorPipeline.cpp: 735: DoISPProcessing: ISP block process isp failed
NOTIF_ERROR_ISP_PROCESSING_FAILURE. Pipeline: 8
Pipeline status = NVSIPL_UNKNOWN_ERROR

You can see from the Log of NOTIF_INFO_ICP_PROCESSING_DONE. That the data can be read normally up to ICP.

Why is the Bad input parameter output by the ISP at the Log output level?
Please let me know if there are any APIs or parameters that should be set.

Dear @N.Miya,
Could you share camera model and used command to reproduce the issue. I am assuming you are using nvsipl_camera sample.

Dear @SivaRamaKrishnaNV

We are using IMX390.
I want to do raw to rgba conversion from raw data using SIPL simulator.

Therefore, since the implementation is different from the actual sample, the code of the construction part of the pipeline is shared instead of the command.

Please let me know if there are any missing settings.

  // INvSIPLQuery
  auto query = INvSIPLQuery::GetInstance();
  if (query == nullptr) {
    DLOG_DEBUG(0, "INvSIPLQuery::GetInstance() failed.");
    return kFailure;
  }

  // Parse database
  auto status = query->ParseDatabase();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("INvSIPLQuery::ParseDatabase() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  // Set verbosity level
  INvSIPLQueryTrace::GetInstance()->SetLevel(INvSIPLQueryTrace::TraceLevel::LevelCritical);
  INvSIPLTrace::GetInstance()->SetLevel(INvSIPLTrace::TraceLevel::LevelWarning);
  CLogger::GetInstance().SetLogLevel(CLogger::LogLevel::LEVEL_WARNING);
  // INvSIPLTrace::GetInstance()->SetLevel(INvSIPLTrace::TraceLevel::LevelDebug);
  // CLogger::GetInstance().SetLogLevel(CLogger::LogLevel::LEVEL_INFORMATION);
  
  SigSetup();

  // Get PlatformCfg
  PlatformCfg platform_cfg;
  status = query->GetPlatformCfg("IMX390_RGGB_CPHY_x4", platform_cfg);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("INvSIPLQuery::GetPlatformCfg() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  // Set SIPL mask

  status = query->ApplyMask(platform_cfg, { 0, 0, 1, 0});
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("INvSIPLQuery::ApplyMask() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }


  if (platform_cfg.numDeviceBlocks != 1) {
      DLOG_DEBUG(0, "Only one device block is supported in simulator mode. Please correct mask.");
      return kFailure;
  }
  if (platform_cfg.deviceBlockList[0].numCameraModules != 1) {
      DLOG_DEBUG(0, "Only one camera module is supported in simulator mode. Please correct mask.");
      return kFailure;
  }

  platform_cfg.deviceBlockList[0].isSimulatorModeEnabled = true;

  sipl_inputter_.reset(new SIPLDataInputter());
  const auto& vcinfo = platform_cfg.deviceBlockList[0].cameraModuleInfoList[0].sensorInfo.vcInfo;
  status = sipl_inputter_->Init(vcinfo, &inputter_quit_);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("SIPLDataInputter::Init() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  sipl_notify_handler_.reset(new CPipelineNotificationHandler());
  sipl_master_.reset(new CNvSIPLMaster());

  NvSciBufModule buf_module = nullptr;
  NvSciSyncModule sync_module = nullptr;
  status = sipl_master_->Setup(&platform_cfg, &buf_module, &sync_module);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::Setup() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  sipl_consumer_.reset(new CNvSIPLConsumer());
  status = sipl_consumer_->Init();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLConsumer::Init() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  INvSIPLClient::ConsumerDesc client;
  client.eOutputType = INvSIPLClient::ConsumerDesc::OutputType::ISP0;
  client.callback = sipl_consumer_.get();

  NvSIPLPipelineCfg pipeline_cfg;
  pipeline_cfg.clients.push_back(client);
  pipeline_cfg.notifier = sipl_notify_handler_.get();
  pipeline_cfg.imageGroupWriter = sipl_inputter_.get();

  uint32_t sensor_id = platform_cfg.deviceBlockList[0].cameraModuleInfoList[0].sensorInfo.id;
  status = sipl_master_->SetPipelineConfig(sensor_id, pipeline_cfg);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::SetPipelineConfig() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  status = sipl_master_->Init();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::Init() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  status = sipl_master_->AllocateAndRegisterBuffers(sensor_id, true, false);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::AllocateAndRegisterBuffers() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  std::string module_name = platform_cfg.deviceBlockList[0].cameraModuleInfoList[0].name;
  std::vector<std::uint8_t> blob;
  status = LoadNITOFile("/opt/nvidia/nvmedia/nit/", module_name, blob);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("LoadNITOFile() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

  status = sipl_master_->RegisterAutoControl(sensor_id, NV_PLUGIN, nullptr, blob);
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::RegisterAutoControl() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }
  
  auto status = sipl_master_->Start();
  if (status != NVSIPL_STATUS_OK) {
    DLOG_DEBUG(0, ("CNvSIPLMaster::Start() failed. status = " + logger_.GetSIPLStatusMsg(status)).c_str());
    return kFailure;
  }

Thank you.

Do you have any update on this issue?

Dear @N.Miya,
When I set the Log Level of INvSIPLTrace to the Info level, it works normally

Does that mean you could get RGBA without any issue in this case and failed to get only when set to Warning? Or it is different issue than the original query?

Thank you for your reply.

However, we are having trouble at the Warning level.
In the case of Info, it is working normally.

Does that mean you could get RGBA without any issue in this case and failed to get only when set to Warning?

Yes. I can’t get RGBA when we set it to Warning as below.
INvSIPLTrace::GetInstance()->SetLevel(INvSIPLTrace::TraceLevel::LevelWarning);

Dear @SivaRamaKrishnaNV ,

When I set the output to VUYX, it worked without the Bad input Parameter even at the Warning level.

The RGBA output is set as shown in the commented out part below. Is the setting wrong?

    SIPLStatus AllocateAndRegisterBuffers(std::uint32_t pip,
                                          bool isp0Enabled,
                                          bool isp1Enabled)
    {
        // Get attributes
        auto pDevice = m_upDevice.get();
        NvSIPLImageAttr attr;
        auto status = m_pCamera->GetImageAttributes(pip, INvSIPLClient::ConsumerDesc::OutputType::ICP, attr);
        if (status != NVSIPL_STATUS_OK) {
            LOG_ERR("GetImageAttributes failed\n");
            return status;
        }
        std::vector<NvMediaImageGroup*> imageGroups;
        status = AllocateImageGroups(pDevice, attr.surfaceType, attr.surfaceAllocAttr, 1U, imageGroups);
        if (status != NVSIPL_STATUS_OK) {
            LOG_ERR("AllocateImageGroups failed\n");
            return status;
        }
        status = m_pCamera->RegisterImageGroups(pip, imageGroups);
        if (status != NVSIPL_STATUS_OK) {
            LOG_WARN("RegisterImageGroups failed\n");
            DeleteImageGroups(imageGroups);
            return status;
        }
        m_imageGroupList.push_back(imageGroups);

        if (isp0Enabled) {
            // Get attributes
            NvSIPLImageAttr attr;
            status = m_pCamera->GetImageAttributes(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP0, attr);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("GetImageAttributes failed\n");
                return status;
            }
            /* surf set rgba*/
            NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
            NVM_SURF_FMT_SET_ATTR_YUV (surfFormatAttrs, VUYX,NONE,PACKED,UINT,8,PL);
            // NVM_SURF_FMT_SET_ATTR_RGBA (surfFormatAttrs, RGBA, UINT, 8, PL);

            attr.surfaceType =NvMediaSurfaceFormatGetType(surfFormatAttrs, NVM_SURF_FMT_ATTR_MAX);
            attr.surfaceAllocAttr[3].value = NVM_SURF_ATTR_COLOR_STD_REC709_ER;
            // attr.surfaceAllocAttr[3].value = NVM_SURF_ATTR_COLOR_STD_SRGB;
            /* end */

            std::vector<NvMediaImage*> images;
            status = AllocateImages(pDevice, attr.surfaceType, attr.surfaceAllocAttr, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("AllocateImages failed\n");
                return status;
            }
            status = m_pCamera->RegisterImages(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP0, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_WARN("RegisterImageGroups failed\n");
                DeleteImages(images);
                return status;
            }
            m_imagesList.push_back(images);
        }

        if (isp1Enabled) {
            // Get attributes
            NvSIPLImageAttr attr;
            status = m_pCamera->GetImageAttributes(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP1, attr);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("GetImageAttributes failed\n");
                return status;
            }
            std::vector<NvMediaImage*> images;
            status = AllocateImages(pDevice, attr.surfaceType, attr.surfaceAllocAttr, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("AllocateImageGroups failed\n");
                return status;
            }
            status = m_pCamera->RegisterImages(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP1, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_WARN("RegisterImageGroups failed\n");
                DeleteImages(images);
                return status;
            }
            m_imagesList.push_back(images);
        }

        return NVSIPL_STATUS_OK;
    }

Dear @N.Miya,
As for as I know, there should not be any dependency on INvSIPLTrace::TraceLevel parameters. I will check internally and confirm.

Hi, @N.Miya

Why do you overwrite the value of index 3 attribute? Please make sure you overwrite the value of NVM_SURF_ATTR_COLOR_STD_TYPE attribute.

Dear @VickNV ,

Why do you overwrite the value of index 3 attribute?

When I checked the type of index 3 attribute, the value was 8.

According to the following, 8 is NVM_SURF_ATTR_COLOR_STD_TYPE, so the place to overwrite should be correct.
https://docs.nvidia.com/drive/drive-os-5.2.0.0L/drive-os/DRIVE_OS_Linux_SDK_Development_Guide/baggage/group__surface__handling__api.html#ggab71504a12ce4af12741d26b723c21878a6f21929ba1a11263bf644e65d0d885a6

Please try to overwrite it with the below snippet and see if it helps.

            for (std::vector<int>::size_type i = 0U; i < attr.surfaceAllocAttr.size(); i++) {                                                                                          
                if (attr.surfaceAllocAttr[i].type == NVM_SURF_ATTR_COLOR_STD_TYPE) {
                    attr.surfaceAllocAttr[i].value = NVM_SURF_ATTR_COLOR_STD_SRGB;
                    break;                                                      
                }                                                               
            }

I changed the code for below and confirmed that NVM_SURF_ATTR_COLOR_STD_TYPE was set as intended.
However, the situation remains the same and I get an ISP error at the Warning level.

        NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
        NVM_SURF_FMT_SET_ATTR_RGBA (surfFormatAttrs, RGBA, UINT, 8, PL);

        attr.surfaceType =NvMediaSurfaceFormatGetType(surfFormatAttrs, NVM_SURF_FMT_ATTR_MAX);

        for (std::vector<int>::size_type i = 0U; i < attr.surfaceAllocAttr.size(); i++) {                                                                                          
            if (attr.surfaceAllocAttr[i].type == NVM_SURF_ATTR_COLOR_STD_TYPE) {
                attr.surfaceAllocAttr[i].value = NVM_SURF_ATTR_COLOR_STD_SRGB;
                cout << "attr.surfaceAllocAttr i = " << i << "\n";
                break;                                                      
            }                                                               
        }

Console output

attr.surfaceAllocAttr i = 3
[SiplControlSatCscProductCheck:1525] [SiplControlSatCscProductCheck:1525] Bad input parameter
[SiplControlIspRangeCheck:1600] [SiplControlIspRangeCheck:1600] Bad input parameter
[SiplControlIspProcess:614] [SiplControlIspProcess:614] Bad input parameter
NOTIF_INFO_ICP_PROCESSING_DONE. Pipeline: 8
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 503: ProgramISP: NvSIPLCtrlProcess failed. status:127
SIPL_ICP_ISP_8: CNvMISPBlock.cpp: 382: ProcessISP: ProgramISP failed
SIPL_ICP_ISP_8: CNvMSensorPipeline.cpp: 735: DoISPProcessing: ISP block process isp failed
NOTIF_ERROR_ISP_PROCESSING_FAILURE. Pipeline: 8

I haven’t tried on DRIVE OS 5.2.0. It works on DRIVE OS 5.2.6. Please try on 5.2.6. Thanks.

Does it work on 5.2.6 on your side?

@N.Miya
is this problem solved? I didn’t see any issue on 5.2.0 too.
If not, please share your nvsipl_camera command and changed source files.

@VickNV

No, this plroblem is not solved.

This is nvsipl_camera command and changed source files.

./nvsipl_camera -c “IMX390RGGBE3_200FOV_RGGB_CPHY_x4” -m “0x0001 0x0000 0x0000 0x0000” -i “imx390.raw” --disableISP1Output -v 1

CNvSIPLMaster.hpp (21.2 KB)

I tried 18 times and got an error once.

I will attach the logs for 3 times including before the occurrence.

nvidia@tegra-ubuntu:/ota/pkg_data/work$ ./nvsipl_camera -c "IMX390RGGBE3_200FOV_RGGB_CPHY_x4" -m "0x0001 0x0000 0x0000 0x0000" -i "imx390.raw"  --disableISP1Output -v 1
nvsipl_camera: Opened NITO file for module "IMX390RGGBE3_200FOV_RGGB"
Enter 'les' followed by sensor ID to enable LED
Enter 'lds' followed by sensor ID to disable LED
Enter 'q' to quit the application
-
q
SUCCESS
nvidia@tegra-ubuntu:/ota/pkg_data/work$ ./nvsipl_camera -c "IMX390RGGBE3_200FOV_RGGB_CPHY_x4" -m "0x0001 0x0000 0x0000 0x0000" -i "imx390.raw"  --disableISP1Output -v 1
nvsipl_camera: Opened NITO file for module "IMX390RGGBE3_200FOV_RGGB"
Enter 'les' followed by sensor ID to enable LED
Enter 'lds' followed by sensor ID to disable LED
Enter 'q' to quit the application
-
q
nvsipl_camera: ERROR: imageSize(4735456) does not match expected surface size(0)

SIPL_ICP_ISP_0: CNvMSensorPipeline.cpp: 674: GetFeederBuffer: FillRawBuffer failed
nvsipl_camera: ERROR: Pipeline: 0, NOTIF_ERROR_INTERNAL_FAILURE

nvsipl_camera: ERROR: Pipeline failure

nvidia@tegra-ubuntu:/ota/pkg_data/work$ ./nvsipl_camera -c "IMX390RGGBE3_200FOV_RGGB_CPHY_x4" -m "0x0001 0x0000 0x0000 0x0000" -i "imx390.raw"  --disableISP1Output -v 1
nvsipl_camera: Opened NITO file for module "IMX390RGGBE3_200FOV_RGGB"
Enter 'les' followed by sensor ID to enable LED
Enter 'lds' followed by sensor ID to disable LED
Enter 'q' to quit the application
-
[SiplControlSatCscProductCheck:1525] [SiplControlSatCscProductCheck:1525] Bad input parameter
[SiplControlIspRangeCheck:1600] [SiplControlIspRangeCheck:1600] Bad input parameter
[SiplControlIspProcess:614] [SiplControlIspProcess:614] Bad input parameter
SIPL_ICP_ISP_0: CNvMISPBlock.cpp: 503: ProgramISP: NvSIPLCtrlProcess failed. status:127
SIPL_ICP_ISP_0: CNvMISPBlock.cpp: 382: ProcessISP: ProgramISP failed
SIPL_ICP_ISP_0: CNvMSensorPipeline.cpp: 735: DoISPProcessing: ISP block process isp failed
nvsipl_camera: ERROR: Pipeline: 0, NOTIF_ERROR_ISP_PROCESSING_FAILURE

nvsipl_camera: ERROR: Pipeline failure

We didn’t know that you were talking about a problem with failure rate.
Is this problem still related to the logging level?

Yes, it is related.

The sample is less reproducible, but when incorporated into our application, it is more reproducible.
When incorporated into our program, it occurs every time at the Warning level.

Could you try to narrow down your application and clarify what increases the reproduction rate?

We also want to identify the cause, but it is difficult because there is only information on the Bad input parameter as Log.

I think there is a problem within SiplControlSatCscProductCheck. What are the conditions under which the Bad input parameter is output?

[SiplControlSatCscProductCheck:1525] [SiplControlSatCscProductCheck:1525] Bad input parameter
[SiplControlIspRangeCheck:1600] [SiplControlIspRangeCheck:1600] Bad input parameter
[SiplControlIspProcess:614] [SiplControlIspProcess:614] Bad input parameter