About ISP interface in Nvmedia

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after clicking “+ Create Topic”):
Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.6.0.8170
1.5.1.7815
1.5.0.7774
other

Host Machine Version
native Ubuntu 18.04
other

Hi,

I’ve noticed that in the previous DriveOS document, for example the 5.1.6.0, has a section that describes the ISP interface. But in latter (5.2) versions that section no longer exist. I can also find the ISP sample code on 5.1.6.0 DriveOS image, but not on 5.2 version. But there are ISP related .h files in the drive-t186ref-linux/include on the 5.2 version.

Are old ISP document and example applicable to new DriveOS version? Is there any reason of removing ISP document & sample? I found the ISP functionality is included in SIPL pipeline by registering image with ISP type, Is it not recommended to use ISP directly, and use RegisterImages() with ISP instead?

2

Dear @leqiang.wang,
Note that, IPP is deprecated and we recommend to use SIPL instead. Could you please check https://docs.nvidia.com/drive/drive-os-5.2.0.0L/drive-os/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/NvMedia/nvmedia_porting_guide.html#wwpID0EQHA.

3

What about ISP? SIPL usage of ISP is to put images in a buffer queue, config the buffer, and process them in a pipeline manner, am I correct? Can I write my own function that process single image using ISP accelerator with SIPL?

4

Hi SivaRamaKrishnaNV,

Few more questions about ISP.

  1. Looking to understand overall capabilities of ISP - what operations can it do ? I know debayer/scale - can it do CSC (color space conversion) - can it do histograms etc ?
  2. How many instances of ISP are there (on xavier) ?
  3. Is there any sample or doc or recommended way of time sharing ISP among multiple cameras ?
  4. Are there any examples/samples which illustrates usage of ISP with other blocks like encoder, GPU, DLA, PVA ?

Thanks.

5

Please take a look at SIPL API.
What CSC are you looking for?
Please check histogram related fields in nvsipl::INvSIPLClient::ImageMetaData Struct Reference.

There are two instances. You can check SIPL Camera (nvsipl_camera).

Please check the above sample application.

You can do it with NvStreams.

6

I would like to pipe camera output to GPU, PVA and encoder all at the same time (i.e. same camera frame going to 3 different frame consumer accelerators). GPU wants it in RGB, Encoder wants it in YUV420. I’m not sure about format required by PVA yet (please let me know what are possible input formats for PVA) . So I would say I need conversion from bayer to RGB ; bayer to yuv 420, and possibly others (which I do not know yet).

Could CSC also handle bayer to yuv420 Nv12 vs Nv21 ? Could it handle converting it to tiled rather than raster ?

Thanks.

7

Hi VickNV,
Any update on aforesaid questions please ?
Thanks

8

Hi VickNV,

I would like to pipe camera output to GPU, PVA and encoder all at the same time (i.e. same camera frame going to 3 different frame consumer accelerators). GPU wants it in RGB, Encoder wants it in YUV420. I’m not sure about format required by PVA yet (please let me know what are possible input formats for PVA) . So I would say I need conversion from bayer to RGB ; bayer to yuv 420, and possibly others (which I do not know yet).

Is there a DW or D-OS sample that gets close this description ?

Thanks.

9

You can leverage SIPL, 2D to do it. Also you can refer to Vision Programming document and nvm_vpi sample application to understand PVA more.