ISP APIs on TX1

Greetings,

How can we use ISP on TX1. Does/will nVidia provide some APIs to use them or ISP functionaries will be appear gst plugins? In our usecase, we are trying to we have a couple of CSI camera and intend to use ISP to do some of the features from TX1 datasheet section ‘1.4 Image Signal Processor (ISP)’

-Bayer domain hardware noise reduction
-Per-channel black-level compensation
-High-order lens-shading compensation
-3x3 color transform
-Bad pixel correction
-Programmable coefficients for de-mosaic with color artifact reduction
Color Artifact Reduction: a two-level (horizontal and vertical) low-pass filtering scheme that is used to reduce/remove
any color artifacts that may result from Bayer signal processing and the effects of sampling an image.
-Enhanced down scaling quality
-Edge Enhancement

Does anyone know how to access the ISP on TX1? Can someone from nVidia please answer this?

Thanks

Currently it’s used through gstreamer. You can inspect the available settings by running gst-inspect-1.0 nvcamerasrc

Thanks dusty_nv for prompt reply.

My understanding is nvcamerasrc is hardcoded for on-board CSI cameras. If we are going to design our own CSI-V4L2 driver for our own cameras, how we can use those ISP APIs? I see the source code for nvcamerasrc has been released in gst-omx. Do we only have to use that as a reference if we want to use those ISP APIs with our own V4L2 drivers?

Hi dk1900,

My tests with this component show the same. Moreover, the name ov5693 (this is the camera TX1 dev board is supplied with) seems to be hardcoded as well, so I couldn’t start any other camera. The only way to do it was to modify the ov5693 driver itself. However, resolutions are hardcoded also, as far as I couldn’t start the pipeline with anything different from 2592x1944.

Where did you see the source for nvcamerasrc exactly?

Hi Amitev,

My bad. I though some code in gst-omx is for NvCamera. gst-inspect-1.0 nvcamerasrc shows below information. The license for this plugin marked as ‘Proprietary’. No sure NVIDIA will release the source code to us or not.

Plugin Details:
Name nvcamera
Description Nvidia Video Capture Component
Filename /home/ubuntu/gst-1.6.0/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstnvcamera.so
Version 1.2.0
License Proprietary
Source module nvcamera
Binary package NvCamera
Origin URL http://nvidia.com

Actually, the source code can be found here:

http://developer.nvidia.com/embedded/dlc/l4t-23-2-nv-gst-apps-sources

This is the source code of the sample applications, not the plugin “nvcamerasrc”, nor the mysterious camera daemon.

Hi dk1900,

We’ll announce new camera functionality with the release of L4T 24.1.
For now, we support USB, we support YUV, we support RAW in ISP-bypass, and we support OV5693 RAW processed.
We do not currently support ISP processing for any other RAW cameras at this time.

Thanks

Hey - kaycc. Thanks for your information. Do you have a rough timeline about L4T24.1?

Is there any update on this? Why isn’t nvidia releasing any documentation or source for the ISP? It’s impossible for us to develop applications or even use the TX1 for evaluation purposes unless we can get access to ISP.

I think you can find the information here: https://devtalk.nvidia.com/default/topic/975962/jetson-tx1/ridgerun-support-for-six-raspberry-pi-v2-imx219-and-v1-ov5647-cameras-using-isp-multi-camera/2