ISP configuration files

Hi,

I read in [1] that it is possible to use configuration files for the ISP in the file system and therefore use either v4l2 (for YUV sensors) and nvcamerasrc (for bayer sensors). 

a) Am I correct about saying that if we initialize the camera sensor (with our driver) and use a configuration file for the ISP, it would be possible to capture from any sensor with nvcamerasrc? Basically that nvcamerasrc is agnostic of the sensor if you have the correct external initialization for the sensor and the correct ISP configuration?

b) Where should I put the ISP configuration file? I see this in the documentation for L4T:

ISP Support

ISP support can be enabled as follows:
•Built-in to the Camera Core where the release package includes initial ISP configuration files for reference sensors.
•Place the ISP configuration file into RootFS so that when the system boots, it scans the pre-defined folders to search for the ISP configuration files that match the sensor module. This can be performed at runtime to provide more flexibility.

c) I read about openKcam as well, can it be used currently with other sensors?

I noticed that libnvodm_imager.so seems to be the one that parses the file.

[1] https://devtalk.nvidia.com/default/topic/968475/jetson-tx1/yavta-is-creating-output-file-with-only-zero/1

a. You are rights, nvcamerasrc is agnostic of sensor initialization
b. We have an integrated ISP file for dedicated ov5693. But for the other sensors, you may need to ask help from our camera partners for ISP tuning. NVIDIA doesn’t provide explicit support for it.
c. Not very clear about this question

Thanks for your answers. My last question was about the use of openKcam on tegra X1, however, it seems that instead it should be libargus:

https://devtalk.nvidia.com/default/topic/965351/jetson-tx1/l4t-24-2-isp-configuration-options/post/4994588/#4994588

but that seems like a library that is not using the ISP but instead the GPU.

In any case, I think I will try using the ISP configuration file to overwrite the default configuration.

Thank you,
David

The ISP configuration files are only to calibrate the ISP to work correctly with the sensor. To support other sensors the default configuration can be used. We already have a calibration file that could use if modifications are needed.

https://devtalk.nvidia.com/default/topic/975962/support-for-raspberry-pi-v2-imx219-and-v1-ov5647-cameras-using-isp-multi-camera/

Hi DavidSoto/nVConan ,

I am playing around with arguslib to understand how to use TX1 ISP. arguslib samples (E.g.: oneShot) are working well with Ov5693 coming with TX1 developer kit. However, I am getting some error with other sensors (which are working fine using V4L2 apis).

When the below piece of code is executed, though driver calls like power_on/power_off are called, application exits with an error message.

Argus::ICameraProvider *iCameraProvider = Argus::interface_cast<Argus::ICameraProvider>(cameraProvider);

Error is:

(Argus) Error Success: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 211)
(Argus) Error EndOfFile: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 286)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 303)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 314)
(Argus) Error EndOfFile: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 147)
(Argus) Error EndOfFile:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 101)

Is it because I am not using any ISP configuration file (I assumed that default ISP configuration file is enough - I am not worried about fine tuning at the moment)? Should I mention the path to ISP configuration file somewhere?

Any suggestions/hints?

Hi Rajessh,

 I haven't played enough with libargus yet but what I heard on a training about is that it only supports a predefined set of sensors. Which sensor are you using?  

 We have used the ISP but capturing directly with nvcamerasrc, have you tried using it? This would help you to verify that your driver is working correctly.

-David

Hi David,

Thanks for the response. My sensor is IMX219.

As you suggested, I gave a try with nvcamerasrc. It is not working either.

Command used:

gst-launch-1.0 nvcamerasrc num-buffers=200 sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! omxh264enc ! qtmux ! filesink location=test.mp4 -ev

Error;

Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingSocket read error. Camera Daemon stopped functioning.....
gst_nvcamera_open() failed ret=0
ERROR: Pipeline doesn't want to pause.
ERROR: from element /GstPipeline:pipeline0/GstNvCameraSrc:nvcamerasrc0: GStreamer error: state change failed and some element failed to post a proper error message with the reason for the failure.
Additional debug info:
gstbasesrc.c(3354): gst_base_src_start (): /GstPipeline:pipeline0/GstNvCameraSrc:nvcamerasrc0:
Failed to start
Setting pipeline to NULL ...
Freeing pipeline ...

The reason I am not suspecting driver is that the error is thrown even before the app calls v4l2 ioctls like set_fmt, s_ctrl. Also, sensor is working well with v4l2-ctl tool.

Any hints from the nvcamerasrc errot log?

Hi Rajeesh,

RidgeRun offers a driver that is ready to be used for that camera sensor, you can find information and pipelines here:

https://developer.ridgerun.com/wiki/index.php?title=Sony_IMX219_Linux_driver_for_Tegra_X1

If your device tree is not correct it might still work with v4l2 but not with nvcamerasrc. Try a simpler pipeline like:

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=1 ! “video/x-bayer, format=rggb, width=1920, height=1080” ! fakesink -v

or

gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange=“21 21” ! ‘video/x-raw(memory:NVMM), width=(int)3280, height=(int)2464, format=(string)I420, framerate=(fraction)21/1’ ! fakesink -v

Hi Rajeesh,

You can find info about ISP control with the imx219 sensor here:
https://devtalk.nvidia.com/default/topic/975962/jetson-tx1/ridgerun-support-for-six-raspberry-pi-v2-imx219-and-v1-ov5647-cameras-using-isp-multi-camera/2

Hi ACervantes,

Thank you for the info; i checked your wiki page and sure, will spend sometime adjusting wb, contrast and saturation of IMX219 :)

Meanwhile, as an update to the issue (arguslib not working) I posted, our team got it resolved; couple of modification in the dtsi file (never knew position tag in the dtsi was so important!) and minor modification in the driver fixed it.

Thank You and David for the support.

Regards,
Rejeesh