Timeout error with nvmimg_cap using custom driver

Hi,

I modified the following file, created a custom driver and checked with nvmimg_cap,
but a timeout error occurred and I could not get the image.

Could you give me some advice, please?

[modifed files]
drive-t186ref-linux/samples/nvmedia/ext_dev_prgm/drv/isc_ov10640.c
drive-t186ref-linux/samples/nvmedia/ext_dev_prgm/drv/isc_ov10640.h
drive-t186ref-linux/samples/nvmedia/ext_dev_prgm/drv/isc_ov10640_setting.h
drive-t186ref-linux/samples/nvmedia/ext_dev_prgm/drv/isc_max9271.c
drive-t186ref-linux/samples/nvmedia/ext_dev_prgm/drv/isc_max9271.h

drive-t186ref-linux/samples/nvmedia/ext_dev_prgm/img_dev/maxim/ref_max9286_9271_ov10640.c

drive-t186ref-linux/samples/nvmedia/img_cap/drive-px2-b.conf

sensor is not ov10640, and serializer is not max9271.

[execution environment]
connect one camera to Tegra-B
image size is 1936x1114
serializer slave address is 0x6a
sensor slave address is 0x1a

[execution command]
nvmimg_cap -cf drive-px2-b.conf -c dvp-ov10640-raw12-1280x1080-ab -d 0 -w 1 --aggregate 1 -v 3

[part of the execution log]

nvmedia: _GetParamSetID: Param set found (3)
nvmedia: Using params set: dvp-ov10640-raw12-1280x1080-ab for capture
nvmedia: ImgGetDevice: Found the driver for ref_max9286_9271_ov10640
nvmedia: Init: Set image device property
nvmedia: Init: Create root device
***** aurix state *****
 CAM_PWR_ON: 0x1
 TEGRA_GMSL: TEGRA B
 TEGRA_FPDL: TEGRA B
 TEGRA_A_HDMI: MXM
 TEGRA_B_HDMI: MXM
 TEGRA_PCI: TEGRA B
 FAN_STATE: UNKNOWN
***********************
nvmedia isc: Camera(s) are already powered on. Power cycling the camera(s)
***** aurix state *****
 CAM_PWR_ON: 0x0
 TEGRA_GMSL: UNKNOWN
 TEGRA_FPDL: UNKNOWN
 TEGRA_A_HDMI: UNKNOWN
 TEGRA_B_HDMI: UNKNOWN
 TEGRA_PCI: UNKNOWN
 FAN_STATE: UNKNOWN
***********************
***** aurix state *****
 CAM_PWR_ON: 0x1
 TEGRA_GMSL: UNKNOWN
 TEGRA_FPDL: UNKNOWN
 TEGRA_A_HDMI: UNKNOWN
 TEGRA_B_HDMI: UNKNOWN
 TEGRA_PCI: UNKNOWN
 FAN_STATE: UNKNOWN
***********************
nvmedia: Init: Create aggregator device on address 0x6a
nvmedia: Init: Create broadcast serializer device on address 0x62
nvmedia: ImgDevGetModuleConfig: moduleCfgName: ref_ov10640
nvmedia: Init: Create broadcast sensor device on address 0x1a
nvmedia: SetupConfigLink: Disable forward/reverse channel
nvmedia: SetupConfigLink: Check the aggregator is present
nvmedia: SetupConfigLink: Power up reverse channel transmitter
nvmedia: SetupConfigLink: Set aggregator device defaults
nvmedia: SetupConfigLink: Enable all of reverse channels
nvmedia: SetupConfigLink: Enable Deserializer reverse channel
nvmedia: SetupConfigLink: Set Des reverse channel amplitude
nvmedia: SetupConfigLink: Enable reverse channel
nvmedia: SetupConfigLink: Set serielizer device defaults
nvmedia: SetupConfigLink: Disable csi out
nvmedia: SetupConfigLink: Set aggregator's data type
nvmedia: SetupConfigLink: Set aggregator's fsync mode
nvmedia: SetupConfigLink: Enabling 4 links: 0, 1, 2 and 3
nvmedia: SetupConfigLink: Set serializer input mode
nvmedia: SetupConfigLink: Enable aggregator auto ack
nvmedia: SetupVideoLink: Disable reverse channel
nvmedia: SetupVideoLink: Enable reverse channel. id = 0
nvmedia: SetupVideoLink: Check sensor is present
nvmedia: SetupVideoLink: Enable all of reverse channels
nvmedia: SetupVideoLink: Set OV10640 defaults
nvmedia: SetupVideoLink: Enable aggregator's auto ack
nvmedia: SetupVideoLink: Enabling link: 0
nvmedia: SetupVideoLink: Enable serial link
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: CaptureInit: Capture Input Queue 0: 1936x1114, images: 5 
nvmedia: Start: Enable streaming
nvmedia: Start: Get Link(0) Status
nvmedia: Start: Video Link(0) is detected!
nvmedia: Start: Enable csi out
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: SaveInit: Save Conversion Queue 0: 968x557, images: 3 
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: _CreateImageQueue: NvMediaImageCreateNew
nvmedia: CompositeInit: Composite Queue: 968x557, images: 3 
Used display - Display Info - Enabled:1 displayId:0 type:B306 size:1920x1080 refresh:60.000000 type:HDMI
Type "h" for a list of options
-
nvmedia: _SaveThreadFunc: saveThread input queue 0 is empty
nvmedia: _CompositeThreadFunc: Waiting for input image from queue 0
nvmedia: _DisplayThreadFunc: Display input queue empty

I checked in img_cap, and in the _CaptureThreadFunc function in capture.c,
the return values of NvQueueGet and NvMediaICPFeedFrame function were also NVMEDIA_STATUS_OK.

However, it occured a timeout error in NvMediaICPGetFrameEx function.

  1. When the above issue faced, please let me know what are the issues can be considered?

  2. What is the difference between the NvMediaICPFeedFrame function and the NvMediaICPGetFrameEx function?

Regards,
nissy

Dear Naoki.Nishimura,
Could you please share the changes in the each file and steps to reproduce the issue on ourside.
Can you check the API documentation to know the difference.
https://docs.nvidia.com/drive/nvvib_docs/NVIDIA%20DRIVE%20Linux%20SDK%20Development%20Guide/baggage/group__image__capture__api.html#gaf767eba4b6136e9df54de656d3dc1eec
https://docs.nvidia.com/drive/nvvib_docs/NVIDIA%20DRIVE%20Linux%20SDK%20Development%20Guide/baggage/group__image__capture__api.html#ga34be0e6e95a9058f1769797145b3a910

Dear SivaRamaKrishna,

Thanks for your reply.

Could you please share the changes in the each file and steps to reproduce the issue on ourside.

Yes, it is possible to share them.
However, sensor and serializer are not commercially available.

They are Sony’s sensor and Maxim’s serializer,

and the main change is the setting value from the reference device(ov10640 and MAX9271).

Should we shared them even in the above situation?

Can you check the API documentation to know the difference.

In my recognition, If NVMEDIA_STATUS_OK returns with NvMediaICPFeedFrame function,
the image is stored in the image capture pool.
If images are stored in the image capture pool, we can get the image with NvMediaICPGetFrameEx function.
Is the above recognition correct? (But NvMediaICPGetFrameEx returns NVMEDIA_STATUS_TIMED_OUT)

Regards,
nissy

Dear Naoki.Nishimura,
Is your objective is same as https://devtalk.nvidia.com/default/topic/1038498/drive-hardware/connect-max9271-serializer-with-nvidia-drive-px2/? If so, lets follow it in that thread.

Dear SivaRamaKrishna,

Although I did not know the cause of the issue yet, the issue was resolved.

As measures taken , we changed the initialization sequence of Ser/Des and sensor setting values.

Thank you for your support.

Regards,
nissy