Using nvmedia to do image process

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
[√] DRIVE OS 6.0.4 SDK
other

Target Operating System
[√] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
[√] DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
1.9.2.10884
[√] other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
[√] other

I am using the NvsciBuf-related APIs in the DriveOS SDK for image processing. For example, I first use the Nvmedia2D APIs to convert the YUV422 image to YUV420, then use the NvmediaLDC APIs to do some distortion correction, and finally use the NvmediaJPEGEncode APIs to compress the image into JPEG format.

My question is whether I need to call the NvSciSync-related APIs to wait for each operation to complete. I found that if I don’t use NvSciSync, the compressed image will be from the previous frame. However, using NvSciSync will significantly increase the processing time (reaching 60-70ms in the case of multiple cameras running in parallel), which is unacceptable.

Could you please help me with this question?

2

Dear @zhixin.zhou,
So application pipeline is capture image->ISP->Nvmedia2D->NvmediaLDC->NvmediaJpegEncode.
So, may I know where are you keeping nvscisync calls in pipeline and which API calls?
Could you share some perf stats like pipeline time for single/multiple camera with(with out) NvSciSync calls. Generally, I would expect ~20 msec for capture + ISP processing in pipeline for 4 cameras and also synchronization calls needed as one engine has to wait for output from another.

3

I am directly obtaining the image from the ICP because the ISP has already been done at the camera end.So the pipeline is capture image from ICP->Nvmedia2D(yuyv->yuv420)->NvmediaLDC->Nvmedia2D(crop or rotate)->NvmediaJpegEncode.
I have shown in the attachment how I call the nvscisync APIs in my code.

code sample.txt (6.9 KB)
perf stats: 10ms for one single pipeline,If 7 cameras are used in parallel, then each camera will take 30-50ms to process.

4

@SivaRamaKrishnaNV Can you help answer the question?

8

Dear @zhixin.zhou,
My apologies for missing this topic.
Do you still need support on this issue?

9

@ SivaRamaKrishnaNV yes，the time cost of the image process pipeline(ICP->Nvmedia2D(yuyv->yuv420)->NvmediaLDC->Nvmedia2D(crop or rotate)->NvmediaJpegEncode) is too long. How can I shorten this time?