NvSciSync / NvSciBuf between SIPL & NvMedia (IEP)

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
DRIVE OS 6.0.10.0
DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description
Hi all, I am hoping to understand the steps & requirements for setting up the buffer & synchronization mechanism between passing the SIPL buffers to IEP for encoding. Both SIPL (ISP output only) & IEP are running in the same process.

NvSciBuf
Do I need to follow the steps in how-to-shared-scibufobj-between-two-function-engines to create the NvSciBufObj? It seems like it is possible to just pass the SIPL buffer pointer into NvMediaIEPFeedFrame (see pseudo code below).

  • The SDK API also strongly recommended that the NvSciBufObj be registered with NvMediaIEPRegisterNvSciBufObj(), does this need to be done, if yes do I only need to register it once with the first SIPL buffer received?
INvSIPLClient::INvSIPLNvMBuffer* nvm_buf; // buffer from SIPL
NvSciBufObj buf_ptr = nvm_buf->GetNvSciBufImage();
NvMediaIEPFeedFrame(encoder_context_, 
                    buf_ptr,
                    &encode_pic_params_H265_, 
                    NVMEDIA_ENCODER_INSTANCE_0);

NvSciSync
Pre-fence
For synchronization of the SIPL ISP output into IEP, is it possible to use the EOF from SIPL Buffer as a pre-fence for NVMediaIEPFeedFrame? Similar to what was suggested in how-to-shorten-the-image-process-time.

  • I also see that the encoder sample app has a preSyncObj (of type NvSciSyncObj). Is this needed because the sample app is retrieving the image from a queue and putting it into a buffer?
NvSciSyncFence fence = NvSciSyncFenceInitializer;
auto status = nvm_buf->GetEOFNvSciSyncFence(&fence);
auto status = NvMediaIEPInsertPreNvSciSyncFence(encoder_context_, fence);
/*** Then call NvMediaIEPFeedFrame() ***/

End-fence
I think that an EOF is also needed to signify that NvMediaIEPFeedFrame() has completed, similar to the IEP encoder sample app. Does the pseudo code look correct?

  • For creating the NvSciSync for IEP, can it be independent of SIPL since it is only signalling the EOF for IEP feed frame, or do I need to setup the NvSciSync list with both SIPL & IEP, and then reconcile it? If I need to set it up with SIPL & IEP, do I just call NvMediaIEPFillNvSciSyncAttrList and INvSIPLCamera::FillNvSciSyncAttrList to set up the attribute list before reconciling?
// Setup the sync object attributes.
NvSciSyncAttrListCreate()
NvSciSyncAttrListSetAttrs()
NvSciSyncAttrListReconcile()
NvSciSyncObjAlloc() // Alloc for eofSyncObj.

NvSciSyncFence eof_fence = NvSciSyncFenceInitializer;
status = NvMediaIEPRegisterNvSciSyncObj(encoder_context_, NVMEDIA_EOFSYNCOBJ, eofSyncObj);
status = NvMediaIEPSetNvSciSyncObjforEOF(encoder_context_, eofSyncObj)
status = NvMediaIEPGetEOFNvSciSyncFence(encoder_context_, eofSyncObj, &eof_fence);
/*** Call NvMediaIEPFeedFrame() ***/
status = NvMediaIEPGetEOFNvSciSyncFence(encoder_context_, eofSyncObj, &eof_fence);

Thanks in advance for the help!

3

Dear @samuel.ongzx ,
Firstly, your use case seems like, you are looking to encode the live camera data. We have recorder tool in DW which serve the purpose. Did you check it?

4

Hi @SivaRamaKrishnaNV, thanks for the response!

Is the Recorder Tool source code open source? If yea, could you point me to where it is in the driveworks SDK? Additionally, encoding & logging live images is just one of the function for the application that we are building, so using just the Recorder Tool would not work for us.

5

@SivaRamaKrishnaNV any updates? I’m hoping to get clarification and advice on best practices to synchronize the the buffer across SIPL & IEP, and whether NvSciBuf needs to be set up in a different way for SIPL & IEP to work together. Thanks in advance!

6

I think I got a working solution, but it would be helpful if @SivaRamaKrishnaNV could help confirm.

NvSciBuf
NvMediaIEPRegisterNvSciBufObj() seems to be optional if SIPL & IEP are running in the same application. My guess is that the ISP produced outputs are already in the format that IEP would accept, so no prior registration or reconciliation of attributes are needed.

NvSciSync
Pre-fence
The pre-fence for IEP encoding in the same application can be skipped as long as we wait for the expiry of the eof fence provided in the SIPL buffer INvSIPLBuffer. Since we ensure that ISP have completed using the buffer, we can just call IEPFeedFrame without a fence unless there is a prior operation on the buffer.

Eof-fence
This is necessary and can be used to notify that IEPFeedFrame has completed which would allow the SIPL ISP buffer to be freed up for the next frame.

Hopefully this is useful for others.

7

It is also recommended to call this API though it is optional(in same application case). If app is not registering, then video driver will register these buffers during runtime in NvMediaIEPFeedFrame() call

Wait time of fence in IEP is zero if the app has already waited for the expiry of the eof frame of SIPL. So adding prefence for IEP will not impact the perf.

Yes.

1 Like
8

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.