YUV 420 Semi-planar block linear format to JPEG encoding

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after clicking “+ Create Topic”):
Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.6.0.8170
1.5.1.7815
1.5.0.7774
other

Host Machine Version
native Ubuntu 18.04
other

Hi,

I’m basically combining the img_jpegenc and nvsipl_camera samples to be able to log jpeg images directly after the ISP (to avoid saving to .yuv and then reading to encode).

For SF3325, I have checked the surface format attributes of the NvMediaImage that is obtained from ISP0, in order to create the jpeg encoder object accordingly. However the following configuration gives “Invalid input format specified” error during NvMediaIJPECreate.

NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
NVM_SURF_FMT_SET_ATTR_YUV(surfFormatAttrs,YUV,420,SEMI_PLANAR,UINT,8,BL);
inputSurfType = NvMediaSurfaceFormatGetType(surfFormatAttrs, NVM_SURF_FMT_ATTR_MAX);
	    testEncoder = NvMediaIJPECreate(device,
	                                   inputSurfType,               // inputFormat
	                                   maxOutputBuffering,          // maxOutputBuffering
	                                   MAX_BITSTREAM_SIZE);              // maxBitstreamBytes

I would like to know if the above configuration is supported by the jpeg encoder. I know that changing block linear (BL) to (PL) allows the encoder object to be initialized, however the encoding is faulty since thats not what ISP0 gives as output.

Thanks

2

Hi @goksani ,

This forum is not for QNX release.
Back to your question. Yes, NvMediaIJPECreate() doesn’t support BL.

3

Thanks for your answer.

May I ask which forum is for QNX then? I don’t think it changes anything NvMediaIJPE library though.

4

Dear @goksani,
Yes. It is same

May I ask which forum is for QNX then?

Could you check with your NVIDIA representative for access to QNX forum.

5

Ok thanks.

As a follow up, I want to clarify something.

Since the NvMediaIJPECreate does not support block-linear format, does the ReadImage function in the img_jpegenc sample convert the block-linear YUV frame read from a file, into a pitch-linear format on the CPU first, and then feed into the encoder?

If so, is there a more efficient way of handling block-linear frames without going back and forth to the CPU?

6

You need to override the surface type returned by the SIPL GetImageAttributes API called in AllocateAndRegisterBuffers() (defined in ~/nvidia/nvidia_sdk/DRIVE_OS_5.2.0_SDK_Linux_OS_DDPX/DRIVEOS/drive-t186ref-linux/samples/nvmedia/nvsipl/test/camera/CNvSIPLMaster.hpp) to get the pitch linear output. Thanks.

7

Thanks for your suggestion,

However, overriding the output of GetImageAttributes to pitch linear surface type, before calling RegisterImages inside CNvSIPLMaster.hpp, leads to the following error in runtime:

[CheckImageInfo:202] [CheckImageInfo:202] Bad input parameter : Output Color STD & SURF Type
[NvMediaCamProcess:132] [NvMediaCamProcess:132] Bad input parameter
libnvsipl.so: CNvMISPBlock.cpp: 574: ProcessISP: NvMediaCamProcess failed. nvmStatus:1
libnvsipl.so: CNvMISPBlock.cpp: 487: ThreadFunc: ProcessISP failed. nvmStatus:1
nvsipl_camera: ERROR: Pipeline: 12, NOTIF_ERROR_INTERNAL_FAILURE

The code snippet where surface type override is being made in CNvSIPLMaster.hpp can be seen below:

        if (isp0Enabled) {
            // Get attributes
            NvSIPLImageAttr attr;
            status = m_pCamera->GetImageAttributes(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP0, attr);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("GetImageAttributes failed\n");
                return status;
            }
            // GISIL
            // Overriding surface type to make it pitch linear
            NVM_SURF_FMT_DEFINE_ATTR(surfFormatAttrs);
            NVM_SURF_FMT_SET_ATTR_YUV(surfFormatAttrs,YUV,420,SEMI_PLANAR,UINT,8,PL); // generate a surface format attribute that matches the ISP Output, BUT in Pitch Linear format instead        
            attr.surfaceType = NvMediaSurfaceFormatGetType(surfFormatAttrs, NVM_SURF_FMT_ATTR_MAX);
            // END OF GISIL
            std::vector<NvMediaImage*> images;
            status = AllocateImages(pDevice, attr.surfaceType, attr.surfaceAllocAttr, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_ERR("AllocateImageGroups failed\n");
                return status;
            }
            status = m_pCamera->RegisterImages(pip, INvSIPLClient::ConsumerDesc::OutputType::ISP0, images);
            if (status != NVSIPL_STATUS_OK) {
                LOG_WARN("RegisterImageGroups failed\n");
                DeleteImages(images);
                return status;
            }
            m_imagesList.push_back(images);
        }```
8

Was this messages from DRIVE OS 5.2.0?

9

Its from 5.1.9.0.

10

DRIVE OS 5.1.9.0 isn’t a devzone release (not available in NVIDIA DRIVE Downloads | NVIDIA Developer).
Please try it on DRIVE OS 5.2.0 (I didn’t see the problem on it). Thanks.