DeepStream: How to save decoded frame in .jpg format (or OpenCV Mat) in decoder(decPerf) pipeline

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Dear DeepStream community,

I would like to access the decoded frame. Following is my query

  1. After decoding is done, I want the frame to be copied to OpenCV Mat and perform further analysis on Mat data

Can anyone please help me how to achieve this?

Thanks,
Siddartha

2

Hi,

Please remember to apply a color space conversion before translating image into cv::Mat.
Flow: Decode ▶ Color Converter ▶ OpenCV

After conversion, data is located at vpInputTensors[0]->getConstCpuData().
Then, you can create cvMat from the data pointer:

const float *data = reinterpret_cast<const float*>(vpInputTensors[0]->getConstCpuData());
...
cv::Mat mat(height, width, 16, data);

Thanks.

3

Thank you for the reply.

I have tried the following:

Flow: Decode ▶ Color Converter ▶ OpenCV
Have added the custom module where after addColorSpaceConvertorTask to BGR_PLANAR type, I create Mat object in execute() API. But what I get is the corrupted frame when I save.

Below is the snippet:

IDeviceWorker *pDeviceWorker = createDeviceWorker(g_nChannels, g_devID_infer);
pDeviceWorker->addDecodeTask(cudaVideoCodec_H264);
IModule *pConvertor = pDeviceWorker->addColorSpaceConvertorTask(BGR_PLANAR);

PRE_MODULE_LIST module1;
module1.push_back(std::make_pair(pConvertor, 0));
OpencvModule *opencv = new OpencvModule(module1, …);
assert(nullptr != opencv);
pDeviceWorker->addCustomerTask(opencv);
…
…
void OpencvModule::execute(const ModuleContext& context, const std::vector<IStreamTensor *>& vpInputTensors, const std::vector<IStreamTensor *>& vpOutputTensors)
{
assert(1 == vpInputTensors.size());

cout<<vpInputTensors[0]->getTensorType()<<endl;   //  0
cout<<vpInputTensors[0]->getMemoryType()<<endl;   //  0
cout<<vpInputTensors[0]->getElemSize()<<endl;     //  4
cout<<shape_0[0]<<endl; // 1
cout<<shape_0[1]<<endl; // 3
cout<<shape_0[2]<<endl; // 720
cout<<shape_0[3]<<endl; // 1280

const float *pCov = reinterpret_cast<const float*>(vpInputTensors[0]->getConstCpuData());
cv::Mat image(shape_0[2], shape_0[3], 16, &pCov);        
cv::imwrite("img1.jpg",image);

}

Can you please help me to get the correct image.
Please find attached output image

img1.jpg
img1.jpg1280×720 700 KB

img1.jpg
img1.jpg1280×720 700 KB

4

Dear DeepStream community,

Can anyone help me to resolve the above issue?

Thanks,
Siddartha

5

Hi,

You may also need a data format converter.

addColorSpaceConvertor: image format is CHW.
Ex. b1, b2, …, g1, g2, …, r1, r2, …

OpenCV: image format is HWC.
Ex. b1, g1, r1, b2, g2, r2, …

Thanks.

6

Is decPerf available in deepstream sdk for jetson Xavier?
I am not able to locate source code in samples or source folders.

7

Hi,

Sorry for the late.

The module changes location. You can enable the perf with the configure like this:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

The source code is located at ${deepstream_sdk_on_jetson}/sources/apps/apps-common/src/deepstream_perf.c.

Thanks.