Hi, i am trying to get decode a jpeg on a jetson agx orin to an uncompressed ros image:
i am currently failing, to convert it to cv::mat
uint32_t width, height, pixfmt;
NvBuffer *buffer;
uint64_t in_buf_size = msg->data.size();
unsigned char* in_buf = new unsigned char[in_buf_size];
std:memcpy(in_buf, msg->data.data(), in_buf_size);
int status = jpegdec->decodeToBuffer(&buffer, in_buf, in_buf_size, &pixfmt ,&width, &height);
i would really appreciate it if someone could help
hello again, this does not help at all
is this some kind of new bot?
so my current state is:
// Decode the JPEG image
int fd = 0;
uint32_t width, height, pixfmt;
NvBuffer *buffer;
uint64_t in_buf_size = msg->data.size();
unsigned char* in_buf = new unsigned char[in_buf_size];
std:memcpy(in_buf, msg->data.data(), in_buf_size);
int status = jpegdec->decodeToFd(fd, in_buf, in_buf_size, pixfmt, width, height);
if(status){
RCLCPP_ERROR(rclcpp::get_logger("ImageResize"), "Failed to decode");
}
NvBufSurface *nvbuf_surf = 0;
status = NvBufSurfaceFromFd(fd, (void**)(&nvbuf_surf));
unsigned int plane;
//NvBufSurfaceMap(nvbuf_surf, 0, plane, NVBUF_MAP_READ_WRITE);
status = NvBufSurfaceMap(nvbuf_surf, -1, -1, NVBUF_MAP_READ);
status = NvBufSurfaceSyncForCpu(nvbuf_surf, -1, -1);
void* data_ptr = nvbuf_surf->surfaceList[0].mappedAddr.addr[0];
cv::Mat image = cv::Mat(height, width, CV_8UC4, data_ptr);
but is doesnt work
it is very confusing, due to the api change
Hi,
Please refer to the patch:
How to create OpenCV cv::Mat from NvBuffer in Jetpack 5.1 - #8 by DaveYYY
You would need to allocate a NvBufsurface in RGBA, convert decoded data to RGBA and map to cv::Mat. Please check the patch and try.
thanks @DaneLLL
i am trying to get this running, but am still a litte bit confused
so i have to get my file descriptor (FD) to a NvBufferSurface, then transform it and than i can map it to cv?
Hi,
Yes, please allocate another NvBufSurface in RGBA, convert decoded FD to the buffer throughNvBufSurfTransform(), and map to cv::Mat.
thanks, it works now kinda
but if i run it in a loop, the memory wont refresh and i have always the same decoded image
do i have to more than the following to clean up my memory?
i call the code below in a loop @DaneLLL
in_buf = nullptr;
in_buf = new unsigned char[in_buf_size];
std:memcpy(in_buf, msg->data.data(), in_buf_size);
status = jpegdec->decodeToFd(fd, in_buf, in_buf_size, pixfmt, width, height);
if(status){
RCLCPP_ERROR(rclcpp::get_logger("ImageResize"), "Failed to decode");
}
delete[] in_buf;
dstCompRect[0].top = 0;
dstCompRect[0].left = 0;
dstCompRect[0].width = width;
dstCompRect[0].height = height;
input_params = {0};
input_params.width = width;
input_params.height = height;
input_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
input_params.layout = NVBUF_LAYOUT_PITCH;
input_params.memType = NVBUF_MEM_SURFACE_ARRAY;
m_compositedFrame = 0;
if (NvBufSurf::NvAllocate(&input_params, 1, &m_compositedFrame) != 0) {
RCLCPP_ERROR(rclcpp::get_logger("ImageResize"), "Failed to allocate composite buffer");
}
if (NvBufSurfaceFromFd(m_compositedFrame, reinterpret_cast<void**>(&pdstSurf)) != 0) {
RCLCPP_ERROR(rclcpp::get_logger("ImageResize"), "Failed to map buffer to NvBufSurface");
}
memset(&m_compositeParam, 0, sizeof(m_compositeParam)); // set all element to zero
m_compositeParam.params.composite_blend_flag = NVBUFSURF_TRANSFORM_COMPOSITE;
m_compositeParam.params.input_buf_count = 1;
m_compositeParam.params.composite_blend_filter = NvBufSurfTransformInter_Algo3;
m_compositeParam.dst_comp_rect = static_cast<NvBufSurfTransformRect*>
(malloc(sizeof(NvBufSurfTransformRect) * 1));
m_compositeParam.src_comp_rect = static_cast<NvBufSurfTransformRect*>
(malloc(sizeof(NvBufSurfTransformRect)
* m_compositeParam.params.input_buf_count));
memcpy(m_compositeParam.dst_comp_rect, &dstCompRect[0],
sizeof(NvBufSurfTransformRect) * 1);
m_compositeParam.src_comp_rect[0].top = 0;
m_compositeParam.src_comp_rect[0].left = 0;
m_compositeParam.src_comp_rect[0].width = width;
m_compositeParam.src_comp_rect[0].height = height;
batch_surf = 0;
batch_surf = new NvBufSurface*[1];
//batch_surf[0] = NULL;
m_dmabufs[0] = 0;
memset(m_dmabufs, 0, sizeof(m_dmabufs));
if(NvBufSurfaceFromFd(fd,(void**)(&batch_surf[0])) != 0){
RCLCPP_ERROR(rclcpp::get_logger("ImageResize"), "Failed");
}
if(NvBufSurfTransformMultiInputBufCompositeBlend(batch_surf, pdstSurf, &m_compositeParam) !=0)
{
RCLCPP_ERROR(rclcpp::get_logger("ImageResize"), "Failed Transform");
}
NvBufSurfaceMap(pdstSurf, -1, 0, NVBUF_MAP_READ);
NvBufSurfaceSyncForCpu(pdstSurf, -1, 0);
cv::Mat image(height, width, CV_8UC4, pdstSurf->surfaceList[0].mappedAddr.addr[0], pdstSurf->surfaceList[0].pitch);
NvBufSurfaceSyncForCpu(pdstSurf, -1, 0);
NvBufSurfaceSyncForDevice(pdstSurf, -1, 0);
if(NvBufSurfaceUnMap(pdstSurf, -1, 0) != 0)
{
RCLCPP_ERROR(rclcpp::get_logger("ImageResize"), "Failed to Unmpa");
}
NvBufSurfaceDestroy(pdstSurf);
delete[] m_compositeParam.dst_comp_rect;
delete[] m_compositeParam.src_comp_rect;
m_compositedFrame = 0;
Hi,
Do you use Jetpack 6.2 or 6.1? Would like to know whether you are using latest release, or using which version.
i think it is jetpack 6.0 since i have to use cuda 12.2
@DaneLLL
Hi,
It sounds to be a known issue on certain Jetpack 5 releases:
Jetson/L4T/r35.5.x patches - eLinux.org
[MMAPI/gstramer] NvJpegDecoder/nvjpegdec does not work properly
It shoudl work fine if you use Jetpack 6. Please check again. If the issue still persists, please share a patch to 06 sample. So that we can set up developer kit and check.
hi, it is L4T 35, Revision 4.1 with 5.1.2-b104 on the native system
i use a docker image with jetpack 6.0
i just tested the following which give two independent images, so the problem is likely my code:
./jpeg_decode num_files 2 ../../data/Picture/nvidia-logo.jpg ../../data/Picture/nvidia-logo.yuv ../../data/Picture/Drawing.jpeg ../../data/Picture/drawing.yuv
i am going to create the patch to 06 sample
Hi,
Please upgrade your native system to Jetpack 5.1.4 and try.
i am not allowed to upgrade my native system
Hi,
Since the issue is known on Jetpack 5.1.2. If you are not able to upgrade, would suggest use software JPEG decoder.
hi @DaneLLL
thanks for your help
would i be able to use the decodeToBuffer function instead of decodeToFD to get this working?
it seems unfortunately, like the Buffer API deprecated
Hi,
On Jetpack 5.1.2, both decodeToBuffer() and decodeToFd() do not work properly in MJPEG decoding.
strange decodeToBuffer() seems to be working but decodeToFd() not
Hi,
Thanks for the information. So on Jetpack 5.1.2, you can call decodeToBuffer(). Here NvBuffer is defined in
/usr/src/jetson_multimedia_api/samples/common/classes/NvBuffer.cpp
It is not the deprecated NvBuffer APIs.
nvm
seems the decodeToBuffer() only works sometime, guess i have to manually decode
thanks for your responses @DaneLLL
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.