I’m trying to run raw images I have saved through the ISP pipeline using the file component. To load the images in during the NvMediaIPPImgGrpReadCallback in NvMediaIPPFileReaderComponentConfig I followed the ipp_file example and am using NvMediaImagePutBits.
My image storage solution stores the metadata in separate buffers from the image data so I am hoping to avoid an extra copy by using NvMediaImagePutBits with a NvMediaRect to copy the top/bottom metadata and image data directly.
Unfortunately when I use NvMediaRect I get an error from NvMediaImagePutBits with status = 1 NVMEDIA_STATUS_BAD_PARAMETER. It appears that NvMediaImagePutBits has a bounds check on the NvMediaRect for the size of the image which doesn’t include the top and bottom metadata. Is this expected behavior? Is there another way to copy the metadata in? None of the samples use PutBits with an NvMediaRect.
If I use NvMediaRect and only copy up to the size of my image 1920x1008 the image out of the ISP is purple and the bottom 16 lines of the image are cut off. The purple seems to be caused by the missing metadata. If I do an additional copy into a buffer with top + image + bottom and then call NvMediaImagePutBits with no NvMediaRect the images are produced as I would expect.
What is the right way to use NvMediaRect with NvMediaImagePutBits to copy metadata?
Here is some sample code that illustrates the issue:
auto top_metadata = raw_image.top_metadata, bottom_metadata = raw_image.bottom_metadata;
const auto* img_data = raw_image.data();
uint16_t image_width = raw_image.cols();
uint16_t image_height = raw_image.rows();
uint16_t buffer_height = image_height + ((raw_image.top_metadata.size() + raw_image.bottom_metadata.size()) / (image_width * bytesPerPixel));
ImageBuffer buf(buffer_height, image_width);
CopyInto(buf, top_metadata, img_data, bottom_metadata);
// Put Bits of entire buffer with top + image + bottom.
auto* src_full_img_data = buf.data();
status = NvMediaImagePutBits(image_group->imageList[0], NULL /* NvMediaRect */, (void **) &src_full_img_data, &src_img_pitch);
CHECK_EQ(status, NVMEDIA_STATUS_OK) << "NvMediaImagePutBits failed for full img data.";
// Success! Setting NvMediaRect to NULL will copy top and bottom meta to the right location.
// Images out of the ISP look nominal.
// Put Bits of top metadata.
NvMediaRect dst_tmeta_rect = {0, 0, (uint16_t)(image_width/2) /* 960 */, (uint16_t)(top_metadata.size() / (2*image_width)) /* 16 */};
auto* src_tmeta_data = top_metadata.data();
status = NvMediaImagePutBits(image_group->imageList[0], &dst_tmeta_rect, (void **) &src_tmeta_data, &src_img_pitch);
CHECK_EQ(status, NVMEDIA_STATUS_OK) << "NvMediaImagePutBits failed for top metadata.";
// Call succeeds, but puts the metadata in the first 16 rows of the image not the metadata
// section. Shows up as a block of pink pixels in the first 16 rows.
// Put Bits of raw image (clamped to image height)
NvMediaRect dst_img_rect = {0, dst_tmeta_rect.y1 /* 16 */, (uint16_t)(image_width/2), 1008};
const auto* src_img_data = raw_image.image.data();
status = NvMediaImagePutBits(image_group->imageList[0], &dst_img_rect, (void **) &src_img_data, &src_img_pitch);
CHECK_EQ(status, NVMEDIA_STATUS_OK) << "NvMediaImagePutBits failed for raw image.";
// Call succeeds, but the image is purple and the last 16 lines of the image out of the ISP are black.
// Put Bits of raw image
NvMediaRect dst_img_rect = {0, dst_tmeta_rect.y1 /* 16 */, (uint16_t)(image_width/2), (uint16_t)(dst_tmeta_rect.y1 + raw_image.image.rows()) /* 16 + 1008 */};
const auto* src_img_data = raw_image.image.data();
status = NvMediaImagePutBits(image_group->imageList[0], &dst_img_rect, (void **) &src_img_data, &src_img_pitch);
CHECK_EQ(status, NVMEDIA_STATUS_OK) << "NvMediaImagePutBits failed for raw image.";
// Check failed: status == NVMEDIA_STATUS_OK (1 vs. 0) NvMediaImagePutBits failed for raw image.