Getting image data from DepthImageProto

Hello,

I am trying to parse out the image data (pixels) from the ImageProto in DepthCameraProto. I see from reading through code that there is some way of converting an ImageProto to an image view but this is very unclear. Does anyone know how to get at the pixel data from an ImageProto? Particularly the depth data fron the ImageProto in the DepthCameraProto?

Thank you in advance!

Hi,

It seems we are missing this vital information in our documentation, thanks for bringing it up.

I’ll prepare a more complete sample, but assuming you have

ISAAC_PROTO_RX(ColorCameraProto, color_listener);

in your header file, here is how you would read a color image:

#include "engine/gems/image/color.hpp"
#include "engine/gems/image/io.hpp"
#include "engine/gems/image/utils.hpp"
#include "messages/camera.hpp"

// ....
ImageConstView3ub color_image;
FromProto(rx_color_listener().getProto().getImage(), rx_color_listener().buffers(), color_image);
SavePng(color_image, "path_to_file.png");

For depth, please use DepthCameraProto instead of ColorCameraProto and ImageConstView1f instead of ImageConstView3ub.

I’ll prepare a complete sample soon and update the documentation for our next release. Thanks again.

Best,
Oguz

Hello,
I encountered the same problem,but what we want is the data in buffers rather than the image

file.Can you provide a method to get the buffers data in ColorCameraProto?

I try to get the data in this way,auto databuff=rx_color_listener().buffers(); what the type of the

databuff?

Thanks.

ImageConstView3ub class and FromProto() function provides an easy way to parse the data from ColorCameraProto. If you need to access the buffer, you can do so using the data() member function of ImageConstView3ub. Please see engine/core/image/image.hpp for details. Also FromProto() in messages/image.hpp shows how to read buffers from ImageProto.

Please see apps/tutorials/opencv_edge_detection/EdgeDetector.{cpp,hpp}, where we have
ISAAC_PROTO_RX(ColorCameraProto, input_image);
ISAAC_PROTO_TX(ColorCameraProto, output_image);

Hello
I am still confused about how to get the buffer data of ColorCameraProto. I used the data() member function, and then? How can I print the data to the screen for example? I haven’t found the realize of the data() function.

Thank you in advance!

Can you please post some code snippets describing what you are trying to accomplish. The SDK provides several viewer nodes that can be used to view data in Sight and if you have an ImageView after using the FromProto function you should have the ability to manipulate the data.

Without a deeper understanding of what you’re trying to accomplish, we cannot provide additional insight.

Hi,

I think Isaac SDK 2020.2 is the next release you were talking about. Unfortunately I couldn’t really find any information about the image types like ImageConstView1f, ImageConstView3ub, etc whatsoever.
I am trying to save the produced RGB, depth and segmentation images locally as PNG or JPEG, which should be pretty straight forward, considering how “mighty” isaac sdk is and how much it has to offer. Yet it is not possible to do it because I don’t even know what “ImageConstView1f” is supposed to mean. Completing the documentation before releasing a software should be a priority!

The definition of the image types in Isaac SDK 2020.2 is in engine/engine/core/image/image.hpp, but the nested macros may not be easy to follow. (see ISAAC_DECLARE_CPU_IMAGE_TYOES_IMPL(N, T, S)). The naming convention is “Image{ConstView}{1}{f}” = (view meaning the object has a reference to a buffer but does not own it, 1: single channel, f = float, single-precision).

For future reference, to save segmentation images I did:

void DensefusionDatasetGenerator::tick(){

        ImageConstView1ub input_image_segmentation;
        FromProto(rx_input_image_segmentation().getProto(), rx_input_image_segmentation().buffers(), input_image_segmentation);

        const size_t rows = input_image_segmentation.rows();
        const size_t cols = input_image_segmentation.cols();
        cv::Mat image_segmentation = cv::Mat(rows, cols, CV_8U, const_cast<void *>(static_cast<const void *>(input_image_segmentation.data().pointer())));
        
        image_segmentation = image_segmentation * 255;
        cv::imwrite("path/to/save/", image_segmentation);
}