Sending ImageProto from python codelet

I am developing a machine learning application (in python) which requires me to send/receive messages of the ImageProto type. The ImageProto protobuffer has the field “dataBufferIndex”. After a bit of digging, I found out how to use this field to read a buffer, using self.rx_hook.get_buffer_content(index), which allows me to receive ImageProto messages. However, I was unable to find any way of setting the buffer contents for transmitting. I know I could probably get something to work using the TensorProto, but then I have no way of sending results to a ColorCameraViewer. Am I missing something?

Hi, sorry to disappoint but I don’t have an answer; I’m looking for advice.

I am trying to do something similar; I am working with the ROS-Isaac bridge and am trying to pass image data out of Isaac and into Ros via the bridge.

Could you tell me a little bit about how you accessed the data buffer?

I have a node hooked into the left_camera_subscriber receiving ColorCameraProto messages which I am able to deconstruct, but (as i’m sure you have noticed) the messages don’t actually contain the image data.

If I can gain access to the image data I should be able to convert the data into a sensor_msgs::Image message type for ROS.

If you think I can help you in any way, I’d be happy to return the favour.


I’ve been using as a reference for how the Isaaac messages are set up. The ColorCameraProto contains an ImageProto in the image field. This ImageProta has a field called dataBufferIndex. This tells you which buffer to look in for the image data. I’ve been reading this data into a numpy array. So to get the image data I do something like this:

image_data = np.frombuffer(rx_color_camera_hook.get_buffer_content(color_camera_proto.image.dataBufferIndex))

Hope that helps!

Hi, @ngilbert,

Sorry we don’t have the support in PyCodelet for publishing messages with buffers yet. Coding a C++ codelet might help before the feature is there.

How can we do this in c++ Codelet ? Can you provide a sample ?

Check packages/dummies/ImageLoader.cpp for sample.

In my packages/dummies/ directory there is just BUILD file which loads the shared object.

Maybe packages/realsense has what you need.


Here is a helpful breakdown of Protos in general ([)

Hi, is there any update on if it’s possible to use PyCodelets for publishing messages with buffers now? I want to use my PyCodelet for getting an image from a camera and then applying my image processing algorithms. I then want to publish the image for viewing on WebSight. Thanks!

Hi, @adhamija,

With 2020.1 release it is possible to publish message with buffers now. Please check the unit tests in that comes with the release.

1 Like