I am developing a machine learning application (in python) which requires me to send/receive messages of the ImageProto type. The ImageProto protobuffer has the field “dataBufferIndex”. After a bit of digging, I found out how to use this field to read a buffer, using self.rx_hook.get_buffer_content(index), which allows me to receive ImageProto messages. However, I was unable to find any way of setting the buffer contents for transmitting. I know I could probably get something to work using the TensorProto, but then I have no way of sending results to a ColorCameraViewer. Am I missing something?
Hi, sorry to disappoint but I don’t have an answer; I’m looking for advice.
I am trying to do something similar; I am working with the ROS-Isaac bridge and am trying to pass image data out of Isaac and into Ros via the bridge.
Could you tell me a little bit about how you accessed the data buffer?
I have a node hooked into the left_camera_subscriber receiving ColorCameraProto messages which I am able to deconstruct, but (as i’m sure you have noticed) the messages don’t actually contain the image data.
If I can gain access to the image data I should be able to convert the data into a sensor_msgs::Image message type for ROS.
If you think I can help you in any way, I’d be happy to return the favour.
I’ve been using https://docs.nvidia.com/isaac/isaac/doc/message_api.html as a reference for how the Isaaac messages are set up. The ColorCameraProto contains an ImageProto in the image field. This ImageProta has a field called dataBufferIndex. This tells you which buffer to look in for the image data. I’ve been reading this data into a numpy array. So to get the image data I do something like this:
image_data = np.frombuffer(rx_color_camera_hook.get_buffer_content(color_camera_proto.image.dataBufferIndex))
Hope that helps!
Sorry we don’t have the support in PyCodelet for publishing messages with buffers yet. Coding a C++ codelet might help before the feature is there.
How can we do this in c++ Codelet ? Can you provide a sample ?
Check packages/dummies/ImageLoader.cpp for sample.
In my packages/dummies/ directory there is just BUILD file which loads the libdummies_module.so shared object.
Maybe packages/realsense has what you need.
Here is a helpful breakdown of Protos in general (https://devtalk.nvidia.com/default/topic/1064465/sdk/reading-content-of-detections2proto-message/post/5390513/#reply[)
Hi, is there any update on if it’s possible to use PyCodelets for publishing messages with buffers now? I want to use my PyCodelet for getting an image from a camera and then applying my image processing algorithms. I then want to publish the image for viewing on WebSight. Thanks!
With 2020.1 release it is possible to publish message with buffers now. Please check the unit tests in message_test.py that comes with the release.