Reading Content of Detections2Proto Message

Hi there,

I’m trying to get the yolo_detections.subgraph to send data to a custom node. My understanding is that I can receive detection data using Detections2Proto messages. From the detections.capnp file in isaacSDK/messages:

A message containing detections made by sensor(s)

Each detection has a bounding box, label, and confidence

struct Detections2Proto {

List of predictions made

predictions @0: List(PredictionProto);

List of bounding boxes where we detected objects

boundingBoxes @1: List(RectangleProto);
}

I’d like to access the predictions @0 part of the message and print the result to the console.

I’ve set everything up correctly and my program compiles when I don’t try to access the data. My issue comes up in accessing the data coming through Detections2Proto. Here are my header and cpp files:

#pragma once
#include “engine/alice/alice_codelet.hpp”
#include “messages/messages.hpp”

class SampleList : public isaac::alice::Codelet {
public:
void start() override;
void tick() override;

ISAAC_PROTO_RX(Detections2Proto, samples);
ISAAC_PARAM(int, count, 3);
};

ISAAC_ALICE_REGISTER_CODELET(SampleList);

#include “SampleList.hpp”
#include

void SampleList::start() {
tickOnMessage(rx_samples());

}
void SampleList::tick() {
auto proto = rx_samples().getProto();
const std::string message = proto[0];
std::printf(message);
}

The error message I get is:

apps/samples/realsense_camera/SampleList.cpp:11:36: error: no match for ‘operator’ (operand types are ‘Detections2Proto::Reader’ and ‘int’)
const std::string message = proto[0]
^
How do I access Detections2Proto::Reader to get the predictions @0: List(PredictionProto) info. I’m new to C++ so any help is appreciated. Thanks!!!

Hi,
when calling the getProto() function, you are getting the Reader of the Detection2Proto, as your error message states.
Once you have the Reader, you should use the functions it provides to access the elements in the Proto. Check the cap’n’proto documentations for more infos(https://capnproto.org/cxx.html).

In detail, you should use the getPredictions() to have access to the List(PredictionProto).
Once you have the list, you can access the elements with the operator.
Note that using the operator will give you the Reader of the PredictionProto. Again you should use the method getLabel() to get the label in the PredictionProto.
The code should look like this

auto detections_list = rx_samples().getProto().getPredictions();
auto detection = detections_list[0];
std::string label = detection.getLabel();
LOG_INFO("Label: %s", label.c_str());

Hope this helps.
Cheers

1 Like

Hi d.desimone,

Thank you so much for your reply. This is exactly what I needed to get the code working!!! And thanks for the general insight into proto readers.

From the link you sent me, there should be a .cpp file in Isaac that defines all of the functions for the reader of a given Proto. For instance, where can I find the .cpp file that lists all of the functions of the Detections2Proto Reader? I searched for PredictionProto on my local machine and that did not return any results.

Thanks!!

Hi excit0n,
from the page I linked:

For every field bar defined in Foo, Foo::Reader has a method getBar(). For primitive types, get just returns the type, but for structs, lists, and blobs, it returns a Reader for the type.

This means that for a Detection2Proto::Reader you have the methods

getPredictions(); //returns a List(PredictionProto)
getBoundingBoxes(); // returns a List(RectangleProto)

Accessing an element of the list with the operator gives you a Reader for the element in the list

auto detections_list = rx_samples().getProto().getPredictions(); //returns a List(PredictionProto)
auto detection = detections_list[0]; //returns a PredictionProto::Reader

Now you have a new Reader, the PredictionProto::Reader. Just like the previous one you have a get() method for each element in the Struct, i.e.

getLabel(); //returns a Text (i.e. a string)
getConfidence(); // returns a Float64

So basically to access the elements in a Proto, all you need to know are the elements contained in the Struct (https://docs.nvidia.com/isaac/isaac/doc/message_api.html#protobuffers).
Then the reader will provide you a get() method to access them (just be careful with the syntax and the upper/lower cases). You have to use those methods to “dig” into the nested structures until you get a primitive type.

You can find the definition of the Proto::Reader in "isaac-sdk-folder/engine/alice/hooks/message_hook.hpp, but TBH it’s not that easy to understand from there.

Good luck.
Cheers

1 Like

Thank so much!! That is super useful!