Calling nvidias ORB implementation from source

Autonomous Machines Robotics - Isaac Isaac SDK
1

There is an implementation of ORB by nvidia for the ISAAC SDK, however I could not find any docs on how to call that routine from another codebase. Basically I am looking for an API for:

https://docs.nvidia.com/isaac/isaac/packages/orb/doc/orb.html

Is it even possible to call that routine from traditional C(++)? The article is talking about codelets, but also mentions that “The complete ORB source code is contained in the Isaac SDK source code”. I could not find that source code though.

I would like to compare the performance of that implementation to the OpenCV implementation, and maybe replace some of the steps (keypoint detection) with hardware accelerated functions on a Jetson AGX.

I would appreciate any hints or links into the right direction :)

2

Just checking back, I basically am looking for headers for this binary library:

libextract_visualize_orb_module.so

This library is included in the Isaac SDK, and I would like to use it inside an existing cpp code base without migrating everything to Isaac.

I am not asking for any implementation details or the actual source code, just looking for the corresponding header file.

3

The header and implementation for orb::ExtractOrbFeatures() from //packages/orb/gems was not published with the public 2020.1NX release. We are considering packaging the headers and perhaps the implementation too with a later release of Isaac.

For now, as you know, it is only exposed at the moment through the ExtractAndVisualizeOrb Isaac component which you could interface with through a bridge with your existing code.

4

Thanks for the info. Are there any performance implications when interfacing through a bridge?

Imagine some unified memory image and output buffers, could a bridge operate directly on that? I have no idea about ROS, so this might be a very naive question.

5

You could run an Isaac app with the ExtractAndVisualizeOrb and your own custom bridge component that exposes a service, either as a RESTful HTTP service, gRPC service, or something else. See here for an example bridge between Isaac and ROS (your Isaac bridge component would be between Isaac and your code, either in-process over RPC).

There will be performance implications that could be exacerbated by how you implement your bridge component. If you run your code as an Isaac codelet in the same node, then the overhead is minimal, but if you need to marshal and serialize inputs to transport them over Websockets for example to something out-of-process, things could slow down significantly as you would imagine.