Custom annotator

I need some hints for implementing a custom annotator for replicator in Python. Any examples would help a lot. The documentation here Annotators Information — Omniverse Extensions latest documentation does not help.

What I am trying is to retrieve parts of the geometry of annotated objects and make them available to a writer (also custom).

Any advice appreciated
Bruno

Hi there,

creating custom annotators is not documented at the moment. You can however do this, either by accessing available data from omnigraph nodes, or data from already existing annotators.

You can get started by taking a look at the annotators examples from replicator:

  • /omni/replicator/core/scripts/annotators_default.py

Or the dope/ycb writers from the pose estimation tutorial, which are also using custom nodes and annotators for the writer:

  • /omni.replicator.isaac/python/scripts/writers/dope_writer.py

Could you explain in a bit more detail what data are you trying to access through the annotators?

Best,
Andrei

Thanks @ahaidu, I am in a railway context. From a moving trains perspective, I would like to publish information about the shape of the track centerline (represented by a UsdGeomBasisCurves) that lies in front of the train. I need to output the shape as a list of points in the ego train’s space at every simulation step. I was thinking about using Replicator, but the existing annotators don’t give me access to the curve’s geometry.

I am not aware of any AOV providing curves data. Can you access the lines points data from stage for example, without using annotators?

Is it possible to attach a visible material/mesh on the line? This way you could semantically label it and then the data would be available to the rendering engine and the AOVs. You can also toggle its visibility to have it available only when needed. Here is an example on doing this:

Best,
Andrei

I can easily get the data from the stage. I was not aware that using annotators is directly connected to using AOV (which I don’t know yet). I will have a look at your example, thanks a lot.

Could you provide a snippet on how you access the data in stage so I can get an idea on how the annotator data should look like. Thanks!

I have not implemented it yet, but it would look something like this (very much simplified):

points = UsdGeom.BasisCurves.Get(stage, prim_path).GetPointsAttr().Get()
publish_points(ros_publisher, topic, points)

This would only provide you with the sparse points of the curve in world frame right?

Something like: [(1.2154958, -2.823142, 0), (1.2154958, -2.823142, 0), ..]

Would this be enought for your case? If you for example have a large distance between the curbe points, there will be not data between points. Or are you planning to numerically interpolate between the points yourself to get the points at a higher frequency?

If this approach is would be fine with you, you can use the camera_params annotators to transform these interplolated points from world frame to camera frame for every simulation step.

You would basically cache once all the world points with your desired interpolation resolution, you would make this available to the writer/annotator, and use the camera_params annotator data to constantly transform the points in the camera frustrum to the new transform and publish them. Let me know if this is the approach you had in mind, I can then provide you with snippets on how to continue with this.

Yes, distinct points without interpolation would be fine for the time being. I currently changed my approach for this issue. Right now, I wrote an extension that subscribes to update events and calls a function that gathers all information needed from the stage and publishes it via ROS afterwards. This seems to work quite well. I am not sure if using Replicator with custom writer/annotator would give me a significant advantage.

Best,
Bruno

Since the data is not available in annotators it will basically be the same thing only through different channels.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.