Send out the Message API of Apriltags with Phyton API to the terminal

We detected April tags with the realsense, it´s working.
On ISAAC, now we want to send and show the Message API fidicual_list on the Terminal at the same time as we detect April tags. We know that is possible with a Pong Codelet with phyton, but we found only examples for C/C++.
The current version is Ubuntu 18.04 with Cuda Toolkit 10.0
Thanks in advance

Python print should work still if that’s what you mean using the message attribute of the proto which is the string description. //sdk/apps/tutorials/ping_python/ping_python.py should have an example unless I misunderstood.

The example ping_phyton works too. It sends every second the “Hello Word” Message to the Terminal. But if i try to combine both (means april tags detection and send messages out) togehter by change build, json and py file, its not working. It Only detect the April tags, but they are no the Hello Word message on the terminal.
If we look at the april.tags.phyton.py file, we see that the line " from engine.pyalice import * " to “app.start_wait_stop()” are from the ping_phyton.py file. I have it inserted.
My real actually wish is that the terminal shows me FiducialProto Message API. Because i want to know the camera and Tag coordinates (x,y and z) from the detected April Tags.

Here´s my april.tags.phyton.py file

from engine.pyalice import Application

from engine.pyalice import *

import argparse

class PingPython(Codelet):
    def start(self):

 self.tick_periodically(1.0)

    def tick(self):

print(self.config.message)

def main():
    app = Application(app_filename="apps/tutorials/ping_python/ping_python.app.json")
    app.nodes["ping_node"].add(PingPython)
    app.start_wait_stop()

if __name__ == "__main__":
    parser = argparse.ArgumentParser(
        description="""
Description: Apriltag detection using different cameras - python sample application
Example usage(for Jetson):
             $python3 apps/samples/april_tags/april_tags_python.py
             $python3 apps/samples/april_tags/april_tags_python.py --camera realsense --resolution 1280x720 --framerate 30
             $python3 apps/samples/april_tags/april_tags_python.py --camera v4l2 --resolution 1920x1080 --framerate 15 --device_id 3""",
        formatter_class=argparse.RawTextHelpFormatter)
    parser.add_argument(
        '--camera',
        dest='camera',
        action='store',
        default='zed',
        choices=['zed', 'realsense', 'v4l2'],
        help='Camera type')
    parser.add_argument(
        '--resolution',
        dest='resolution',
        action='store',
        default='1280x720',
        help='Camera resolution')
    parser.add_argument(
        '--framerate',
        dest='framerate',
        action='store',
        type=int,
        default=60,
        help='Camera framerate')
    parser.add_argument(
        '--device_id', dest='device_id', action='store', type=int, help='Camera device id')
    args = parser.parse_args()
    # Create april_tag_python application
   app = Application(
        name="april_tags_python",
        modules=[
            "//packages/perception:april_tags", "realsense", "sensors:v4l2_camera", "viewers", "zed"
        ])
    # Setup camera node
    camera = None
    if args.camera == "zed":
        camera = app.add('input_images').add(app.registry.isaac.ZedCamera)
        camera.config.resolution = args.resolution
        camera.config.camera_fps = args.framerate
        camera_out_channel = "left_camera_rgb"
    elif args.camera == "realsense":
        camera = app.add('input_images').add(app.registry.isaac.RealsenseCamera)
        camera.config.cols, camera.config.rows = tuple(
            [int(arg) for arg in args.resolution.split('x')])
        camera.config.color_framerate = args.framerate
        camera_out_channel = "color"
    elif args.camera == "v4l2":
        camera = app.add('input_images').add(app.registry.isaac.V4L2Camera)
        if args.device_id == None:
            raise ValueError('Could not set None. Please provide device id')
        camera.config.device_id = args.device_id
        camera.config.cols, camera.config.rows = tuple(
            [int(arg) for arg in args.resolution.split('x')])
        camera.config.rate_hz = args.framerate
        camera_out_channel = "frame"
    else:
        raise ValueError('Not supported Camera type {}'.format(args.camera))
    # Setup april tag node
    tags_detection = app.add('april_tags_detection').add(
        app.registry.isaac.perception.AprilTagsDetection)
    tags_detection.config.max_tags = 50
    fiducials_viewer = app.nodes['april_tags_detection'].add(
        app.registry.isaac.viewers.FiducialsViewer)
    # Setup viewer node
    viewer = app.add('image_viewers').add(app.registry.isaac.viewers.ColorCameraViewer)
    # Connect message channels
    app.connect(camera, camera_out_channel, tags_detection, "image")
    app.connect(tags_detection, "april_tags", fiducials_viewer, "fiducials")
    app.connect(camera, camera_out_channel, viewer, "color_listener")

    app.load("apps/samples/april_tags/april_tags_python.config.json")
    app.run()

Here´s my Build:

load("//engine/build:isaac.bzl","isaac_pkg", "isaac_app", "isaac_py_app")

isaac_pkg(
    name = "ping_python-pkg",
    srcs = ["ping_python"],
)

isaac_app(
    name = "april_tags",
    modules = [
        "//packages/perception:april_tags",
        "realsense",
        "viewers",
    ],
)

isaac_py_app(
    name = "april_tags_python",
    srcs = [
        "april_tags_python.py",
    ],
    modules = [
        "//packages/perception:april_tags",
        "realsense",
        "sensors:v4l2_camera",
        "viewers",
        "zed",
    ],
    data = [
        "april_tags_python.config.json",
        "//packages:py_init",
    ],
    deps = [
        "//engine/pyalice",
    ],
)

py_binary(
    name = "ping_python",
    srcs = [
        "__init__.py",
        "ping_python.py",
    ],
    data = [
        "ping_python.app.json",
        "//apps:py_init",
        "//apps/tutorials:py_init",
    ],
    deps = [
        "//engine/pyalice",
    ],
)

Here is my april.tags.app.json file:

{
  "name": "april_tags",
  "modules": [
    "//packages/perception:april_tags",
    "viewers",
    "realsense"
  ],
  "graph": {
    "nodes": [
      {
        "name": "input_images",
        "components": [
          {
            "name": "isaac.alice.MessageLedger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "isaac.Realsense",
            "type": "isaac::RealsenseCamera"
          }
        ]
      },
     {
    "name": "ping_node",
        "components": []
    },
      {
        "name": "april_tags_detection",
        "components": [
          {
            "name": "isaac.alice.MessageLedger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "isaac.perception.AprilTagsDetection",
            "type": "isaac::perception::AprilTagsDetection"
          },
          {
            "name": "isaac.viewers.FiducialsViewer",
            "type": "isaac::viewers::FiducialsViewer"
          }
        ]
      },
      {
        "name": "image_viewers",
        "components": [
          {
            "name": "isaac.alice.MessageLedger",
            "type": "isaac::alice::MessageLedger"
          },
          {
            "name": "isaac.viewers.ColorCameraViewer",
            "type": "isaac::viewers::ColorCameraViewer"
          }
        ]
      }
    ],
    "edges": [
      {
        "source": "input_images/isaac.Realsense/color",
        "target": "april_tags_detection/isaac.perception.AprilTagsDetection/image"
      },
      {
        "source": "april_tags_detection/isaac.perception.AprilTagsDetection/april_tags",
        "target": "april_tags_detection/isaac.viewers.FiducialsViewer/fiducials"
      },
      {
        "source": "input_images/isaac.Realsense/color",
        "target": "image_viewers/isaac.viewers.ColorCameraViewer/color_listener"
      }
    ]
  },
  "config": {
    "april_tags_detection": {
      "isaac.perception.AprilTagsDetection": {
        "max_tags": 50
      }
    },
   "ping_node": {
      "PyCodelet": {
        "message": "Hello Word!"
      }
    },
    "input_images": {
      "isaac.V4L2Camera": {
        "device_id": 0,
        "rows": 448,
        "cols": 800,
        "rate_hz": 30
      }
    },
    "websight": {
      "WebsightServer": {
        "port": 3000,
        "ui_config": {
          "windows": {
            "Tags": {
              "renderer": "2d",
              "channels": [
                { "name": "april_tags/image_viewers/isaac.viewers.ColorCameraViewer/Color" },
                { "name": "april_tags/april_tags_detection/isaac.viewers.FiducialsViewer/fiducials" }
              ]
            }
          }
        }
      }
    }
  }
}

The code is setting up an Application and then you are loading an app graph too which may be getting a little confusing there.

Either way, there are at least two options to do what I think you’re trying to do:

  1. (not recommended) Write a new Python codelet that receives the Fiducial Detection messages and prints it out with print in tick(). Update the graph to use this component and create an edge between the output of the tags detector and its Fiducual Detection message receiver.

  2. (recommended) Use Isaac Sight (add the viewer component, see other samples for example) and view output channel of “april_tags” component “fiducials.” You will only see the last one, but that is enough to debug with.

To 2: Whats the name of the viewer component that must be added? “isaac.viewers.Fiducialsviewer” was already added. How can i view the output channel “fiducials”? Please more details.

Im just want to know the tag coordinates (x,y,z, rotation) from the detected april tags.

To.1: Here too. I need an example line in Phyton that receives the Fidicual Detection Messages. The ping_phyton example sends only a myself definied text.

For 2, you should see in Isaac Sight (http://localhost:3000 when launched) a set of channels to update on the left-hand side, specifically one for april_tags_detection/fiducials. If you run the “april_tags” Isaac app target, it should read the april_tags.app.json file which is configured to bring up Isaac Sight at port 3000 where the list of channels should appear on the left side of the window if you haven’t already seen it.

The isaac_py_app target (april_tags_python) will setup the application based on the code in april_tags_python.py which sets up the graph and then appends in april_tags_python.config.json which configures Websight the same way.

For 1, the “ping pong” app should give you an idea of receiving a message, and you could look at sdk/packages/pyalice/tests/buffer_test.py::PyImageReader(Codelet) for an example of receiving messages in Python and ticking on message (see start(). You can then just print to console the message you received.

To 2 : Yes, i already know about the websight (http://localhost:3000). I can detect april tags and see their tag id on the realsense. i see the channels on the left side too. But i dont understand how i can see now the coordiantes from the detected april tags and dont know what exactly i´m must to do. Where must i place the output channel? Directly under fidicuals if we look at http://localhost:3000? Whats the name of the output channel? i think the solution on Isaac Sight which u recommended isnt so difficult, but i have problems with it. Can u show me your json code which u used? For my Bachelor Thesis, i must find out the coordinates from the detected april tags. Thanks for your support.

To 1: Thats exactly what i´m asking and searching for. Thanks. I try this example of receiving messages in phyton and then i report here if its work.