We detected April tags with the realsense, it´s working.
On ISAAC, now we want to send and show the Message API fidicual_list on the Terminal at the same time as we detect April tags. We know that is possible with a Pong Codelet with phyton, but we found only examples for C/C++.
The current version is Ubuntu 18.04 with Cuda Toolkit 10.0
Thanks in advance
1 Like
Python print should work still if that’s what you mean using the message attribute of the proto which is the string description. //sdk/apps/tutorials/ping_python/ping_python.py should have an example unless I misunderstood.
The example ping_phyton works too. It sends every second the “Hello Word” Message to the Terminal. But if i try to combine both (means april tags detection and send messages out) togehter by change build, json and py file, its not working. It Only detect the April tags, but they are no the Hello Word message on the terminal.
If we look at the april.tags.phyton.py file, we see that the line " from engine.pyalice import * " to “app.start_wait_stop()” are from the ping_phyton.py file. I have it inserted.
My real actually wish is that the terminal shows me FiducialProto Message API. Because i want to know the camera and Tag coordinates (x,y and z) from the detected April Tags.
Here´s my april.tags.phyton.py file
from engine.pyalice import Application
from engine.pyalice import *
import argparse
class PingPython(Codelet):
def start(self):
self.tick_periodically(1.0)
def tick(self):
print(self.config.message)
def main():
app = Application(app_filename="apps/tutorials/ping_python/ping_python.app.json")
app.nodes["ping_node"].add(PingPython)
app.start_wait_stop()
if __name__ == "__main__":
parser = argparse.ArgumentParser(
description="""
Description: Apriltag detection using different cameras - python sample application
Example usage(for Jetson):
$python3 apps/samples/april_tags/april_tags_python.py
$python3 apps/samples/april_tags/april_tags_python.py --camera realsense --resolution 1280x720 --framerate 30
$python3 apps/samples/april_tags/april_tags_python.py --camera v4l2 --resolution 1920x1080 --framerate 15 --device_id 3""",
formatter_class=argparse.RawTextHelpFormatter)
parser.add_argument(
'--camera',
dest='camera',
action='store',
default='zed',
choices=['zed', 'realsense', 'v4l2'],
help='Camera type')
parser.add_argument(
'--resolution',
dest='resolution',
action='store',
default='1280x720',
help='Camera resolution')
parser.add_argument(
'--framerate',
dest='framerate',
action='store',
type=int,
default=60,
help='Camera framerate')
parser.add_argument(
'--device_id', dest='device_id', action='store', type=int, help='Camera device id')
args = parser.parse_args()
# Create april_tag_python application
app = Application(
name="april_tags_python",
modules=[
"//packages/perception:april_tags", "realsense", "sensors:v4l2_camera", "viewers", "zed"
])
# Setup camera node
camera = None
if args.camera == "zed":
camera = app.add('input_images').add(app.registry.isaac.ZedCamera)
camera.config.resolution = args.resolution
camera.config.camera_fps = args.framerate
camera_out_channel = "left_camera_rgb"
elif args.camera == "realsense":
camera = app.add('input_images').add(app.registry.isaac.RealsenseCamera)
camera.config.cols, camera.config.rows = tuple(
[int(arg) for arg in args.resolution.split('x')])
camera.config.color_framerate = args.framerate
camera_out_channel = "color"
elif args.camera == "v4l2":
camera = app.add('input_images').add(app.registry.isaac.V4L2Camera)
if args.device_id == None:
raise ValueError('Could not set None. Please provide device id')
camera.config.device_id = args.device_id
camera.config.cols, camera.config.rows = tuple(
[int(arg) for arg in args.resolution.split('x')])
camera.config.rate_hz = args.framerate
camera_out_channel = "frame"
else:
raise ValueError('Not supported Camera type {}'.format(args.camera))
# Setup april tag node
tags_detection = app.add('april_tags_detection').add(
app.registry.isaac.perception.AprilTagsDetection)
tags_detection.config.max_tags = 50
fiducials_viewer = app.nodes['april_tags_detection'].add(
app.registry.isaac.viewers.FiducialsViewer)
# Setup viewer node
viewer = app.add('image_viewers').add(app.registry.isaac.viewers.ColorCameraViewer)
# Connect message channels
app.connect(camera, camera_out_channel, tags_detection, "image")
app.connect(tags_detection, "april_tags", fiducials_viewer, "fiducials")
app.connect(camera, camera_out_channel, viewer, "color_listener")
app.load("apps/samples/april_tags/april_tags_python.config.json")
app.run()
Here´s my Build:
load("//engine/build:isaac.bzl","isaac_pkg", "isaac_app", "isaac_py_app")
isaac_pkg(
name = "ping_python-pkg",
srcs = ["ping_python"],
)
isaac_app(
name = "april_tags",
modules = [
"//packages/perception:april_tags",
"realsense",
"viewers",
],
)
isaac_py_app(
name = "april_tags_python",
srcs = [
"april_tags_python.py",
],
modules = [
"//packages/perception:april_tags",
"realsense",
"sensors:v4l2_camera",
"viewers",
"zed",
],
data = [
"april_tags_python.config.json",
"//packages:py_init",
],
deps = [
"//engine/pyalice",
],
)
py_binary(
name = "ping_python",
srcs = [
"__init__.py",
"ping_python.py",
],
data = [
"ping_python.app.json",
"//apps:py_init",
"//apps/tutorials:py_init",
],
deps = [
"//engine/pyalice",
],
)
Here is my april.tags.app.json file:
{
"name": "april_tags",
"modules": [
"//packages/perception:april_tags",
"viewers",
"realsense"
],
"graph": {
"nodes": [
{
"name": "input_images",
"components": [
{
"name": "isaac.alice.MessageLedger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "isaac.Realsense",
"type": "isaac::RealsenseCamera"
}
]
},
{
"name": "ping_node",
"components": []
},
{
"name": "april_tags_detection",
"components": [
{
"name": "isaac.alice.MessageLedger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "isaac.perception.AprilTagsDetection",
"type": "isaac::perception::AprilTagsDetection"
},
{
"name": "isaac.viewers.FiducialsViewer",
"type": "isaac::viewers::FiducialsViewer"
}
]
},
{
"name": "image_viewers",
"components": [
{
"name": "isaac.alice.MessageLedger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "isaac.viewers.ColorCameraViewer",
"type": "isaac::viewers::ColorCameraViewer"
}
]
}
],
"edges": [
{
"source": "input_images/isaac.Realsense/color",
"target": "april_tags_detection/isaac.perception.AprilTagsDetection/image"
},
{
"source": "april_tags_detection/isaac.perception.AprilTagsDetection/april_tags",
"target": "april_tags_detection/isaac.viewers.FiducialsViewer/fiducials"
},
{
"source": "input_images/isaac.Realsense/color",
"target": "image_viewers/isaac.viewers.ColorCameraViewer/color_listener"
}
]
},
"config": {
"april_tags_detection": {
"isaac.perception.AprilTagsDetection": {
"max_tags": 50
}
},
"ping_node": {
"PyCodelet": {
"message": "Hello Word!"
}
},
"input_images": {
"isaac.V4L2Camera": {
"device_id": 0,
"rows": 448,
"cols": 800,
"rate_hz": 30
}
},
"websight": {
"WebsightServer": {
"port": 3000,
"ui_config": {
"windows": {
"Tags": {
"renderer": "2d",
"channels": [
{ "name": "april_tags/image_viewers/isaac.viewers.ColorCameraViewer/Color" },
{ "name": "april_tags/april_tags_detection/isaac.viewers.FiducialsViewer/fiducials" }
]
}
}
}
}
}
}
}
The code is setting up an Application and then you are loading an app graph too which may be getting a little confusing there.
Either way, there are at least two options to do what I think you’re trying to do:
-
(not recommended) Write a new Python codelet that receives the Fiducial Detection messages and prints it out with
printintick(). Update the graph to use this component and create an edge between the output of the tags detector and its Fiducual Detection message receiver. -
(recommended) Use Isaac Sight (add the viewer component, see other samples for example) and view output channel of “april_tags” component “fiducials.” You will only see the last one, but that is enough to debug with.
To 2: Whats the name of the viewer component that must be added? “isaac.viewers.Fiducialsviewer” was already added. How can i view the output channel “fiducials”? Please more details.
Im just want to know the tag coordinates (x,y,z, rotation) from the detected april tags.
To.1: Here too. I need an example line in Phyton that receives the Fidicual Detection Messages. The ping_phyton example sends only a myself definied text.
For 2, you should see in Isaac Sight (http://localhost:3000 when launched) a set of channels to update on the left-hand side, specifically one for april_tags_detection/fiducials. If you run the “april_tags” Isaac app target, it should read the april_tags.app.json file which is configured to bring up Isaac Sight at port 3000 where the list of channels should appear on the left side of the window if you haven’t already seen it.
The isaac_py_app target (april_tags_python) will setup the application based on the code in april_tags_python.py which sets up the graph and then appends in april_tags_python.config.json which configures Websight the same way.
For 1, the “ping pong” app should give you an idea of receiving a message, and you could look at sdk/packages/pyalice/tests/buffer_test.py::PyImageReader(Codelet) for an example of receiving messages in Python and ticking on message (see start(). You can then just print to console the message you received.
To 2 : Yes, i already know about the websight (http://localhost:3000). I can detect april tags and see their tag id on the realsense. i see the channels on the left side too. But i dont understand how i can see now the coordiantes from the detected april tags and dont know what exactly i´m must to do. Where must i place the output channel? Directly under fidicuals if we look at http://localhost:3000? Whats the name of the output channel? i think the solution on Isaac Sight which u recommended isnt so difficult, but i have problems with it. Can u show me your json code which u used? For my Bachelor Thesis, i must find out the coordinates from the detected april tags. Thanks for your support.
To 1: Thats exactly what i´m asking and searching for. Thanks. I try this example of receiving messages in phyton and then i report here if its work.
Also, if you want the AprilTag detector to spit out a 6DOF pose, then you’ll need calibrated camera intrinsics. Without knowing the distortion model, image center, and focal length, the detector can only determine the presence and pixel coordinates of the tags, but can’t work back the 3D pose. Take a look at april_tags.app.json' and the intrinsics` source in V4L2Camera to make sure those parameters are configured correctly.
Thanks. i already try that to find out the 6DOF pose… But first i need to see the messages from fidicuals as text. Can u tell me step by step what i must do to see it?
And to 2: Here is my new py Code that receives Fiducial Detection Messages:
def start(self):
self.rx1 = self.isaac_proto_rx("FiducialProto", "apriltag_id_0")
self.rx2 = self.isaac_proto_rx("FiducialProto", "apriltag_id_1")
self.tick_on_message(self.rx1)
self.cnt = 0
self.tick_periodically(1.0)
def tick(self):
img_msg1 = self.rx1.message
img_msg2 = self.rx2.message
print(self.rx1.message)
But i have found that if run “bazel run //apps/samples/april_tags”, its not using the april_tags_python.py file. Why? I have found out this, because i shift short the file to another folder, but it works without the april_tags_python.py file.
Can u tell me how i update a graph to use this? Another question ist, where must be the edge create, in the april_tags.app.json file?
The JSON file and the Python file for april tags end up setting up different application graphs which you can see in the Bazel targets. I would recommend reading through some of the other samples and tutorials in the Isaac SDK to get a flavor of how this works in addition to the documentation.
@hemals
we are also looking to get text continious output of python april tag implementation [ or cpp implementation or c implementation]
However, there are certain issues with getting it printed continiously.
Yet devs were able to print outputs somehow once after termination of the app with cntrl+c.
Is there any simple way to get continious text outputss in terminal/ or in text form somehow?
Could you extend the sight method approach? some other approach?
2 Likes