Hello,
I am trying to test isaac.perception.DepthImageFlattening, but it is not found in lib perception module. Its this feature active? or is it located in another module?
Thanks!
Hello there,
DepthImageFlattening was part of perception module in 2019.1 (https://docs.nvidia.com/isaac/archive/2019.1/doc/component_api.html#isaac-perception-depthimageflattening).
However, it is moved to rgbd_processing module with the recent 2019.2 release (https://docs.nvidia.com/isaac/isaac/doc/component_api.html#isaac-rgbd-processing-depthimageflattening).
Which version of Isaac are you using and what is the error message that you are getting?
Best,
Oguz
Some possible errors and solutions:
engine/alice/backend/modules.cpp@337: Could not load component 'isaac::rgbd_processing::DepthImageFlattening'
means you forgot “rgbd_processing” in the “modules” section of your app.json file.
engine/alice/backend/modules.cpp@307: rgbd_processing: ... /rgbd_processing/librgbd_processing_module.so: cannot open shared object file: No such file or directory
means you forgot “rgbd_processing” in the “modules” section of your BUILD file.
If you are using 2019.1, you need “perception” instead of “rgbd_processing” for DepthImageFlattening.
Here is a minimal example for 2019.2:
BUILD file
load("//engine/build:isaac.bzl", "isaac_app")
isaac_app(
name = "depth_image_flattening",
modules = [
"rgbd_processing"
],
)
depth_image_flattening.app.json file
{
"name": "depth_image_flattening",
"modules": [
"rgbd_processing"
],
"graph": {
"nodes": [
{
"name": "image_flattening",
"components": [
{
"name": "message_ledger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "DepthImageFlattening",
"type": "isaac::rgbd_processing::DepthImageFlattening"
}
]
}
],
"edges": [ ]
},
"config": { }
}
Thanks for the example. The issue was with the module, the documentation stated it was in perception but it was in rgbd_processing.
After fixing that im running into this error:
2019-08-10 21:20:43.456 PANIC engine/alice/hooks/pose_hook.cpp@29: Could not get the transformation ground_T_camera.
Im guessing this is related to the pose tree and a ground reference that needs to be initialized. I tried to add isaac.alice.PoseInitializer component in a node to resolve the issue but it did not.
Do you have suggestions of how to get this to run in a minimal example?
Also I have a question abotu the documentation, what is the difference between DepthImageFlattening and FreeSpaceFromDepth ? The documentation state they output the same thing. Can i get an elaboration on the difference in functionality?
Thanks!
Seems like ground_T_camera did not get initialized. Could you post your isaac.alice.PoseInitializer usage please? Also, please check PoseTree window on Sight webpage (possibly located at localhost:3000). Are camera and ground frames connected?
DepthImageFlattening codelet has a “ground_frame” parameter, which you can set to “camera” or any other connected pose for testing.
I would also suggest having a look at apps/kaya/waste_sorting.app.json as a more complete example on how to use DepthImageFlattening.
DepthImageFlattening and FreeSpaceFromDepth have alternative algorithms for the same functionality. We should explain their differences in the documentation, thanks for pointing this out. For now, please have a look at their parameters.
Hi OguzSaglam,
I took a look at Kaya and we combined it with the zed depth example. We still get the same error. Here is our .app.json maybe you can see where we went wrong.
{
"name": "aros-depth-unit-test",
"modules": [
"//packages/stereo_depth:coarse_to_fine",
"viewers",
"rgbd_processing",
"zed"
],
"graph": {
"nodes": [
{
"name": "input_images",
"components": [
{
"name": "isaac.alice.MessageLedger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "isaac.ZedCamera",
"type": "isaac::ZedCamera"
}
]
},
{
"name": "depth_visualization",
"components": [
{
"name": "isaac.alice.MessageLedger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "isaac.viewers.DepthCameraViewer",
"type": "isaac::viewers::DepthCameraViewer"
}
]
},
{
"name": "depth_estimation",
"components": [
{
"name": "isaac.alice.MessageLedger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "isaac.stereo_depth.CoarseToFineStereoDepth",
"type": "isaac::stereo_depth::CoarseToFineStereoDepth"
}
]
},
{
"name": "image_flattening",
"components": [
{
"name": "message_ledger",
"type": "isaac::alice::MessageLedger"
},
{
"name": "DepthImageFlattening",
"type": "isaac::rgbd_processing::DepthImageFlattening"
},
{
"name": "initial_pose",
"type": "isaac::alice::PoseInitializer"
}
]
}
],
"edges": [
{
"source": "input_images/isaac.ZedCamera/left_camera_rgb",
"target": "depth_estimation/isaac.stereo_depth.CoarseToFineStereoDepth/left_image"
},
{
"source": "input_images/isaac.ZedCamera/extrinsics",
"target": "depth_estimation/isaac.stereo_depth.CoarseToFineStereoDepth/extrinsics"
},
{
"source": "input_images/isaac.ZedCamera/right_camera_rgb",
"target": "depth_estimation/isaac.stereo_depth.CoarseToFineStereoDepth/right_image"
},
{
"source": "depth_estimation/isaac.stereo_depth.CoarseToFineStereoDepth/left_depth_image",
"target": "depth_visualization/isaac.viewers.DepthCameraViewer/depth_listener"
},
{
"source": "depth_estimation/isaac.stereo_depth.CoarseToFineStereoDepth/left_depth_image",
"target": "image_flattening/DepthImageFlattening/depth"
}
]
},
"config": {
"image_flattening": {
"DepthImageFlattening": {
"min_distance": 0.1,
"max_distance": 1.7,
"height_min": 0.08,
"height_max": 0.20,
"ground_frame": "camera",
"camera_frame": "camera",
"range_delta": 0.015,
"sector_delta": 0.008
}
},
"depth_estimation": {
"isaac.stereo_depth.CoarseToFineStereoDepth": {
"min_depth": 0.0,
"max_depth": 20.0,
"baseline" : 0.12
}
},
"depth_visualization": {
"isaac.viewers.DepthCameraViewer": {
"max_visualization_depth": 20.0,
"camera_name" : "left_camera"
}
},
"input_images": {
"isaac.ZedCamera": {
"resolution": "672x376",
"tick_period": "60Hz"
}
},
"image_flattening": {
"initial_pose": {
"lhs_frame": "world",
"rhs_frame": "camera",
"pose": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
],
"attach_interactive_marker": false
}
},
"websight": {
"WebsightServer": {
"port": 3000,
"ui_config": {
"windows": {
"Depth": {
"renderer": "2d",
"channels": [
{
"name": "aros-depth-unit-test/depth_visualization/isaac.viewers.DepthCameraViewer/Depth"
}
]
},
"LeftCamera": {
"renderer": "2d",
"channels": [
{
"name": "aros-depth-unit-test/input_images/isaac.ZedCamera/left_rgb"
}
]
}
}
}
}
}
}
}
This is a tricky one. As a result of one (unfortunate in my opinion) feature of JSON (RFC 7159: The JavaScript Object Notation (JSON) Data Interchange Format), your first “image_flattening” config is discarded because of the second “image_flattening” config. Please try merging them, i.e.,
“image_flattening”: {
“DepthImageFlattening”: {
“min_distance”: 0.1,
…
},
“initial_pose”: {
…
}
}