Disparity between Pose Trees in Carter Sim Tutorial

Hi all,

My group has been playing around with pose trees in Isaac SIM trying to get the precise locations of all objects within the environment.

In one of our tests we tried to create a simple codelet that would print the relative positions of various objects within the environment as a brief sanity check as follows:

#include "PosePrinter.hpp"

#include <memory>
#include <utility>
#include <cstdio>

#include "engine/core/assert.hpp"

#include "engine/gems/image/conversions.hpp"
#include "engine/gems/image/utils.hpp"
#include "messages/messages.hpp"
#include "engine/core/tensor/tensor.hpp"

namespace isaac{

void print_pose(Pose3d pose){ 
  auto translation = pose.translation;
  auto rpy = pose.rotation.eulerAnglesRPY();
  auto quaternion = pose.rotation.quaternion();
  // print translation
  std::cout << "Translation: {X: " << translation[0] << ", Y: " << translation[1] ;
  std::cout << ", Z: " << translation[2] << "}\n";
  // print rotation roll pitch yaw
  std::cout << "RPY: {Roll: " << rpy[0] << ", Pitch: " << rpy[1] << ", Yaw: ";
  std::cout << rpy[2] << "}\n";
  // print rotation quaternion
  std::cout << "Quaternion: {W: " << quaternion.w() << ", X: " << quaternion.x() << ", Y: ";
  std::cout << quaternion.y() << ", Z: " << quaternion.z() << "}\n";
}

void PosePrinter::start(){
  tickPeriodically();
  count = 0;
}

void PosePrinter::tick(){
  count ++;
  std::cout << "############# TICK " << count << "#############\n";
  
  if (count > 10){
    Pose3d world_T_robot = get_world_T_robot(getTickTime());
    Pose3d world_T_left_camera = get_world_T_left_camera(getTickTime());
    Pose3d robot_T_left_camera = get_robot_T_left_camera(getTickTime());
    // print robot position in world
    std::cout << "---------- world_T_robot ----------\n";
    print_pose(world_T_robot);
    std::cout << "---------- world_T_left_camera ----------\n";
    print_pose(world_T_left_camera);
    std::cout << "---------- robot_T_left_camera ----------\n";
    print_pose(robot_T_left_camera);
    
  }
  
}

} // isaac namespace

The problem is that when we viewed the robot_T_camera pose, it differed from what we were expecting. This is designed to work with the carter_sim application.

We expected to see the same values as are in the carter_full_confifg.json where parent_T_camera was defined as [1, 0, 0, 0, 0.11, 0.06, 1.5] (we had been playing with camera height which is where we noticed the disparity).

Instead we got robot_T_camera as [0.5, -0.5, 0.5, -0.5, 0.11, 0.06, 0.55] which we later found was the configuration in the carter_config.json.

Is this what should be expected when using Isaac Sim? Will we need to always update a separate config file for the simulator and Isaac SDK whenever we want to use the sim and make sure that the config files are absolutely identical? This seems like a lot of hassle for now. Are there plans to improve upon this so only one config file needs to be updated to affect both? Have we just completely misunderstood something?

Any help you can give would be greatly appreciated.

Hi D20,

I’ll refer to the latest release from today for the values below:

“parent_T_camera”: [1, 0, 0, 0, 0.11, 0.06, 0.31],

parent_T_camera specified is with respect to the center of mass of the robot (or in the case of a URDF, the root link for that urdf).

“LeftCameraSubscriber”: {
“left_camera_initializer”: {
“lhs_frame”: “robot”,
“rhs_frame”: “left_camera”,
“pose”: [0.5, -0.5, 0.5, -0.5, 0.11, 0.06, 0.55]
},
The pose in the left_camera_initializer is with respect to the ground.

You are correct in that both poses would need to be updated to remain consistent and they will be slightly different (one wrt to center of mass, and one wrt to ground).

Because the simulator and app are both initialized/started separately, can be running on completely separate hardware, and the simulator/SDK use different robot representations its not possible to use the same values… we can try to make this process less of a hassle in the future.