Isaac lab failed to generate dataset

Omniverse Isaac Sim
, ,
1

I’ve added teleoperation equipment to my Isaac lab, controlling a Franka robotic arm to perform 10 successful stacking demonstrations, then using automatic labeling. However, when I use this dataset to generate mini-batches of examples, I’m finding that it’s misclassifying failed examples as successful ones. This happens even when all the blocks are scattered across the table. I don’t understand why this happens, and could someone tell me how to troubleshoot the issue? Thanks!

Isaac Sim Version

4.5.0
4.2.0
4.1.0
4.0.0
4.5.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):

Operating System

Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):

Steps to Reproduce

step a: create folder for datasets

mkdir -p datasets

step b: collect data with a selected teleoperation device. Replace <teleop_device> with your preferred input device.

Available options: spacemouse, keyboard, handtracking, handtracking_abs, dualhandtracking_abs

./isaaclab.sh -p scripts/tools/record_demos.py --task Isaac-Stack-Cube-Franka-IK-Rel-v0 --teleop_device <teleop_device> --dataset_file ./datasets/dataset.hdf5 --num_demos 10

step c: replay the collected dataset

./isaaclab.sh -p scripts/tools/replay_demos.py --task Isaac-Stack-Cube-Franka-IK-Rel-v0 --dataset_file ./datasets/dataset.hdf5

step d: Self-annotated datasets

./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/annotate_demos.py
–device cuda --task Isaac-Stack-Cube-Franka-IK-Rel-Mimic-v0 --auto
–input_file ./datasets/dataset.hdf5 --output_file ./datasets/annotated_dataset.hdf5

step e: Generate mini-batch dataset

./isaaclab.sh -p scripts/imitation_learning/isaaclab_mimic/generate_dataset.py
–device cuda --num_envs 10 --generation_num_trials 10
–input_file ./datasets/annotated_dataset.hdf5 --output_file ./datasets/generated_dataset_small.hdf5
(Add more steps as needed)

3

@Denvi-halli i would suggest making a post on the github repo in addition for support/bug reports related to Isaac Lab in case they are able to be more responsive in addressing your issues:

To ensure efficient support and collaboration, please submit your topic to its GitHub repo following the instructions provided on Isaac Lab’s Contributing Guidelines regarding discussions, submitting issues, feature requests, and contributing to the project.

1 Like