Are there plans to develop a demo that utilizes large language models (LLMs) in the Isaac-ROS environment in the future?

I’d like to generate XML files through LLMs and use them to construct a tree that determines robot behavior in the Isaac-ROS environment.

If it’s currently possible, could you explain how to implement the above example in either the Isaac-ROS or Isaac-SIM environments?

Hi there, thank you for your post, re-tagging your post to the Isaac ROS forum.

Hi @lee1062005

We are still exploring how to leverage LLMs for task planning and looking forward to any ideas he may yield.

Raffaello