Livestreaming from Jupyter Notebook (no nucleus server)

Omniverse Isaac Sim
1

Hello. I’m wondering how to control a running simulation and livestream from within Jupyter notebook. I see a couple solutions but each has downsides. Would be grateful for help.
Questions:

  1. How to use jupyter to connect to running simulation (ran by ./runheadless.native.sh -v)?
    I start the simulation & load a usd scene. I observe using Omniverse Streaming Client. But from jupyter, SimulationContext can not be imported without first running SimulationApp(). From what I understand SimulationApp itself starts a new simulation. How can i connect to a sim run by " ./runheadless.native.sh -v" and access & control primitives within?

The second option is to start the sim using Python. I see two workflows but again each has problems.
2. Possible to start non-blocking livestream?
If i don’t run ./runheadless.native.sh -v, I can start simulation using

kit = SimulationApp(launch_config=CONFIG)
enable_extension("omni.kit.livestream.native")
while kit._app.is_running() and not kit.is_exiting():
    # Run in realtime mode, we don't specify the step size
    kit.update()

This will load a scene and it will be visible within Omniverse Streaming Client. However it’s blocking, so i can’t add objects. Could i “connect to” this sim from another jupyter notebook? But then again i’d have to run SimulationApp which probably wouldn’t work.

  1. The third example is using “livesync_usd” option when starting SimulationApp. However, this requires a Nucleus server. There is no nucleus within Isaac Docker, so my SimulationApp fails.
    From what i understand, Nucleus can only be installed locally by buying an Enterprise Subscription?

Thanks and best regards.

2

Hi @random.fella1

You can take a look at the Isaac Sim docs Running Python Code from External Editors that show how to execute code from VS Code or Jupyter Notebook in a running Isaac Sim instance