From the press release:
Physics-based photoreal synthetic data generation, using Cosmos models to generate photoreal videos from controlled 3D scenarios developed in the NVIDIA Omniverse ™ platform.
NVIDIA today announced NVIDIA Cosmos™, a platform comprising state-of-the-art generative world foundation models, advanced tokenizers, guardrails and an accelerated video processing pipeline built to advance the development of physical AI systems...
Which model should be used for this? Probably i’m missing something, but the models i currently see in the catalog are all seem to be for generating new frames, and none of them are for enhancing existing ones.
3 Likes
andras4
February 3, 2025, 12:30pm
2
Also, in the Isaac Sim 4.5 release post:
Cosmos can ingest images and videos and output photoreal video clips to then retrain policy models.
At CES 2025, NVIDIA announced key updates to NVIDIA Isaac, a platform of accelerated libraries, application frameworks, and AI models that accelerate the development of AI robots.
Is there any example for that?
andras4
February 7, 2025, 12:54am
3
I can finally answer this. During GTC we launched Cosmos Transfer1:
Cosmos-Transfer1 is a world-to-world transfer model designed to bridge the perceptual divide between simulated and real-world environments.
For now, you’ll need to be fairly technical to get this setup. Instructions here:
## Environment setup
Clone the `cosmos-transfer1` source code
```bash
git clone git@github.com:nvidia-cosmos/cosmos-transfer1.git
cd cosmos-transfer1
git submodule update --init --recursive
```
Cosmos runs only on Linux systems. We have tested the installation with Ubuntu 24.04, 22.04, and 20.04.
Cosmos requires the Python version to be `3.10.x`. Please also make sure you have `conda` installed ([instructions](https://docs.conda.io/projects/conda/en/latest/user-guide/install/index.html)).
### Inference
The below commands creates the `cosmos-transfer1` conda environment and installs the dependencies for inference:
```bash
# Create the cosmos-transfer1 conda environment.
conda env create --file cosmos-transfer1.yaml
# Activate the cosmos-transfer1 conda environment.
conda activate cosmos-transfer1
show original
There’s also some initial data to run through it for SDG purposes: