Training Object Detection from Simulation in Docker on Ubuntu 20.04

Hi all,

is it possible to run the object detection tutorial
on a system with Ubuntu 20.04?

I could successfully create and execute the script opening the jupyter notebook. When running the first cells an Unity 3D window pops up constantly flashing generated training images. However, the notebook responses with “Generated Samples: 0” (see appended picture) while localhost:3000 is not accessible.

Hope you can help!

Hi I got the same problem on Ubuntu 18.
This is the same problem Cannot generate samples data from Docker Isaac Object Detection Training

1 Like

Could you verify that GPU support in Docker is working correctly with the following command from the tutorial?

docker run --gpus all nvidia/cuda:10.0-base nvidia-smi

We have seen this issue when GPUs are not available properly in the container before at least.

Looks good to me:

jesser@9700-FJLTC63-Ubuntu:~/dev-isaac$ docker run --gpus all nvidia/cuda:10.0-base nvidia-smi
Fri Feb 19 13:49:01 2021       
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|   0  GeForce RTX 206...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   46C    P8     4W /  N/A |    528MiB /  5934MiB |      5%      Default |
|                               |                      |                  N/A |
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |

The outputs are similiar to Cannot generate samples data from Docker Isaac Object Detection Training - #5 by sofiahradzi

Any suggestions from your side on how to debug this issue?

Hi, I got my solutions on Ubuntu 18.

Modify the like this

#if [[ $IS_19 == 1 ]]; then
#    NV_FLAG="--runtime=nvidia -e CUDA_VISIBLE_DEVICES=all"

Because I found my docker’s version is 20.
It will use NV_FLAG="--runtime=nvidia -e CUDA_VISIBLE_DEVICES=all", but this will cause the problem.
So I change to NV_FLAG="--gpus=all"

This actually solved my problem. my docker version was 20 as well
I checked it by

docker version

Thank you.

I had made the script change. Running it seems that it’s stuck - I saw roughly 30 rows of:

Generated 0 samples

over a few minutes

before it started making progress. Then it worked to completion.