Docker with gpu

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, ,
1

Hello, I am using a jetson device for federated learning pytorch training, how can I call my GPU in a docker container to complete pytorch training?The docker image version I have installed is nvidia-jetson-l4t-ml-r35.1.0-py3 and my jetpack version is 5.1.0

3

Hi,

The Pytorch package in the l4t-ml container has enabled CUDA support already.
So you can train it with GPU in the same way as other environments.

Thanks.

1 Like
4

Thanks, I just found out that it works, and I think the problem may be with my federated learning framework

5

微信图片_20231112161834
微信图片_202311121618341920×1097 156 KB

The federated learning framework I use is fedml, and I can successfully check the CUDA in Doker, but why is it really running IGPU during training.The instructions to start the container are:docker run -t -i --runtime nvidia fedml/fedml:latest-nvida-jetson-l4t-ml-r35.1.0-py3 /bin/bash.
屏幕截图 2023-11-12 162507
屏幕截图 2023-11-12 1625071765×294 43.4 KB

6

Hi,

Is the GPU usage measured when the training time?

If yes, it does use GPU but just not occupy all the resources.
A common cause is the task is memory-bound so GPU needs to wait for the data to compute.

Thanks.

7

Yes, I started training when I looked at JTOP and the usage of the right GPU was always a flat line during the whole process, but the IGPU occupation always kept a high number, and the training of Cifar10 image classification took two hours, and my RTX3060 graphics card only took thirty minutes, all I thought was just calling the core graphics card

8

Hi,

Could you explain more about the IGPU you mean?
Nano is iGPU so the usage is high?

Moreover, we don’t expect Nano can achieve the same performance as a desktop GPU.
4 times slower sounds okay for Nano.

Thanks.

9

I’m very sorry, I chose the wrong post tag before, my device is Jetson AGX Xavier, according to the information I checked, the IGPU is a graphics integrated graphics card in the CPU, its performance is not as good as the discrete graphics card, so I think I only called the integrated graphics card, and did not successfully call the discrete graphics card.

10

Hi,

Do you mean the training works on iGPU (Jetson) but not work on dGPU (3060)?
If yes, we can move your topic to the desktop GPU board.

Thanks.

12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.