Limiting GPU memory usage in a container

[dennis.zhang97]

As title, how can I limit the usage of GPU memory in a container?

[AastaLLL]

Hi,

Suppose you can do this in a similar way as the below comment:

CSI Camera freezes every time I try image classification

Hi @jdwwhetten2, can you try the suggestions from this post: https://forums.developer.nvidia.com/t/camera-freezes-in-image-classification/158686/9 Namely, since you already have lots of SWAP mounted, try adding --memory=500M --memory-swap=8G to your docker run script. Also, try disabling the X-server from starting and also disable ZRAM which can save memory. Use these commands and reboot: $ sudo systemctl set-default multi-user.target $ sudo systemctl disable nvzramconfig Here are the comm…

Thanks.

[dennis.zhang97]

Hi @AastaLLL ,

The problem is docker won’t calculate cuda and pytorch used memory, if you use docker stats of a pytorch container, it will be ~100MB memory usage, but actually it took over 3GB memory to run the container, most of them are used in GPU.

[AastaLLL]

Hi,

How do you check if the memory is used in the CPU or GPU?
Since Jetson is a shared memory system, the memory might. be used for loading the libraries.

Thanks.

[dennis.zhang97]

Hi, I am using jetson_stats to check the memory usage.

[AastaLLL]

Hi,

The output from jtop is the info of physical memory which contains both CPU and GPU allocation.

Thanks.

[dennis.zhang97]

Yes, but on the docker stats, it only used ~100MB memory, on the jtop or htop, ~3GB memory was used.

[AastaLLL]

Hi,

Yes, because it measures the physical memory usage so not only the usage of docker is taken into account.
For example, enabling the Ubuntu system also consumes some memory.

Thanks.