I’ve tried two ways of limiting GPU growth: session config and at the GPU level
For nano there is 4GB RAM, only less than 2GB could be assigned to the GPU side.
On jetson, tensorRT engine with FP16 mode is preferred to do the inference.
If you stick in with Tensorflow. There is also a parameter can be used to fix the GPU allocation.
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
config.gpu_options.per_process_gpu_memory_fraction = 0.4
session = tf.Session(config=config, ...)
For both, I get this error:
ImportError: /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block
Upon googling, I found people recommending setting this in bashrc: export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1:/$LD_PRELOAD
This unfortunately did not help my problem
This is what I’m using
Python 3.6.9
Tensorflow 2.4.1 (also tried 2.5.0+nv21.8)
JetPack 4.6.2
Full stack trace:
2022-10-11 10:14:29.349921: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 449 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X1, pci bus id: 0000:00:00.0, compute capability: 5.3)
1 Physical GPUs, 1 Logical GPUs
Traceback (most recent call last):
File "/home/user/face.py", line 1, in <module>
import cv2
File "/usr/lib/python3.6/dist-packages/cv2/__init__.py", line 89, in <module>
bootstrap()
File "/usr/lib/python3.6/dist-packages/cv2/__init__.py", line 79, in bootstrap
import cv2
ImportError: /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block
Any ideas on how to fix this?
Managed to make tensorflow 2.5.0+nv21.8 work with this code:
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
config.gpu_options.per_process_gpu_memory_fraction = 0.4
session = tf.Session(config=config, ...)
2.4 would not work
Hi,
Please noted that there are some dependencies between the TensorFlow package and JetPack software.
If you are using JetPack 4.6.2, which is a minor update from JetPack 4.6.1, it’s recommended to install v2.7.0+nv22.1 instead:
https://developer.download.nvidia.com/compute/redist/jp/v461/tensorflow/
Thanks.
Thank you! I’ll do that.
Might be worth updating the official Tensorflow thread with a link for the 4.6 and 4.6.1 users for posterity!
Thanks.
Already add some information to the topic.
system
Closed
November 2, 2022, 4:45am
8
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.