Jetpack 5.0.2 Docker Container: cannot run Tensorflow

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
1

Hello,

I just pulled the docker image of the latest Jetpack 5.0.2 version. However, when I try to run Tensorflow I get the following:

nvidia@nvidia-desktop:~$ docker run -it --rm --runtime nvidia --network host nvcr.io/nvidia/l4t-tensorflow:r35.1.0-tf2.9-py3
root@nvidia-desktop:/# python3
Python 3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.constant([1,2,3])
2022-10-13 07:59:02.981299: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:977] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-10-13 07:59:03.052606: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:977] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-10-13 07:59:03.052847: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:977] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-10-13 07:59:03.055257: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:977] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-10-13 07:59:03.055515: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:977] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
2022-10-13 07:59:03.055685: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:977] could not open file to read NUMA node: /sys/bus/pci/devices/0000:00:00.0/numa_node
Your kernel may have been built without NUMA support.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 267, in constant
    return _constant_impl(value, dtype, shape, name, verify_shape=False,
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 279, in _constant_impl
    return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 304, in _constant_eager_impl
    t = convert_to_eager_tensor(value, ctx, dtype)
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/constant_op.py", line 101, in convert_to_eager_tensor
    ctx.ensure_initialized()
  File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/context.py", line 611, in ensure_initialized
    context_handle = pywrap_tfe.TFE_NewContext(opts)
tensorflow.python.framework.errors_impl.InternalError: cudaGetDevice() failed. Status: initialization error

Searching the web for this error, I find mentions of incompatible CUDA and cuDNN versions, but this surely cannot be the problem as this is running in the official docker container. Have I just done something stupid? Doing the above works just fine in the JetPack 4.6 container. Any advice is appreciated.

3

Hi,

Please noted that there are some depedencies between GPU driver and libraries.

Did you setup Jetson with JetPack 5.0.2 as well?
If not, please give it a try.

Thanks.

4

I’m not used to the Jetson ecosystem, so bear with me. I have a few Jetson AGX Xavier units (not sure of the distinction between development kit and module, but since I can ssh into the device and don’t need to flash the finished software into the device, I assume they are development kits). These have been set up such that I can run the l4t-tensorflow docker containers for JetPack 4.6. Now that I want to run the newer containers for 5.0.2 I need to also update the software on the device, it is not sufficient to change the container, are these (How to Install JetPack :: NVIDIA JetPack Documentation) the correct instructions for this? if I’ve read these instructions correctly they all require physical access to the device? I cannot access our devices physically as they are off-site, is there any way that I could update our devices without being there in person? Thank you.

5

Hi,

Please flash the device directly if you want to upgrade from JetPack 4.6 to JetPack 5.0.2.
Thanks.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.