“FROM nvcr.io/nvidia/l4t-ml:r35.2.1-py3” This is my base image in docker. After this i am installing certain libraries using a requirement.txt[easyocr,opencv-python-headless,flask etc].
Once the docker started running inside jetson,I used docker exec command to navigate inside docker and there used python shell to print the “torch.cuda.is_availalble”. It is always giving me False.
I tried the base image with out installing any libraries from requirement file. Then the docker can access the CUDA. How can i solve this by installing the dependencies and getting the CUDA acceleration?
@rakesh.thykkoottathil.jay does torch.cuda.is_available() return true before you install your dependencies? If so, some of those dependencies are installing another version of PyTorch from pip/pypi that was built without CUDA enabled.
What you can do, is try pip3 freeze and --constraint option like used here:
Also, if you change to dustynv/l4t-ml:r35.2.1-py3 , the original PyTorch wheel is saved under /opt so you can just re-install it after the dependencies, hence restoring GPU acceleration.
@dusty_nv Thanks for the quick response. I tried this but I am getting open cv circular import issue. Could you please help me on the dustynv/l4t-ml:r35.2.1-py3 link. How can i pull that to docker and do an installation after the requirement file installation.
Woops sorry, it is dustynv/l4t-ml:r35.2.1 (without the -py3)
https://hub.docker.com/repository/docker/dustynv/l4t-ml/general
Also, these days I have much more granular containers available than l4t-ml, so you might be able to use a smaller one that has just what you want/need in it:
@dusty_nv Thanks, I can use this container as base image. But do you have anything specfic with l4tml + easyocr. Because i am getting challenge in installing dependencies. Or could you guide me how to approach this? or If you can point me a good docker file that will also help me. I need l4t ml as base and my inference code uses easyocr opencv etc. i can run the docker container when i install all the requirements in a separate environment but i dont have the CUDA support. Thanks in advance.
OK, try using that dustynv/l4t-ml:r35.2.1 as your base instead, and you’ll find the original PyTorch wheel under /opt . After you install your dependencies, just reinstall that wheel, and GPU support in PyTorch should be restored.
COPY --from=0 /opt/pytorch-wheel-name.whl /app/
RUN pip install pytorch-wheel-name.whl
Sorry I am new to docker, Will this do the reinstall part. If so I am not getting the pytorch wheel name
I’m not sure if you are doing a multi-stage build from that COPY command or not (and don’t know how that would impact other python packages in the base image that may not be copied over), but here is how I do it:
RUN pip3 install --verbose /opt/torch*.whl
I did the installation of torch after the requirement file. I did a docker exec, i can import the torch and torch version is “2.0.0+nv23.05’”. But now it is creating opencv import issue.
import cv2
Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/lib/python3.8/dist-packages/cv2/init.py”, line 8, in
from .cv2 import *
ImportError: libavcodec-e61fde82.so.58.134.100: cannot open shared object file: No such file or directory
I am installing opencv-contrib-python==4.5.5.64 , opencv-python-headless==4.5.5.64 in the requirement files.
Python 3.8.10 (default, May 26 2023, 14:05:08)
[GCC 9.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
import torch
torch.version
‘2.0.0+nv23.05’
import cv2
Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/lib/python3.8/dist-packages/cv2/init.py”, line 8, in
from .cv2 import *
ImportError: libavcodec-e61fde82.so.58.134.100: cannot open shared object file: No such file or directory
root@4e3cdf7e7246:/app# pip show opencv-python
WARNING: Package(s) not found: opencv-python
root@4e3cdf7e7246:/app# pip show opencv-python-headless
Name: opencv-python-headless
Version: 4.5.5.64
Summary: Wrapper package for OpenCV python bindings.
Home-page: GitHub - opencv/opencv-python: Automated CI toolchain to produce precompiled opencv-python, opencv-python-headless, opencv-contrib-python and opencv-contrib-python-headless packages.
Author:
Author-email:
License: MIT
Location: /usr/local/lib/python3.8/dist-packages
Requires: numpy, numpy, numpy
Required-by: easyocr
Is this all in the container based on l4t-ml, or are you doing a multi-stage build and copying files over? Because libavcodec might not be in that. PyTorch doesn’t install any opencv. Does cv2 import before you reinstall pytorch?
Everything i am doing in a docker container only. base image is l4t ml.
Yes, cv2 import has no issue before reinstall. Actually i do this installation in the requirement file. But when cv2 loads properly then i will get torch version as 2.0.1(without cuda). But for reinstall torch + cuda loads properly but cv2 does not.
Below is my docker file:
# Use an official Python runtime as a parent image
FROM dustynv/l4t-ml:r35.2.1
# Set the working directory inside the container
WORKDIR /app
# Update the system and install necessary dependencies including GStreamer
RUN apt-get update && \
apt-get install -y \
libgstreamer1.0-0 \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
ffmpeg \
python3-venv \
python3-pip \
python3-dev \
&& apt-get clean && \
rm -rf /var/lib/apt/lists/*
# Set the library search path
ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
# Install any needed packages specified in requirements.txt
COPY requirements.txt /app/
RUN pip3 freeze > /app/constraints.txt && \
pip3 install --no-cache-dir --verbose -r requirements.txt --constraint /app/constraints.txt && \
rm /app/constraints.txt
RUN pip3 install --verbose /opt/torch*.whl
# Copy the rest of your application code into the container
COPY . /app/
# Run your application
CMD ["python3", "/app/observer_1.py"]
Requirement file
easyocr
joblib
json5
jsonpointer
jsonschema
jsonschema-specifications
matplotlib
matplotlib-inline
opencv-contrib-python==4.5.5.64
opencv-python-headless==4.5.5.64
Pillow
regex
requests
requests-oauthlib
requests-toolbelt
scikit-image
urllib3
psycopg2-binary
watchdog
flask
Try inserting some RUN python3 -c 'import cv2; print(cv2.getBuildInformation())' into your dockerfile like below, to debug exactly when cv2 stops working:
# Use an official Python runtime as a parent image
FROM dustynv/l4t-ml:r35.2.1
# Set the working directory inside the container
WORKDIR /app
# Update the system and install necessary dependencies including GStreamer
RUN apt-get update && \
apt-get install -y \
libgstreamer1.0-0 \
gstreamer1.0-plugins-base \
gstreamer1.0-plugins-good \
gstreamer1.0-plugins-bad \
gstreamer1.0-plugins-ugly \
ffmpeg \
python3-venv \
python3-pip \
python3-dev \
&& apt-get clean && \
rm -rf /var/lib/apt/lists/*
RUN python3 -c 'import cv2; print(cv2.getBuildInformation())'
# Set the library search path
ENV LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
# Install any needed packages specified in requirements.txt
COPY requirements.txt /app/
RUN pip3 freeze > /app/constraints.txt && \
pip3 install --no-cache-dir --verbose -r requirements.txt --constraint /app/constraints.txt && \
rm /app/constraints.txt
RUN python3 -c 'import cv2; print(cv2.getBuildInformation())'
RUN pip3 install --verbose /opt/torch*.whl
RUN python3 -c 'import cv2; print(cv2.getBuildInformation())'
# Copy the rest of your application code into the container
COPY . /app/
# Run your application
CMD ["python3", "/app/observer_1.py"]
You may want to try just using dustynv/pytorch:r35.2.1 or dustynv/torchvision:r35.2.1 as base instead, because these don’t have the opencv_cuda version pre-installed. Or figure out which of your dependencies is uninstalling pytorch and force it not to.
Or you can try dustynv/pytorch:2.1-r35.2.1 which already has PyTorch 2.1, maybe that would prevent it from getting uninstalled by your dependencies…
These dustynv/pytorch:r35.2.1 or dustynv/torchvision:r35.2.1 are coming with torch + CUDA?
Before requirement file installation
General configuration for OpenCV 4.5.0 =====================================
Version control: 4.5.0
Extra modules:
Location (extra): /opt/opencv_contrib/modules
Version control (extra): 4.5.0
Platform:
Timestamp: 2021-10-23T16:52:07Z
Host: Linux 5.10.59-tegra aarch64
CMake: 3.16.3
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: RELEASE
CPU/HW features:
Baseline: NEON FP16
required: NEON
disabled: VFPV3
After requirement file installation
Successfully installed PyWavelets-1.4.1 Shapely-2.0.1 blinker-1.6.2 click-8.1.7 contourpy-1.1.1 cycler-0.11.0 easyocr-1.7.1 flask-2.3.2 fonttools-4.42.1 imageio-2.31.3 itsdangerous-2.1.2 kiwisolver-1.4.5 lazy_loader-0.3 matplotlib-3.7.3 opencv-contrib-python-4.5.5.64 opencv-python-headless-4.5.5.64 psycopg2-binary-2.9.7 pyclipper-1.3.0.post5 pyparsing-3.1.1 python-bidi-0.4.2 regex-2023.8.8 requests-toolbelt-1.0.0 scikit-image-0.21.0 tifffile-2023.7.10 watchdog-3.0.0
Step 8/12 : RUN python3 -c ‘import cv2; print(cv2.getBuildInformation())’
—> [Warning] The requested image’s platform (linux/arm64) does not match the detected host platform (linux/amd64) and no specific platform was requested
—> Running in 4583c14db6a1
Traceback (most recent call last):
File “”, line 1, in
File “/usr/local/lib/python3.8/dist-packages/cv2/init.py”, line 8, in
from .cv2 import *
ImportError: libavcodec-e61fde82.so.58.134.100: cannot open shared object file: No such file or directory
The command ‘/bin/sh -c python3 -c ‘import cv2; print(cv2.getBuildInformation())’’ returned a non-zero code: 1
Run ID: cf40 failed after 5m59s. Error: failed during run, err: exit status 1
Run failed
Hi @dusty_nv Thanks , It worked with dustynv/pytorch:2.1-r35.2.1. I will keep you posted if i encounter with any other issue.
@dusty_nv I am getting a runtime error while doning the inference.
I understood like torch version is 2.1.0 + Cuda and torchvision is 0.14 is not compactable. Below is the error.
“RuntimeError: Couldn’t load custom C++ ops. This can happen if your PyTorch and torchvision versions are incompatible, or if you had errors while compiling torchvision from source. For further information on the compatible versions, check GitHub - pytorch/vision: Datasets, Transforms and Models specific to Computer Vision for the compatibility matrix. Please check your PyTorch version with torch.version and your torchvision version with torchvision.version and verify if they are compatible, and if not please reinstall torchvision so that it matches your PyTorch install.”
Torch 2.1.0 requires torchvision 0.15 and above. I did a upgrade of torchvision. But that reinstalled the torch with 2.1 cpu version.
I am trying out different container on your repo.
If you could suggest me any solution that will be great.