An error in installing pyAerial

Accelerated Computing Aerial Forum
1

The following error occurs in the “Installing pyAerial” stage.
If I use wget command directly, I checked to receive a pandoc-3.1.11.1-1-amd64.deb.
I ask for your help to resolve the issue below.
thank you.

when use build.sh.

yogurt@134servwe:~/cuBB$ AERIAL_BASE_IMAGE=nvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran:24-3-cubb $cuBB_SDK/pyaerial/container/build.sh
/home/yogurt/cuBB/pyaerial/container/build.sh starting...
[+] Building 2742.1s (6/14)                                                                                                                                                                                    docker:default
 => [internal] load build definition from Dockerfile_tmp                                                                                                                                                                 0.0s
 => => transferring dockerfile: 1.46kB                                                                                                                                                                                   0.0s
 => WARN: JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals (line 47)                                                                                         0.0s
 => [internal] load metadata for nvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran:24-3-cubb                                                                                                                              0.0s
 => [internal] load .dockerignore                                                                                                                                                                                        0.0s
 => => transferring context: 2B                                                                                                                                                                                          0.0s
 => CACHED [ 1/10] FROM nvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran:24-3-cubb                                                                                                                                       0.0s
 => [internal] load build context                                                                                                                                                                                        0.0s
 => => transferring context: 44B                                                                                                                                                                                         0.0s
 => ERROR [ 2/10] RUN wget https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb &&     dpkg -i pandoc-3.1.11.1-1-amd64.deb &&     rm pandoc-3.1.11.1-1-amd64.deb                      2742.1s
------
 > [ 2/10] RUN wget https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb &&     dpkg -i pandoc-3.1.11.1-1-amd64.deb &&     rm pandoc-3.1.11.1-1-amd64.deb:
0.043 --2024-12-23 05:28:48--  https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb
0.046 Resolving github.com (github.com)... 20.200.245.247
0.074 Connecting to github.com (github.com)|20.200.245.247|:443... failed: Connection timed out.
128.8 Retrying.
128.8
129.8 --2024-12-23 05:30:58--  (try: 2)  https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb
129.8 Connecting to github.com (github.com)|20.200.245.247|:443... failed: Connection timed out.
259.9 Retrying.
259.9
261.9 --2024-12-23 05:33:10--  (try: 3)  https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb
261.9 Connecting to github.com (github.com)|20.200.245.247|:443... failed: Connection timed out.
391.0 Retrying.
391.0
. . .
2463.6
2473.6 --2024-12-23 06:10:02--  (try:19)  https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb
2473.6 Connecting to github.com (github.com)|20.200.245.247|:443... failed: Connection timed out.
2602.8 Retrying.
2602.8
2612.8 --2024-12-23 06:12:21--  (try:20)  https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb
2612.8 Connecting to github.com (github.com)|20.200.245.247|:443... failed: Connection timed out.
2742.1 Giving up.
2742.1
------

 1 warning found (use docker --debug to expand):
 - JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals (line 47)
Dockerfile_tmp:5
--------------------
   4 |
   5 | >>> RUN wget https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb && \
   6 | >>>     dpkg -i pandoc-3.1.11.1-1-amd64.deb && \
   7 | >>>     rm pandoc-3.1.11.1-1-amd64.deb
   8 |
--------------------
ERROR: failed to solve: process "/bin/sh -c wget https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb &&     dpkg -i pandoc-3.1.11.1-1-amd64.deb &&     rm pandoc-3.1.11.1-1-amd64.deb" did not complete successfully: exit code: 4
yogurt@134servwe:~/cuBB$

when use wget directly.

yogurt@134servwe:~$ wget https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb
--2024-12-23 14:32:55--  https://github.com/jgm/pandoc/releases/download/3.1.11.1/pandoc-3.1.11.1-1-amd64.deb
Connecting to 10.xxx.x.xxx:8080... connected.
Proxy request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/571770/181fb1e9-f1da-4f35-9d85-e063f9b83b6d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241223%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241223T053420Z&X-Amz-Expires=300&X-Amz-Signature=a18ea73cbfad7d9fab96c68c211f249489c2a8247f429406e228dd3ca23c19b9&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dpandoc-3.1.11.1-1-amd64.deb&response-content-type=application%2Foctet-stream [following]
--2024-12-23 14:32:56--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/571770/181fb1e9-f1da-4f35-9d85-e063f9b83b6d?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241223%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241223T053420Z&X-Amz-Expires=300&X-Amz-Signature=a18ea73cbfad7d9fab96c68c211f249489c2a8247f429406e228dd3ca23c19b9&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dpandoc-3.1.11.1-1-amd64.deb&response-content-type=application%2Foctet-stream
Connecting to 10.xxx.x.xxx:8080... connected.
Proxy request sent, awaiting response... 200 OK
Length: 30881308 (29M) [application/octet-stream]
Saving to: ‘pandoc-3.1.11.1-1-amd64.deb.2’
 
pandoc-3.1.11.1-1-amd64.deb.2                           100%[=============================================================================================================================>]  29.45M  2.76MB/s    in 9.1s
 
2024-12-23 14:33:06 (3.25 MB/s) - ‘pandoc-3.1.11.1-1-amd64.deb.2’ saved [30881308/30881308]
 
yogurt@134servwe:~$
yogurt@134servwe:~$ ls -lrt
-rw-rw-r--  1 yogurt yogurt   30881308 Jan  6  2024 pandoc-3.1.11.1-1-amd64.deb
drwxrwxr-x 13 yogurt yogurt       4096 Dec 23 14:28 cuBB
2

I solved the issue by adding proxy option to wget and pip3 command in pyaerial_recipe.py file.
Then, there is an error as shown in the log below.
I added proxy option to pip command in install_dev_pkg.sh file as above problem.
But the problem has not been solved
I ask for your help to resolve the issue below.
thank you.

yogurt@134servwe:~/cuBB$ AERIAL_BASE_IMAGE=nvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran:24-3-cubb $cuBB_SDK/pyaerial/container/build.sh
/home/yogurt/cuBB/pyaerial/container/build.sh starting...
[+] Building 556.1s (13/14)                                                                                                                      docker:default
 => [internal] load build definition from Dockerfile_tmp                                                                                                   0.0s
 => => transferring dockerfile: 1.70kB                                                                                                                     0.0s
 => WARN: JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals (line 47)                           0.0s
 => [internal] load metadata for nvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran:24-3-cubb                                                                0.0s
 => [internal] load .dockerignore                                                                                                                          0.0s
 => => transferring context: 2B                                                                                                                            0.0s
 => [ 1/10] FROM nvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran:24-3-cubb                                                                                0.0s
 => [internal] load build context                                                                                                                          0.0s
 => => transferring context: 44B                                                                                                                           0.0s
 => CACHED [ 2/10] RUN wget --no-check-certificate -e use_proxy=yes -e https_proxy=http://10.xxx.x.xxx:8080/ https://github.com/jgm/pandoc/releases/downl  0.0s
 => CACHED [ 3/10] RUN apt-get update -y &&     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends         python3-pip         pyt  0.0s
 => CACHED [ 4/10] COPY requirements-amd64.txt /var/tmp/requirements-amd64.txt                                                                             0.0s
 => CACHED [ 5/10] RUN pip3 --proxy http://10.xxx.x.xxx:8080 --trusted-host pypi.org --trusted-host files.pythonhosted.org install -r /var/tmp/requiremen  0.0s
 => CACHED [ 6/10] RUN apt-get update -y &&     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends         python3-pip         pyt  0.0s
 => CACHED [ 7/10] RUN apt-get update -y &&     DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends         python3-pip         pyt  0.0s
 => CACHED [ 8/10] RUN pip3 --proxy http://10.xxx.x.xxx:8080 --trusted-host pypi.org --trusted-host files.pythonhosted.org install nvidia-cudnn-cu12==8.9  0.0s
 => ERROR [ 9/10] RUN cmake -Bbuild -GNinja --log-level=warning -DNVIPC_FMTLOG_ENABLE=OFF -DCMAKE_TOOLCHAIN_FILE=cuPHY/cmake/toolchains/native &&     c  556.1s
------
 > [ 9/10] RUN cmake -Bbuild -GNinja --log-level=warning -DNVIPC_FMTLOG_ENABLE=OFF -DCMAKE_TOOLCHAIN_FILE=cuPHY/cmake/toolchains/native &&     cmake --build build -t _pycuphy pycuphycpp &&     ./pyaerial/scripts/install_dev_pkg.sh:
0.137 CMake Warning at CMakeLists.txt:48 (message):
0.137   Setting CMAKE_BUILD_TYPE to 'Release' as none was specified.
.
.
.
161.5 [153/156] Linking CXX static library cuPHY/src/cuphy_hdf5/libcuphy_hdf5.a
161.6 [154/156] Linking CXX static library cuPHY/src/cuphy_channels/libcuphy_channels.a
161.8 [155/156] Linking CXX shared library pyaerial/libpycuphycpp.so
183.1 [156/156] Linking CXX shared module pyaerial/_pycuphy.cpython-310-x86_64-linux-gnu.so
183.1 lto-wrapper: warning: using serial compilation of 14 LTRANS jobs
183.1 lto-wrapper: note: see the ‘-flto’ option documentation for more information
183.1 Cleaning up pyaerial output directory
183.1 /opt/nvidia/cuBB/pyaerial/scripts/install_dev_pkg.sh starting...
183.2 * Creating venv isolated environment...
187.5 * Installing packages in isolated environment... (setuptools>=42, wheel)
556.0 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f0d97630850>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/setuptools/
556.0 WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f0d97630a30>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/setuptools/
556.0 WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f0d97630df0>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/setuptools/
556.0 WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f0d97630fa0>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/setuptools/
556.0 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7f0d97631150>: Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/setuptools/
556.0 ERROR: Could not find a version that satisfies the requirement setuptools>=42 (from versions: none)
556.0 ERROR: No matching distribution found for setuptools>=42
556.0
556.0 Traceback (most recent call last):
556.0   File "/usr/local/lib/python3.10/dist-packages/build/__main__.py", line 388, in main
556.0     built = build_call(
556.0   File "/usr/local/lib/python3.10/dist-packages/build/__main__.py", line 239, in build_package_via_sdist
556.0     sdist = _build(isolation, srcdir, outdir, 'sdist', config_settings, skip_dependency_check)
556.0   File "/usr/local/lib/python3.10/dist-packages/build/__main__.py", line 147, in _build
556.0     return _build_in_isolated_env(srcdir, outdir, distribution, config_settings)
556.0   File "/usr/local/lib/python3.10/dist-packages/build/__main__.py", line 113, in _build_in_isolated_env
556.0     env.install(builder.build_system_requires)
556.0   File "/usr/local/lib/python3.10/dist-packages/build/env.py", line 143, in install
556.0     _subprocess(cmd)
556.0   File "/usr/local/lib/python3.10/dist-packages/build/env.py", line 64, in _subprocess
556.0     subprocess.run(cmd, check=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
556.0   File "/usr/lib/python3.10/subprocess.py", line 526, in run
556.0     raise CalledProcessError(retcode, process.args,
556.0 subprocess.CalledProcessError: Command '['/tmp/build-env-94oyqtdp/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/tmp/build-reqs-o17lhwz4.txt']' returned non-zero exit status 1.
556.0
556.0 ERROR Command '['/tmp/build-env-94oyqtdp/bin/python', '-Im', 'pip', 'install', '--use-pep517', '--no-warn-script-location', '-r', '/tmp/build-reqs-o17lhwz4.txt']' returned non-zero exit status 1.
------
 
 1 warning found (use docker --debug to expand):
 - JSONArgsRecommended: JSON arguments recommended for CMD to prevent unintended behavior related to OS signals (line 47)
Dockerfile_tmp:41
--------------------
  40 |
  41 | >>> RUN cmake -Bbuild -GNinja --log-level=warning -DNVIPC_FMTLOG_ENABLE=OFF ${BUILD_ARGS} && \
  42 | >>>     cmake --build build -t _pycuphy pycuphycpp && \
  43 | >>>     ./pyaerial/scripts/install_dev_pkg.sh
  44 |
--------------------
ERROR: failed to solve: process "/bin/sh -c cmake -Bbuild -GNinja --log-level=warning -DNVIPC_FMTLOG_ENABLE=OFF ${BUILD_ARGS} &&     cmake --build build -t _pycuphy pycuphycpp &&     ./pyaerial/scripts/install_dev_pkg.sh" did not complete successfully: exit code: 1
yogurt@134servwe:~/cuBB$
3

Hi @twoheons ,

Can you please check the network connection on your system?

Thank you.

4

I knew that /home/yogurt/cuBB/pyaerial/scripts/install_dev_pkg.sh didn’t have any effect on building pyaerial.
Even though I deleted it, there was no change in the error log.
I copied /home/yogurt/cuBB/pyaerial/scripts/install_dev_pkg.sh to /home/yogurt/cuBB/pyaerial/container/install_dev_pkg.sh.
and I added the below line in pyaerial_recipe.py.

Stage0 += copy(src='install_dev_pkg.sh', dest='/opt/nvidia/cuBB/pyaerial/scripts/install_dev_pkg.sh')

and when I modified echo command line, I could see a change in build log.
As my next step, I added proxy option and PIP_TURSTED_HOST option in /home/yogurt/cuBB/pyaerial/container/install_dev_pkg.sh.

Finally, I built the pyaerial image.

yogurt@134servwe:~/cuBB$ docker images
REPOSITORY                                         TAG                      IMAGE ID       CREATED          SIZE
pyaerial                                           yogurt-24-3-cubb-amd64   d07531869ba5   40 minutes ago   51.4GB
nvcr.io/qhrjhjrvlsbu/aerial-cuda-accelerated-ran   24-3-cubb                7fabfbf4f001   6 weeks ago      36.3GB
hello-world                                        latest                   d2c94e258dcb   20 months ago    13.3kB