Chmod: cannot access '/opt/ngccli/ngc': No such file or directory

[Dustin.Webb]

Executing the following command:
docker run -it --rm --network=host nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.4-py3 /bin/bash

results in the error:
chmod: cannot access '/opt/ngccli/ngc': No such file or directory

The docker image is downloaded and then https://ngc.nvidia.com/downloads/ngccli_reg_linux.zip is downloaded and extracted. It’s at the end of the extraction that the error occurs.

Until recently the command would start the container and execute /bin/bash. Has something changed recently with ngccli_reg_linux.zip?

[Morganh]

Yes, there is a update from ngccli which results in the issue when trigger TAO docker. We’re unable to install the CLI via the entrypoint since the relative path of the binary in the zip file has changed. Internally team will update the launcher with a fix. Currently, please use below workarounds.

Please refer to below 1st workaround.
Just add this: --entrypoint ""

For example,
$ docker run --runtime=nvidia -it --rm --entrypoint "" nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.4-py3 /bin/bash

The 2nd workaround is for uses who want to use tao launcher instead of “docker run”.
Step:

1. Add "entrypoint": "" to ~/.tao_mounts.json

    "DockerOptions":{
          "entrypoint": "" ,
          "shm_size": "16G",

2. Modify lib/python3.6/site-packages/tao/components/docker_handler/docker_handler.py . This file should be available when you install nvidia-tao.

VALID_DOCKER_ARGS = [“user”, “ports”, “shm_size”, “ulimits”, “privileged”, “network”]

to

VALID_DOCKER_ARGS = [“user”, “ports”, “shm_size”, “ulimits”, “privileged”, “network”, “entrypoint”]

[Morganh]

UPDATE:
The new version of the wheel has already been released to PyPI.
nvidia-tao==0.1.24