Tao_tensorflow1_backend Docker container error

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hardware: T4
Network Type: Detectnet_v2

Hi! I am trying to debug my model and run postprocessing on model outputs from an onnx model. Here is my original, post: Onnx post-processing with trafficcamnet

I tried instantiating the devlopment container within tao_tensorflow1_backend with the following command:

tao_tf --gpus all

I received the following error:

#0 230.6 Successfully built addict future nvidia_dali_tf_plugin_cuda110 mpi4py posix_ipc prettytable pycuda retrying seaborn semver shapely fire antlr4-python3-runtime promise pathtools
#0 230.6 Failed to build h5py pycocotools-fix
#0 230.6 ERROR: Could not build wheels for h5py, pycocotools-fix, which is required to install pyproject.toml-based projects
------
Dockerfile:58
--------------------
  57 |     COPY docker/requirements-pip.txt /workspace/
  58 | >>> RUN pip install -r requirements-pip.txt --use-deprecated=legacy-resolver && \
  59 | >>>     rm -rf requirements-pip.txt
  60 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip install -r requirements-pip.txt --use-deprecated=legacy-resolver &&     rm -rf requirements-pip.txt" did not complete successfully: exit code: 1
Traceback (most recent call last):
  File "/home/ubuntu/tao_tensorflow1_backend/runner/tao_tf.py", line 113, in create_base_docker
    subprocess.run(create_command, stdout=sys.stderr, shell=True, check=True)
  File "/usr/lib/python3.8/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'bash /home/ubuntu/tao_tensorflow1_backend/docker/build.sh --build' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/tao_tensorflow1_backend/runner/tao_tf.py", line 279, in <module>
    main(sys.argv[1:])
  File "/home/ubuntu/tao_tensorflow1_backend/runner/tao_tf.py", line 265, in main
    create_base_docker()
  File "/home/ubuntu/tao_tensorflow1_backend/runner/tao_tf.py", line 115, in create_base_docker
    raise RuntimeError(f"Container build failed with error {e}")
RuntimeError: Container build failed with error Command 'bash /home/ubuntu/tao_tensorflow1_backend/docker/build.sh --build' returned non-zero exit status 1.

Is there an updated Dockerfile? How can I run the container?

Thanks

3

Please double check the requirement mentioned in GitHub - NVIDIA/tao_tensorflow1_backend: TAO Toolkit deep learning networks with TensorFlow 1.x backend.
You can use conda to generate a virtual python environment.

4

I double checked the requirements. My environment matches the requirements. I removed this line from the requirements, and was able to build the docker environment

pycocotools-fix==2.0.0.9
5

Thanks for the info. Will also check further.