Please provide the following information when requesting support.
• Hardware (Ubuntu 18.04 , dGPU RTX2080)
• Network Type (Classification)
• TAO Version (format_version: 2.0
toolkit_version: 3.21.11
)
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)
Hi guys, I need some help.
In order to do classification training, I can only use this image “nvcr.io/nvidia/tao/tao-toolkit-tf v3.21.11-tf1.15.5-py3”? and not other older version?
I have installed nvidia-tlt, yet the toolkit version is 3.21.11.
tlt info --verbose
/home/sapphire/.virtualenvs/launcher/lib/python3.6/site-packages/tlt/init.py:20: DeprecationWarning:
The nvidia-tlt package will be deprecated soon. Going forward please migrate to using the nvidia-tao package.
warnings.warn(message, DeprecationWarning)
Configuration of the TAO Toolkit Instance
Now i am facing another issue. After executing the command"classification" inside the docker container, i had the error below:
root@bf4443e0d0f0:/workspace# classification
Using TensorFlow backend.
2022-03-10 03:39:05,810 [WARNING] modulus.export._tensorrt: Failed to import TRT and/or CUDA. TensorRT optimization and inference will not be available.
Traceback (most recent call last):
File “/usr/local/bin/classification”, line 8, in
sys.exit(main())
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/entrypoint/makenet.py”, line 12, in main
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/entrypoint/entrypoint.py”, line 256, in launch_job
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/entrypoint/entrypoint.py”, line 47, in get_modules
File “/usr/lib/python3.6/importlib/init.py”, line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File “”, line 994, in _gcd_import
File “”, line 971, in _find_and_load
File “”, line 955, in _find_and_load_unlocked
File “”, line 665, in _load_unlocked
File “”, line 678, in exec_module
File “”, line 219, in _call_with_frames_removed
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/scripts/export.py”, line 8, in
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/makenet/export/classification_exporter.py”, line 11, in
File “/root/.cache/bazel/_bazel_root/ed34e6d125608f91724fda23656f1726/execroot/ai_infra/bazel-out/k8-fastbuild/bin/magnet/packages/iva/build_wheel.runfiles/ai_infra/iva/common/export/keras_exporter.py”, line 22, in
ImportError: cannot import name ‘ONNXEngineBuilder’
root@bf4443e0d0f0:/workspace# " docker run -it --rm nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.08-py3 /bin/bash"
Ok, it is working now!! since I am doing it this way, do i still need to follow the ~/.tao_mounts.json setup for the launcher instance? or do I need to setup the mounting/binding manually myself?