Bad results, while running inference on the pretrained Image Classification models

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (Quadro GV100)
• Network Type Image Classification
• How to reproduce the issue ? Run the inference on the pre-trained image classification models.

I want to use the pretrained image classification models as it is and see their performances on the embedded platforms. The models that I used are present here. Before deployment I want to test that the models are working well in Python, so I download these models and try to run simple inference. Upon some investigations I found that these models are trained on a subset of the open images dataset (only 176 classes are used). I load a jpg image, I perform the imagenet normalization. I.e. scale the image by dividing the pixel values by 255, and then subtract the mean and divide by the variance. Standard PyTorch way. When I run the inference the results are almost always wrong, although the provided table suggest that the accuracies are +70% on almost all models. The class labels that I am using are
labels.txt (1.5 KB). Could you please help me to understand or fix this? Thank you.

2

Could you run inference with tao classification_tf1 inference?
Refer to tao_tutorials/notebooks/tao_launcher_starter_kit/classification_tf1/tao_voc/classification.ipynb at main · NVIDIA/tao_tutorials · GitHub.

3

I tried and I have the following error.

I used resnet_18.hdf5 for the inference. I used the command:

tao model classification_tf1 inference -e ./classification_spec.cfg -m resnet_18.hdf5 -cm classmap.json

The classification_spec.cfg I used is attached here (I added .txt to be able to upload it).
classification_spec.cfg.txt (401 Bytes)
The classmap is here (again I added .txt to be able to upload).
classmap.json.txt (2.8 KB)

The error I got is like this:

tao model classification_tf1 inference -e ./classification_spec.cfg -m resnet_18.hdf5 -cm classmap.json>log.txt
2024-10-10 16:11:11,784 [TAO Toolkit] [INFO] root 160: Registry: [‘nvcr.io’]
2024-10-10 16:11:11,928 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5
2024-10-10 16:11:12,006 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True
2024-10-10 16:11:26,498 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.
(tao_launcher_v5) shahnawm@parcxd3883:~/Documents/tao_classification_test$ tao model classification_tf1 inference -e ./classification_spec.cfg -m resnet_18.hdf5 -cm classmap.json
2024-10-10 16:16:53,128 [TAO Toolkit] [INFO] root 160: Registry: [‘nvcr.io’]
2024-10-10 16:16:53,277 [TAO Toolkit] [INFO] nvidia_tao_cli.components.instance_handler.local_instance 360: Running command in container: nvcr.io/nvidia/tao/tao-toolkit:5.0.0-tf1.15.5
2024-10-10 16:16:53,370 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 301: Printing tty value True
Using TensorFlow backend.
2024-10-10 14:16:55.553736: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-10-10 14:16:55,649 [TAO Toolkit] [WARNING] tensorflow 40: Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
2024-10-10 14:16:57,738 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
2024-10-10 14:16:57,817 [TAO Toolkit] [WARNING] tensorflow 42: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
2024-10-10 14:16:57,825 [TAO Toolkit] [WARNING] tensorflow 43: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:150: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_hue(img, max_delta=10.0):
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:173: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_saturation(img, max_shift):
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:183: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_contrast(img, center, max_contrast_scale):
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:192: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_shift(x_img, shift_stddev):
2024-10-10 14:17:01.161419: I tensorflow/core/platform/profile_utils/cpu_utils.cc:109] CPU Frequency: 2200000000 Hz
2024-10-10 14:17:01.163812: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x88bb6b0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2024-10-10 14:17:01.163877: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2024-10-10 14:17:01.167872: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcuda.so.1
2024-10-10 14:17:01.266137: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x841cee0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2024-10-10 14:17:01.266206: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Quadro GV100, Compute Capability 7.0
2024-10-10 14:17:01.266988: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1674] Found device 0 with properties:
name: Quadro GV100 major: 7 minor: 0 memoryClockRate(GHz): 1.627
pciBusID: 0000:2d:00.0
2024-10-10 14:17:01.267090: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-10-10 14:17:01.267281: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcublas.so.12
2024-10-10 14:17:01.271689: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcufft.so.11
2024-10-10 14:17:01.271943: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcurand.so.10
2024-10-10 14:17:01.278375: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusolver.so.11
2024-10-10 14:17:01.280689: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcusparse.so.12
2024-10-10 14:17:01.280857: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudnn.so.8
2024-10-10 14:17:01.281657: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1802] Adding visible gpu devices: 0
2024-10-10 14:17:01.281765: I tensorflow/stream_executor/platform/default/dso_loader.cc:50] Successfully opened dynamic library libcudart.so.12
2024-10-10 14:17:01.296225: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1214] Device interconnect StreamExecutor with strength 1 edge matrix:
2024-10-10 14:17:01.296282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1220] 0
2024-10-10 14:17:01.296301: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1233] 0: N
2024-10-10 14:17:01.297501: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1359] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 21242 MB memory) → physical GPU (device: 0, name: Quadro GV100, pci bus id: 0000:2d:00.0, compute capability: 7.0)
Using TensorFlow backend.
WARNING: Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them.
WARNING: TensorFlow will not use sklearn by default. This improves performance in some cases. To enable sklearn export the environment variable TF_ALLOW_IOLIBS=1.
WARNING: TensorFlow will not use Dask by default. This improves performance in some cases. To enable Dask export the environment variable TF_ALLOW_IOLIBS=1.
WARNING: TensorFlow will not use Pandas by default. This improves performance in some cases. To enable Pandas export the environment variable TF_ALLOW_IOLIBS=1.
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:150: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_hue(img, max_delta=10.0):
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:173: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_saturation(img, max_shift):
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:183: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_contrast(img, center, max_contrast_scale):
/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/utils/helper.py:192: NumbaDeprecationWarning: The ‘nopython’ keyword argument was not supplied to the ‘numba.jit’ decorator. The implicit default value for this argument is currently False, but it will be changed to True in Numba 0.59.0. See Deprecation Notices — Numba 0+untagged.871.g53e976f.dirty documentation for details.
def random_shift(x_img, shift_stddev):
Traceback (most recent call last):
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/inference.py”, line 374, in
main()
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py”, line 717, in return_func
raise e
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/common/utils.py”, line 705, in return_func
return func(*args, **kwargs)
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/inference.py”, line 370, in main
raise e
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/inference.py”, line 358, in main
inference(args)
File “/usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/cv/makenet/scripts/inference.py”, line 188, in inference
s_logger.write(
UnboundLocalError: local variable ‘s_logger’ referenced before assignment
Telemetry data couldn’t be sent, but the command ran successfully.
[WARNING]:
Execution status: FAIL
2024-10-10 16:17:08,664 [TAO Toolkit] [INFO] nvidia_tao_cli.components.docker_handler.docker_handler 363: Stopping container.

4

Could you set ./classification_spec.cfg to an absolute path and retry?

More, to narrow down, you can open a terminal and run inside the docker.
$ tao model classification_tf1 run /bin/bash
then inside the docker, run
# classification_tf1 inference xxx

5

More, please run official notebook to get familiar with it. tao_tutorials/notebooks/tao_launcher_starter_kit/classification_tf1/tao_voc/classification.ipynb at main · NVIDIA/tao_tutorials · GitHub.

6

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks