Tao-toolkit-triton-apps

i am getting error will running tao-toolkit-triton

have installed pip3 install tao_triton also did pip install -r requirements-pip.txt even then i get below error

> 
> root@eb65e4f2d3a9:/opt/nvidia/deepstream/deepstream-6.0/sources/tao-toolkit-triton-apps# python tao_client.py     /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream-apps/model_repository/Weapon_model/model_b1_gpu0_fp32.engine    -m yolov3_tao   -x 1  -b 1   --mode YOLOv3     -i https   -u localhost:8000  --async        --output_path  /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream-apps/model_repository/Weapon_model/1
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
>   setattr(self, word, getattr(machar, word).flat[0])
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
>   return self._float_to_str(self.smallest_subnormal)
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
>   setattr(self, word, getattr(machar, word).flat[0])
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
>   return self._float_to_str(self.smallest_subnormal)
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 163, in get_socket
>     return self._socket_queue.get(block=False)
>   File "src/gevent/queue.py", line 335, in gevent._gevent_cqueue.Queue.get
>   File "src/gevent/queue.py", line 350, in gevent._gevent_cqueue.Queue.get
>   File "src/gevent/queue.py", line 319, in gevent._gevent_cqueue.Queue._Queue__get_or_peek
>     
> _queue.Empty
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "tao_client.py", line 435, in <module>
>     main()
>   File "tao_client.py", line 246, in main
>     model_metadata = triton_client.get_model_metadata(
>   File "/usr/local/lib/python3.8/dist-packages/tritonclient/http/__init__.py", line 500, in get_model_metadata
>     response = self._get(request_uri=request_uri,
>   File "/usr/local/lib/python3.8/dist-packages/tritonclient/http/__init__.py", line 266, in _get
>     response = self._client_stub.get(request_uri)
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/client.py", line 266, in get
>     return self.request(METHOD_GET, request_uri, headers=headers)
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/client.py", line 226, in request
>     sock = self._connection_pool.get_socket()
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 166, in get_socket
>     return self._create_socket()
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 127, in _create_socket
>     raise first_error
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 114, in _create_socket
>     sock = self._connect_socket(sock, sock_info[-1])
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 136, in _connect_socket
>     sock.connect(address)
>   File "/usr/local/lib/python3.8/dist-packages/gevent/_socketcommon.py", line 607, in connect
>     raise _SocketError(err, strerror(err))
> ConnectionRefusedError: [Errno 111] Connection refused
> root@eb65e4f2d3a9:/opt/nvidia/deepstream/deepstream-6.0/sources/tao-toolkit-triton-apps# python3 tao_triton/python/entrypoints/tao_client.py     /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream-apps/model_repository/Helmet_model/yolov4_resnet18_epoch_050.etlt    -m yolov3_tao   -x 1  -b 1   --mode YOLOv3     -i https   -u localhost:8000  --async   --output_path  /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream-apps/model_repository/Helmet_model/1
> Traceback (most recent call last):
>   File "tao_triton/python/entrypoints/tao_client.py", line 40, in <module>
>     from tao_triton.python.types import Frame, UserData
> ModuleNotFoundError: No module named 'tao_triton'
> root@eb65e4f2d3a9:/opt/nvidia/deepstream/deepstream-6.0/sources/tao-toolkit-triton-apps# python3 tao_client.py     /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream-apps/model_repository/Helmet_model/yolov4_resnet18_epoch_050.etlt    -m yolov3_tao   -x 1  -b 1   --mode YOLOv3     -i https   -u localhost:8000  --async   --output_path  /opt/nvidia/deepstream/deepstream-6.0/sources/deepstream-apps/model_repository/Helmet_model/1
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
>   setattr(self, word, getattr(machar, word).flat[0])
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
>   return self._float_to_str(self.smallest_subnormal)
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
>   setattr(self, word, getattr(machar, word).flat[0])
> /usr/local/lib/python3.8/dist-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
>   return self._float_to_str(self.smallest_subnormal)
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 163, in get_socket
>     return self._socket_queue.get(block=False)
>   File "src/gevent/queue.py", line 335, in gevent._gevent_cqueue.Queue.get
>   File "src/gevent/queue.py", line 350, in gevent._gevent_cqueue.Queue.get
>   File "src/gevent/queue.py", line 319, in gevent._gevent_cqueue.Queue._Queue__get_or_peek
>     
> _queue.Empty
> 
> During handling of the above exception, another exception occurred:
> 
> Traceback (most recent call last):
>   File "tao_client.py", line 435, in <module>
>     main()
>   File "tao_client.py", line 246, in main
>     model_metadata = triton_client.get_model_metadata(
>   File "/usr/local/lib/python3.8/dist-packages/tritonclient/http/__init__.py", line 500, in get_model_metadata
>     response = self._get(request_uri=request_uri,
>   File "/usr/local/lib/python3.8/dist-packages/tritonclient/http/__init__.py", line 266, in _get
>     response = self._client_stub.get(request_uri)
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/client.py", line 266, in get
>     return self.request(METHOD_GET, request_uri, headers=headers)
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/client.py", line 226, in request
>     sock = self._connection_pool.get_socket()
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 166, in get_socket
>     return self._create_socket()
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 127, in _create_socket
>     raise first_error
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 114, in _create_socket
>     sock = self._connect_socket(sock, sock_info[-1])
>   File "/usr/local/lib/python3.8/dist-packages/geventhttpclient/connectionpool.py", line 136, in _connect_socket
>     sock.connect(address)
>   File "/usr/local/lib/python3.8/dist-packages/gevent/_socketcommon.py", line 607, in connect
>     raise _SocketError(err, strerror(err))
> ConnectionRefusedError: [Errno 111] Connection refused
> root@eb65e4f2d3a9:/opt/nvidia/deepstream/deepstream-6.0/sources/tao-toolkit-triton-apps#

Could you share more steps you have done?

More, are you running inside one docker? Could you try to run outside the docker?

yes inside the docker nvcr.io/nvidia/deepstream:6.0-triton

did git pull of GitHub - NVIDIA-AI-IOT/tao-toolkit-triton-apps: Sample app code for deploying TAO Toolkit trained models to Triton
and running from the corresponding folder

could you send me the steps to run ouside the docker .

Just follow GitHub - NVIDIA-AI-IOT/tao-toolkit-triton-apps: Sample app code for deploying TAO Toolkit trained models to Triton

python3 tao_client.py …/…/…/…/…/deepstream-apps/model_repository/Weapon_model/model_b1_gpu0_fp32.engine -m yolov3_tao -x 1 -b 1 --mode YOLOv3 -i https -u localhost:8000 --async --output_path …/…/…/…/…/deepstream-apps/model_repository/Weapon_model/1
failed to retrieve the metadata: Request for unknown model: ‘yolov3_tao’ is not found

Did you trigger triton server?
$ bash scripts/start_server.sh

ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$ bash scripts/start_server.sh
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get “https://nvcr.io/v2/”: unauthorized: authentication required
ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$

ubuntu@ip-172-31-11-102:~/.docker$ cat config.json
{
“auths”: {
nvcr.io”: {
“auth”: “JG9hdXRodG9rZW46TVRsdWFqVnRZelptTlROMU5UazBNelJoWW05dmJqbG9hMms2T1dNM01qaGtZakl0TVRBd055MDBObVUzTFRnek56TXRZelJqTURNeE1qUmlZekZs”
}
}
}ubuntu@ip-172-31-11-102:~/.docker$ cd ~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps/scripts
ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps/scripts$ ./start_server.sh --password=JG9hdXRodG9rZW46TVRsdWFqVnRZelptTlROMU5UazBNelJoWW05dmJqbG9hMms2T1dNM01qaGtZakl0TVRBd055MDBObVUzTFRnek56TXRZelJqTURNeE1qUmlZekZs
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get “https://nvcr.io/v2/”: unauthorized: authentication required

Could you
$ docker login nvcr.io

ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$ docker login nvcr.io
Authenticating with existing credentials…
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
docker login | Docker Documentation

Login Succeeded
ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$ bash scripts/start_server.sh
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get “https://nvcr.io/v2/”: unauthorized: authentication required
ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$

See tao-toolkit-triton-apps/start_server.sh at main · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub

Could you try to run with this line ?

ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps/scripts$ docker login nvcr.io -u $oauthtoken
Password:
Error response from daemon: Get “https://nvcr.io/v2/”: unauthorized: authentication required

what would be the password ??

i tried below one as well .
docker login nvcr.io -u $oauthtoken -p ${NGC_API_KEY}
flag needs an argument: ‘p’ in -p
See ‘docker login --help’.

The passwd is your ngc_api_key.

Please find what your key is. Then run following command. Do not change “$oauthtoken”.

docker login nvcr.io --username=\$oauthtoken --password=“your_ngc_key”

ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps/scripts$ docker login nvcr.io --username=$oauthtoken --password=*****
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
docker login | Docker Documentation

Login Succeeded
ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps/scripts$

i got login succeeded and it came out

You can retry below after modifying tao-toolkit-triton-apps/start_server.sh at main · NVIDIA-AI-IOT/tao-toolkit-triton-apps · GitHub with explicit key.
$ bash scripts/start_server.sh

ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$ docker login nvcr.io -u $oauthtoken -p *****
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
docker login | Docker Documentation

Login Succeeded
ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$ bash scripts/start_server.sh
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
Error response from daemon: Get “https://nvcr.io/v2/”: unauthorized: authentication required
ubuntu@ip-172-31-11-102:~/sriharsha/Triton/code/tao/tao-toolkit-triton-apps$

again i get error

Did you modify line42?

@h9945394143 Is this still an issue to support? Any result to share?

its been fixed . thank you

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.