TAO GPU Issue

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc)
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc)
• TLT Version (Please run “tlt info --verbose” and share “docker_tag” here)
• Training spec file(If have, please share here)
• How to reproduce the issue ? (This is for errors. Please share the command line and the detailed log here.)

I’m working on TAO inside the ssh server and I get the following error.

2023-07-12 11:03:41,816 [INFO] root: Registry: ['nvcr.io']
Traceback (most recent call last):
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/api/client.py", line 205, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/api/client.py", line 228, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/requests/sessions.py", line 602, in get
    return self.request("GET", url, **kwargs)
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/urllib3/connectionpool.py", line 496, in _make_request
    conn.request(
TypeError: request() got an unexpected keyword argument 'chunked'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/soo/anaconda3/envs/TAO/bin/tao", line 8, in <module>
    sys.exit(main())
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/tlt/entrypoint/tao.py", line 114, in main
    instance.launch_command(
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/tlt/components/instance_handler/local_instance.py", line 297, in launch_command
    docker_handler = self.handler_map[
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/tlt/components/instance_handler/local_instance.py", line 147, in handler_map
    handler_map[handler_key] = DockerHandler(
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/tlt/components/docker_handler/docker_handler.py", line 62, in __init__
    self._docker_client = docker.from_env()
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/client.py", line 84, in from_env
    return cls(
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/client.py", line 40, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/api/client.py", line 188, in __init__
    self._version = self._retrieve_server_version()
  File "/home/soo/anaconda3/envs/TAO/lib/python3.8/site-packages/docker/api/client.py", line 212, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: request() got an unexpected keyword argument 'chunked'

I’ve previously found the same problem and solved it again with ngc login, but it’s not working now.

How should I solve this problem?

I am afraid you were triggering tao docker based on another docker.
In this case, please
add -v /var/run/docker.sock:/var/run/docker.sock

Refer to Train ngc api key setup issuein TESLA V100

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.