Tlt augment not working

I am trying to run the tlt augment command inside the tlt container, but got the following error:

Command used to run the container is :
sudo docker run --gpus all -it -v /workspace/tlt-experiments/:/workspace/tlt-experiments -p 8888:8888 nvcr.io/nvidia/tlt-streamanalytics:v3.0-py3 /bin/bash

root@1833e5376a0d:/workspace/tlt-experiments# tlt augment -d /workspace/tlt-experiments/data/training/ -a /workspace/tlt-experiments/specs/default_spec.txt -o /workspace/tlt-experiments/augment -v
2021-07-03 13:29:50,128 [INFO] root: Registry: [‘nvcr.io’]
Traceback (most recent call last):
** File “/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py”, line 677, in urlopen**
** chunked=chunked,**
** File “/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py”, line 392, in _make_request**
** conn.request(method, url, httplib_request_kw)
** File “/usr/lib/python3.6/http/client.py”, line 1281, in request**
** self._send_request(method, url, body, headers, encode_chunked)**
** File “/usr/lib/python3.6/http/client.py”, line 1327, in _send_request**
** self.endheaders(body, encode_chunked=encode_chunked)**
** File “/usr/lib/python3.6/http/client.py”, line 1276, in endheaders**
** self._send_output(message_body, encode_chunked=encode_chunked)**
** File “/usr/lib/python3.6/http/client.py”, line 1042, in _send_output**
** self.send(msg)**
** File “/usr/lib/python3.6/http/client.py”, line 980, in send**
** self.connect()**
** File “/usr/local/lib/python3.6/dist-packages/docker/transport/unixconn.py”, line 43, in connect**
** sock.connect(self.unix_socket)**
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
** File “/usr/local/lib/python3.6/dist-packages/requests/adapters.py”, line 449, in send**
** timeout=timeout**
** File “/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py”, line 727, in urlopen**
** method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]**
** File “/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py”, line 403, in increment**
** raise six.reraise(type(error), error, _stacktrace)**
** File “/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py”, line 734, in reraise**
** raise value.with_traceback(tb)**
** File “/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py”, line 677, in urlopen**
** chunked=chunked,**
** File “/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py”, line 392, in _make_request**
** conn.request(method, url, httplib_request_kw)
** File “/usr/lib/python3.6/http/client.py”, line 1281, in request**
** self._send_request(method, url, body, headers, encode_chunked)**
** File “/usr/lib/python3.6/http/client.py”, line 1327, in _send_request**
** self.endheaders(body, encode_chunked=encode_chunked)**
** File “/usr/lib/python3.6/http/client.py”, line 1276, in endheaders**
** self._send_output(message_body, encode_chunked=encode_chunked)**
** File “/usr/lib/python3.6/http/client.py”, line 1042, in _send_output**
** self.send(msg)**
** File “/usr/lib/python3.6/http/client.py”, line 980, in send**
** self.connect()**
** File “/usr/local/lib/python3.6/dist-packages/docker/transport/unixconn.py”, line 43, in connect**
** sock.connect(self.unix_socket)**
urllib3.exceptions.ProtocolError: (‘Connection aborted.’, FileNotFoundError(2, ‘No such file or directory’))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
** File “/usr/local/lib/python3.6/dist-packages/docker/api/client.py”, line 205, in _retrieve_server_version**
** return self.version(api_version=False)[“ApiVersion”]**
** File “/usr/local/lib/python3.6/dist-packages/docker/api/daemon.py”, line 181, in version**
** return self._result(self._get(url), json=True)**
** File “/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py”, line 46, in inner**
** return f(self, args, kwargs)
** File “/usr/local/lib/python3.6/dist-packages/docker/api/client.py”, line 228, in _get
*
** return self.get(url, self._set_request_timeout(kwargs))
** File “/usr/local/lib/python3.6/dist-packages/requests/sessions.py”, line 543, in get**
** return self.request(‘GET’, url, kwargs)
** File “/usr/local/lib/python3.6/dist-packages/requests/sessions.py”, line 530, in request**
** resp = self.send(prep, send_kwargs)
** File “/usr/local/lib/python3.6/dist-packages/requests/sessions.py”, line 643, in send**
** r = adapter.send(request, kwargs)
** File “/usr/local/lib/python3.6/dist-packages/requests/adapters.py”, line 498, in send**
** raise ConnectionError(err, request=request)**
requests.exceptions.ConnectionError: (‘Connection aborted.’, FileNotFoundError(2, ‘No such file or directory’))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
** File “/usr/local/bin/tlt”, line 8, in **
** sys.exit(main())**
** File “/usr/local/lib/python3.6/dist-packages/tlt/entrypoint/entrypoint.py”, line 114, in main**
** args[1:]**
** File “/usr/local/lib/python3.6/dist-packages/tlt/components/instance_handler/local_instance.py”, line 259, in launch_command**
** docker_handler = self.handler_map[**
** File “/usr/local/lib/python3.6/dist-packages/tlt/components/instance_handler/local_instance.py”, line 114, in handler_map**
** docker_mount_file=os.getenv(“LAUNCHER_MOUNTS”, DOCKER_MOUNT_FILE)**
** File “/usr/local/lib/python3.6/dist-packages/tlt/components/docker_handler/docker_handler.py”, line 47, in init**
** self._docker_client = docker.from_env()**
** File “/usr/local/lib/python3.6/dist-packages/docker/client.py”, line 85, in from_env**
** timeout=timeout, version=version, kwargs_from_env(kwargs)
** File “/usr/local/lib/python3.6/dist-packages/docker/client.py”, line 40, in init

** self.api = APIClient(args, kwargs)
** File “/usr/local/lib/python3.6/dist-packages/docker/api/client.py”, line 188, in init
*
** self._version = self._retrieve_server_version()**
** File “/usr/local/lib/python3.6/dist-packages/docker/api/client.py”, line 213, in _retrieve_server_version**
** ‘Error while fetching server API version: {0}’.format(e)**
docker.errors.DockerException: Error while fetching server API version: (‘Connection aborted.’, FileNotFoundError(2, ‘No such file or directory’))

Content inside the /workspace/tlt-experiments/ directory is as follows:
root@1833e5376a0d:/workspace/tlt-experiments# ls
augment.ipynb data specs

Specification file content is:
root@1833e5376a0d:/workspace/tlt-experiments# cat specs/default_spec.txt **
# Spec file for tlt-augment.
spatial_config{
** rotation_config{

** angle: 10.0**
** units: “degrees”**
** }**
}
dataset_config{
** image_path: “image_2”**
** label_path: “label_2”**
}
color_config{
** hue_saturation_config{**
** hue_rotation_angle: 5.0**
** saturation_shift: 1.0**
** }**
}
output_image_width: 1248
output_image_height: 384
output_image_channel: 3

and data is as follows:
image_extension: ".png"root@1833e5376a0d:/workspace/tlt-experiments# ls data/
testing training
root@1833e5376a0d:/workspace/tlt-experiments# ls data/training/
image_2 label_2

currently i am just trying to run the augmentation using tlt on the dataset

I am afraid you were triggering tlt docker based on one docker.
In this case, please
add -v /var/run/docker.sock:/var/run/docker.sock

Reference: Run TLT inside docker - #8 by Morganh

1 Like

Thank you, it worked.

Hi, is there any tlt task to pass the augmented and the training data together, or we need to use manual way of moving to dateset using a script?

In object detection networks, the training can do augmentation if end user enable it in training spec. There is not any manual work.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.