Tao yolov3 custom dataset training, container issue

I am trying to train a tao yolov3 model on a custom dataset. When trying to generate anchor boxes and TFRecords I am faced with a ‘Stopping container’ message.

While trying to generate custom anchor boxes using tao yolo_v3 kmeans -l /labels-path -i /images-path-x 640 -y 640 --min_x 5.54 --min_y 0. I am getting the following error -

2022-07-19 12:36:26,748 [INFO] root: Registry: ['nvcr.io']
2022-07-19 12:36:26,813 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.5-py3
2022-07-19 12:36:26,876 [WARNING] tlt.components.docker_handler.docker_handler: 
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the "user":"UID:GID" in the
DockerOptions portion of the "/home/vast/.tao_mounts.json" file. You can obtain your
users UID and GID by using the "id -u" and "id -g" commands on the
terminal.
2022-07-19 12:36:31,893 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.

While trying to generate the TFRecords using tao yolo_v3 dataset_convert -d /dataspec-path -o /output-file-path. I am getting the following error -

2022-07-19 13:09:19,452 [INFO] root: Registry: ['nvcr.io']
2022-07-19 13:09:19,516 [INFO] tlt.components.instance_handler.local_instance: Running command in container: nvcr.io/nvidia/tao/tao-toolkit-tf:v3.21.11-tf1.15.5-py3
2022-07-19 13:09:19,529 [WARNING] tlt.components.docker_handler.docker_handler: 
Docker will run the commands as root. If you would like to retain your
local host permissions, please add the "user":"UID:GID" in the
DockerOptions portion of the "/home/vast/.tao_mounts.json" file. You can obtain your
users UID and GID by using the "id -u" and "id -g" commands on the
terminal.
Using TensorFlow backend.
2022-07-19 13:09:28,158 [INFO] tlt.components.docker_handler.docker_handler: Stopping container. You can obtain your
users UID and GID by using the "id -u" and "id -g" commands on the
terminal.
2022-07-19 12:36:31,893 [INFO] tlt.components.docker_handler.docker_handler: Stopping container.

My home/user/.tao_mounts.json file looks like this

{
    "Mounts": [
        {
            "source": "/opt/paralaxiom/deeplearninglib/tao",
            "destination": "/workspace/tao-experiments"
        },
        {
            "source": "/opt/paralaxiom/deeplearninglib/tao/yolo_v3/specs",
            "destination": "/workspace/tao-experiments/yolo_v3/specs"
        }
    ]
}

Please refer to solution suggest in below thread for similar issue.

Let us know in case issue persist even after using TAOv22.05.

Thanks

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

@adithya.ajith
Please update tao via following command and check if it works.

$ pip3 install nvidia-tao==0.1.24

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.