Hello,
I tried to use the TAO Toolkit via Python Wheel and trained YOLOv3 on Train on Colab. However, I encountered an error that prevented me from training: “NotFoundError: /usr/local/lib/python3.8/dist-packages/nvidia_tao_tf1/core/processors/…/lib/op_values_and_count_to_sparse_tensor.so: cannot open shared object file: No such file or directory.” I would like to know how to resolve this issue.
Currently, there is an issue when train YOLO network in Colab since Colab is using Ubuntu 22.04 instead of 20.04. We are still working on that.
I see. So, training cannot be conducted on Colab at the moment. Understood.
Are there any other methods that are currently not usable?
I would like to know a method that can definitely be used for training.
You can run in a local dgpu machine or a remote cloud machine(Running TAO Toolkit in the Cloud - NVIDIA Docs).
Which of the four methods introduced in the TAO Toolkit introductory guide does running the TAO Toolkit in the cloud correspond to?
For example, running in AWS, please refer to Running TAO Toolkit on an AWS VM - NVIDIA Docs.
thank you. i will try.
I was trying another method first because the method using AWS required creating an account.
tao model yolo_v3 kmeans -l /workspace/tao-experiments/labels -i /workspace/tao-experiments/images -n 3 -x 512 -y 512
When I executed this code, an error occurred
AssertionError: Must have more boxes than clusters. I will also attach some of the dataset labels.
WIN_20230727_11_16_49_Pro.txt (266 Bytes)
The label file is not expected. You can refer to tao_tutorials/notebooks/tao_launcher_starter_kit/detectnet_v2 at main · NVIDIA/tao_tutorials · GitHub or Data Annotation Format - NVIDIA Docs.
-l specifies the label folder, doesn’t it?
Does this mean the format of the label folder is incorrect?
The format of label is expected to be KITTI format.
For example,
car 0.00 0 0.00 587.01 173.33 614.12 200.12 0.00 0.00 0.00 0.00 0.00 0.00 0.00
See more in Data Annotation Format - NVIDIA Docs
Is it because the number of elements in the label is small? I prepared the dataset using LabelImg.
No, it is expected to be 15 fields.
Please check Data Annotation Format - NVIDIA Docs.
Will this element be created manually? Please let me know if there is an appropriate tool.
You can take a look at Is it possible to generate .tfrecorfs for tlt training directly without using intermediate kitti format? - #2 by Morganh.
Is the file specified for <dataset_spec> in dataset_convert in the format of dataset_config?
There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks
No, you can refer to tao_tutorials/notebooks/tao_launcher_starter_kit/detectnet_v2/specs/detectnet_v2_tfrecords_kitti_trainval.txt at main · NVIDIA/tao_tutorials · GitHub.
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.