Unable to split dataset randomly and generate tfrecords - LPD Model

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

• Hardware Platform : Jetson AGX Xavier
• DeepStream Version: 5.0
• JetPack Version : 4.4[L4T 32.4.3]
• TensorRT Version : 7.1.3.0
• NVIDIA GPU Driver Version : R32 (release), REVISION: 4.3, GCID: 21589087, BOARD: t186ref, EABI: aarch64
**• Issue Type: Unable to split dataset randomly and generate tfrecords - LPD Model
**• How to reproduce the issue ? ** Follow the training method given in the following link :

From the above link go to License plate detection → Training LPD Model
I have followed all the instructions to install for the training, example: Install TLT launcher(docker , nvidia-pyindex , etc.)
But when I run the 3rd command to split dataset randomly and generate tfrecords it is not working.

tlt detectnet_v2 dataset_convert -d /workspace/openalpr/SPECS_tfrecord.txt -o /workspace/openalpr/lpd_tfrecord/lpd

The following is the error displayed after the command is being executed
Error:

tlt detectnet_v2 dataset_convert -d /workspace/openalpr/SPECS_tfrecord.txt -o /workspace/openalpr/lpd_tfrecord/lpd
2021-03-16 10:44:38,995 [INFO] tlt.components.docker_handler.docker_handler: The required docker doesn’t exist locally/the manifest has changed. Pulling a new docker.
2021-03-16 10:44:38,995 [INFO] tlt.components.docker_handler.docker_handler: Pulling the required container. This may take several minutes if you’re doing this for the first time. Please wait here.
…
2021-03-16 10:54:06,275 [INFO] tlt.components.docker_handler.docker_handler: Container pull complete.
Docker instantiation failed with error: 404 Client Error: Not Found (“no such image: nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3: No such image: nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3”)

So I unable to proceed the further steps to complete the training

Can someone help me to solve this issue

5

Moving this topic from Deepstream to TLT forum.

6

I’m afraid there is still something wrong in your tlt launcher environment.
To narrow down, can you run following command in your host PC?
tlt detectnet_v2 run cat /workspace/openalpr/SPECS_tfrecord.txt

7

Hi Morganh

I ran the command as you said the following is the output:

tlt detectnet_v2 run cat /workspace/openlpr/SPECS_tfrecord.txt

2021-03-18 09:38:27,503 [INFO] tlt.components.docker_handler.docker_handler: The required docker doesn’t exist locally/the manifest has changed. Pulling a new docker.
2021-03-18 09:38:27,504 [INFO] tlt.components.docker_handler.docker_handler: Pulling the required container. This may take several minutes if you’re doing this for the first time. Please wait here.
…
2021-03-18 09:50:19,648 [INFO] tlt.components.docker_handler.docker_handler: Container pull complete.
Docker instantiation failed with error: 404 Client Error: Not Found (“no such image: nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3: No such image: nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3”)

8

Please run tlt info

9 
tlt info

Configuration of the TLT Instance
dockers: [‘nvcr.io/nvidia/tlt-streamanalytics’, ‘nvcr.io/nvidia/tlt-pytorch’]
format_version: 1.0
tlt_version: 3.0
published_date: 02/02/2021

10

Can you find the docker in your host PC?

docker images |grep v3.0-dp-py3

11

No it didn’t show anything

12

So, that is the root cause.
Please double check the steps when install TLT launcher.

13

Is that necessary to create new virtualenv using virtualenvwrapper

14

Not necessary.

15

Have you run https://docs.nvidia.com/metropolis/TLT/tlt-user-guide/text/requirements_and_installation.html#installation-prerequisites ?

Perform the following prerequisite steps before installing TLT:

  1. Install Docker.
  2. Install NVIDIA GPU driver v455.xx or above.
  3. Install nvidia docker2
  4. Get an NGC account and API key:
  5. Go to NGC and click the Transfer Learning Toolkit container in the Catalog tab. This message is displayed: “Sign in to access the PULL feature of this repository”.
  6. Enter your Email address and click Next , or click Create an Account .
  7. Choose your organization when prompted for Organization/Team .
  8. Click Sign In .
  9. Execute docker login nvcr.io from the command line and enter these login credentials:
  10. Username: “$oauthtoken”
  11. Password: “YOUR_NGC_API_KEY”

Note

If you have followed the default installation instructions for docker-ce you may need to have sudo access to run docker commands. In order to circumvent this, TLT recommends you to follow these post-installation steps to make sure that the docker commands can be run without sudo.

16

Yes I followed these steps only… but how to pull that particular docker image for

nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3

17

In TLT 3.0, end user need not pull that as we did in TLT 2.0

18

Please run below command to check if it will pull the docker.

(venv_3.0) morganh@dl:~/venv_3.0$ tlt ssd run /bin/bash

19

No, It didn’t pull the docker image

tlt ssd run /bin/bash
2021-03-18 10:51:41,711 [INFO] tlt.components.docker_handler.docker_handler: The required docker doesn’t exist locally/the manifest has changed. Pulling a new docker.
2021-03-18 10:51:41,711 [INFO] tlt.components.docker_handler.docker_handler: Pulling the required container. This may take several minutes if you’re doing this for the first time. Please wait here.
…
2021-03-18 11:02:58,159 [INFO] tlt.components.docker_handler.docker_handler: Container pull complete.
Docker instantiation failed with error: 404 Client Error: Not Found (“no such image: nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3: No such image: nvcr.io/nvidia/tlt-streamanalytics:v3.0-dp-py3”)

20

Have you run

  1. Execute docker login nvcr.io from the command line and enter these login credentials:
  2. Username: “$oauthtoken”
  3. Password: “YOUR_NGC_API_KEY”
21

Yes I did

22

You run without sudo or with sudo?

23

Without sudo