Custom ssd-mobilenet.onnx model not building .engine file - slow to initialise every time

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

G’Day Nvidia team,

I have just built a custom model using Dusty’s Hello AI tutorials + Jetson-inference and its’ slow to initialise every time (not just the first time).
When I run detectnet csi://0 an engine is built within data/networks/SSD-mobilenet-v2.
However, when I run detectnet --model=models/targets/ssd-mobilenet.onnx --labels=models/targets/labels.txt --input-blob=input_0 --output-cvg=scores --output-bbox=boxes 0 an engine is not built and thus, it is slow to initialise this code every time.

I have tried deleting the .engine file, but a new one is not created when I run the custom model. I have read and write access to the directories.

I will take any advice you have for me.

Cheers,

Alan

Further info incase it’s relevant:
I built the project from source, but trained within the container (because I can’t get train_ssd.py to work natively).

My current config:
Jetson Nano 4GB - 64GB SD card - powered through micro-USB.
Jetpack 4.6.2 (L4T 32.7.2)
Python 3.6.9
4GB swap loaded as per NVIDIA Nano intro tutorial.

2

Hi @sanespeed, after you run detectnet, is there any .engine file in your models/targets/ directory?

In the console log, it should have printed out a message like completed saving engine cache to <FILE>

If not, can you post the console log here please? Thanks.

4

Hey @dusty_nv !

Yes, there is a .engine file under models/targets, but it seems it’s not being used when it’s re-run?

When running classic detectnet:
[TRT] device GPU, completed building CUDA engine
[TRT] network profiling complete, saving engine cache to /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.8201.GPU.FP16.engine
[TRT] device GPU, completed saving engine cache to /usr/local/bin/networks/SSD-Mobilenet-v2/ssd_mobilenet_v2_coco.uff.1.1.8201.GPU.FP16.engine

When running custom detectnet:
[TRT] network profiling complete, saving engine cache to models/targets/ssd-mobilenet.onnx.1.1.8201.GPU.FP16.engine
[TRT] failed to open engine cache file for writing models/targets/ssd-mobilenet.onnx.1.1.8201.GPU.FP16.engine
[TRT] device GPU, completed saving engine cache to models/targets/ssd-mobilenet.onnx.1.1.8201.GPU.FP16.engine

Interestingly, at the start of the script running it says something like: found engine file at models/targets/ssd-mobilenet.onnx.1.1.8201.GPU.FP16.engine

5

Can you try running sudo chown -R $USER models/targets ?

My guess is that since you ran train_ssd.py in the container, it created that models/targets directory with root privileges, and now that you are running outside the container it can’t write to it anymore.

6

Once again, you’re the saviour of Jetson AI,
It’s fixed. It’s now saving and loading the engine.

Thanks for all of your great work Dusty, you’ve been a huge help to me!

7

OK great, sorry about that - happy to help!

9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.