How to get processing performance of 30FPS with "ssd_mobilenet_v2"

Hello.

Please understand that I am not good at English.

Currently, I’m using Jetson Nano to verify if “ssd_mobilenet_v2” or “ssdlite_mobilenet_v2” can output 30FPS from a USB camera.
Previously, I checked “ssd_mobilenet_v2” using “Deepstream 4.0” and it was 16-18FPS. (I referred to “How to use ssd_mobilenet_v2”.)
However, it did not work well with “ssdlite_mobilenet_v2”.

Please tell me two points.
・How to convert “ssdlite_mobilenet_v2” to UFF model and operate on “Deepstream 4.0” like “ssd_mobilenet_v2”
・In the Jetson Nano benchmark (https://developer.nvidia.com/embedded/jetson-nano-dl-inference-benchmarks), the processing performance of “ssd_mobilenet_v2” is 39FPS. What could be the cause of this difference?

The version is as follows.
・Jetson Nano
・JetPack: 4.2
・Deepstream: 4.0
・TensorRT: 5.0

Hi,

The performance report only measure the model inference time.
You can reproduce it with our script here:

Please use tx2-nano-benchmarks.csv for benchmarking.
Thanks.

I ran it on Jetson Nano and got the following error:
Please tell me how to resolve.

[08/12/2020-10:25:30] [I] === Model Options ===
[08/12/2020-10:25:30] [I] Format: ONNX
[08/12/2020-10:25:30] [I] Model: models/ssd-mobilenet-v1-bs1.onnx
[08/12/2020-10:25:30] [I] Output:
[08/12/2020-10:25:30] [I] === Build Options ===
[08/12/2020-10:25:30] [I] Max batch: explicit
[08/12/2020-10:25:30] [I] Workspace: 1024 MB
[08/12/2020-10:25:30] [I] minTiming: 1
[08/12/2020-10:25:30] [I] avgTiming: 8
[08/12/2020-10:25:30] [I] Precision: FP32+FP16
[08/12/2020-10:25:30] [I] Calibration:
[08/12/2020-10:25:30] [I] Safe mode: Disabled
[08/12/2020-10:25:30] [I] Save engine:
[08/12/2020-10:25:30] [I] Load engine: models/ssd-mobilenet-v1_b1_ws1024_gpu.engine
[08/12/2020-10:25:30] [I] Builder Cache: Enabled
[08/12/2020-10:25:30] [I] NVTX verbosity: 0
[08/12/2020-10:25:30] [I] Inputs format: fp32:CHW
[08/12/2020-10:25:30] [I] Outputs format: fp32:CHW
[08/12/2020-10:25:30] [I] Input build shapes: model
[08/12/2020-10:25:30] [I] Input calibration shapes: model
[08/12/2020-10:25:30] [I] === System Options ===
[08/12/2020-10:25:30] [I] Device: 0
[08/12/2020-10:25:30] [I] DLACore:
[08/12/2020-10:25:30] [I] Plugins:
[08/12/2020-10:25:30] [I] === Inference Options ===
[08/12/2020-10:25:30] [I] Batch: Explicit
[08/12/2020-10:25:30] [I] Input inference shapes: model
[08/12/2020-10:25:30] [I] Iterations: 10
[08/12/2020-10:25:30] [I] Duration: 180s (+ 200ms warm up)
[08/12/2020-10:25:30] [I] Sleep time: 0ms
[08/12/2020-10:25:30] [I] Streams: 1
[08/12/2020-10:25:30] [I] ExposeDMA: Disabled
[08/12/2020-10:25:30] [I] Spin-wait: Disabled
[08/12/2020-10:25:30] [I] Multithreading: Disabled
[08/12/2020-10:25:30] [I] CUDA Graph: Disabled
[08/12/2020-10:25:30] [I] Skip inference: Disabled
[08/12/2020-10:25:30] [I] Inputs:
[08/12/2020-10:25:30] [I] === Reporting Options ===
[08/12/2020-10:25:30] [I] Verbose: Disabled
[08/12/2020-10:25:30] [I] Averages: 100 inferences
[08/12/2020-10:25:30] [I] Percentile: 99
[08/12/2020-10:25:30] [I] Dump output: Disabled
[08/12/2020-10:25:30] [I] Profile: Disabled
[08/12/2020-10:25:30] [I] Export timing to JSON file:
[08/12/2020-10:25:30] [I] Export output to JSON file:
[08/12/2020-10:25:30] [I] Export profile to JSON file:
[08/12/2020-10:25:30] [I]
[08/12/2020-10:25:30] [E] Error opening engine file: models/ssd-mobilenet-v1_b1_ws1024_gpu.engine
[08/12/2020-10:25:30] [E] Engine creation failed
[08/12/2020-10:25:30] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # ./trtexec --onnx=models/ssd-mobilenet-v1-bs1.onnx --explicitBatch --fp16 --workspace=1024 --avgRuns=100 --duration=180 --loadEngine=models/ssd-mobilenet-v1_b1_ws1024_gpu.engine

The command is as follows.

git clone https://github.com/NVIDIA-AI-IOT/jetson_benchmarks.git
sudo sh install_requirements.sh
mkdir models
python3 utils/download_models.py --all --csv_file_path benchmark_csv/tx2-nano-benchmarks.csv --save_dir models
sudo python3 benchmark.py --all --csv_file_path benchmark_csv/tx2-nano-benchmarks.csv \
                            --model_dir models \
                            --jetson_devkit nano \
                            --gpu_freq 921600000 --power_mode 0 --precision fp16

Hi,

Error opening engine file: models/ssd-mobilenet-v1_b1_ws1024_gpu.engine

This error indicates that TensorRT cannot generate the serialized file correctly.
Would you mind to use absolute path for --model_dir and try it again?

Thanks.