Anyway to boost yolo performance on Jetson Orin?

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
4

@AastaLLL

I got the same result ~ 10FPS, but the streaming is 30FPS/RTP live

Please check jtop/ tegrastats result.

PS: yolo was started at the middle of the video about 40s.

EDIT: Check engine file but failed

  1. I got this file from YOLO11 🚀 NEW - Ultralytics YOLO Docs .
  2. And followed NVIDIA Jetson - Ultralytics YOLO Docs to convert pt file to TensorRT model yolo export model=yolo11n.pt format=engin

Now, it failed???

$ /usr/src/tensorrt/bin/trtexec ./model/yolo11n.engine
[11/19/2024-16:43:32] [E] Model missing or format not recognized

Any idea???

It seems quite similar to Slow FPS on Orin Nano 8 GB - YoloV8

— EDIT: working, can you help to check if it’s OK?

$ /usr/src/tensorrt/bin/trtexec --loadEngine=model/yolo11n.engine
&&&& RUNNING TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --loadEngine=model/yolo11n.engine
[11/19/2024-17:03:33] [I] === Model Options ===
[11/19/2024-17:03:33] [I] Format: *
[11/19/2024-17:03:33] [I] Model:
[11/19/2024-17:03:33] [I] Output:
[11/19/2024-17:03:33] [I] === Build Options ===
[11/19/2024-17:03:33] [I] Max batch: 1
[11/19/2024-17:03:33] [I] Memory Pools: workspace: default, dlaSRAM: default, dlaLocalDRAM: default, dlaGlobalDRAM: default
[11/19/2024-17:03:33] [I] minTiming: 1
[11/19/2024-17:03:33] [I] avgTiming: 8
[11/19/2024-17:03:33] [I] Precision: FP32
[11/19/2024-17:03:33] [I] LayerPrecisions:
[11/19/2024-17:03:33] [I] Calibration:
[11/19/2024-17:03:33] [I] Refit: Disabled
[11/19/2024-17:03:33] [I] Sparsity: Disabled
[11/19/2024-17:03:33] [I] Safe mode: Disabled
[11/19/2024-17:03:33] [I] DirectIO mode: Disabled
[11/19/2024-17:03:33] [I] Restricted mode: Disabled
[11/19/2024-17:03:33] [I] Build only: Disabled
[11/19/2024-17:03:33] [I] Save engine:
[11/19/2024-17:03:33] [I] Load engine: model/yolo11n.engine
[11/19/2024-17:03:33] [I] Profiling verbosity: 0
[11/19/2024-17:03:33] [I] Tactic sources: Using default tactic sources
[11/19/2024-17:03:33] [I] timingCacheMode: local
[11/19/2024-17:03:33] [I] timingCacheFile:
[11/19/2024-17:03:33] [I] Heuristic: Disabled
[11/19/2024-17:03:33] [I] Preview Features: Use default preview flags.
[11/19/2024-17:03:33] [I] Input(s)s format: fp32:CHW
[11/19/2024-17:03:33] [I] Output(s)s format: fp32:CHW
[11/19/2024-17:03:33] [I] Input build shapes: model
[11/19/2024-17:03:33] [I] Input calibration shapes: model
[11/19/2024-17:03:33] [I] === System Options ===
[11/19/2024-17:03:33] [I] Device: 0
[11/19/2024-17:03:33] [I] DLACore:
[11/19/2024-17:03:33] [I] Plugins:
[11/19/2024-17:03:33] [I] === Inference Options ===
[11/19/2024-17:03:33] [I] Batch: 1
[11/19/2024-17:03:33] [I] Input inference shapes: model
[11/19/2024-17:03:33] [I] Iterations: 10
[11/19/2024-17:03:33] [I] Duration: 3s (+ 200ms warm up)
[11/19/2024-17:03:33] [I] Sleep time: 0ms
[11/19/2024-17:03:33] [I] Idle time: 0ms
[11/19/2024-17:03:33] [I] Streams: 1
[11/19/2024-17:03:33] [I] ExposeDMA: Disabled
[11/19/2024-17:03:33] [I] Data transfers: Enabled
[11/19/2024-17:03:33] [I] Spin-wait: Disabled
[11/19/2024-17:03:33] [I] Multithreading: Disabled
[11/19/2024-17:03:33] [I] CUDA Graph: Disabled
[11/19/2024-17:03:33] [I] Separate profiling: Disabled
[11/19/2024-17:03:33] [I] Time Deserialize: Disabled
[11/19/2024-17:03:33] [I] Time Refit: Disabled
[11/19/2024-17:03:33] [I] NVTX verbosity: 0
[11/19/2024-17:03:33] [I] Persistent Cache Ratio: 0
[11/19/2024-17:03:33] [I] Inputs:
[11/19/2024-17:03:33] [I] === Reporting Options ===
[11/19/2024-17:03:33] [I] Verbose: Disabled
[11/19/2024-17:03:33] [I] Averages: 10 inferences
[11/19/2024-17:03:33] [I] Percentiles: 90,95,99
[11/19/2024-17:03:33] [I] Dump refittable layers:Disabled
[11/19/2024-17:03:33] [I] Dump output: Disabled
[11/19/2024-17:03:33] [I] Profile: Disabled
[11/19/2024-17:03:33] [I] Export timing to JSON file:
[11/19/2024-17:03:33] [I] Export output to JSON file:
[11/19/2024-17:03:33] [I] Export profile to JSON file:
[11/19/2024-17:03:33] [I]
[11/19/2024-17:03:33] [I] === Device Information ===
[11/19/2024-17:03:33] [I] Selected Device: Orin
[11/19/2024-17:03:33] [I] Compute Capability: 8.7
[11/19/2024-17:03:33] [I] SMs: 8
[11/19/2024-17:03:33] [I] Compute Clock Rate: 0.624 GHz
[11/19/2024-17:03:33] [I] Device Global Memory: 7451 MiB
[11/19/2024-17:03:33] [I] Shared Memory per SM: 164 KiB
[11/19/2024-17:03:33] [I] Memory Bus Width: 128 bits (ECC disabled)
[11/19/2024-17:03:33] [I] Memory Clock Rate: 0.624 GHz
[11/19/2024-17:03:33] [I]
[11/19/2024-17:03:33] [I] TensorRT version: 8.5.2
[11/19/2024-17:03:33] [I] Engine loaded in 0.0139191 sec.
[11/19/2024-17:03:34] [I] [TRT] Loaded engine size: 11 MiB
[11/19/2024-17:03:34] [E] Error[1]: [stdArchiveReader.cpp::StdArchiveReader::32] Error Code 1: Serialization (Serialization assertion magicTagRead == kMAGIC_TAG failed.Magic tag does not match)
[11/19/2024-17:03:34] [E] Error[4]: [runtime.cpp::deserializeCudaEngine::65] Error Code 4: Internal Error (Engine deserialization failed.)
[11/19/2024-17:03:34] [E] Engine deserialization failed
[11/19/2024-17:03:34] [E] Got invalid engine!
[11/19/2024-17:03:34] [E] Inference set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8502] # /usr/src/tensorrt/bin/trtexec --loadEngine=model/yolo11n.engine