Any way to boost tracking algorithm on Jetson AGX Orin?

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Hi all,

I have a Jetson AGX Orin and I’m working on a red zone detection solution. To achieve this, I’m using a YOLOv11 model along with the ByteTrack algorithm from Ultralytics.

I’m running the following routine to generate the engine model:
model = YOLO(“yolo11s.pt”)
model.export(format=“engine”,simplify=False, half=True)#, project=“models”)
model = YOLO(“yolo11s.engine”)

And this routine to perform inference:
results = model.track(frame,
conf=confidence_score,
persist=True,
classes=target_classes,
tracker="bytetrack.yaml")

Finally, I have some Python code to detect red zone intrusions.

The performance on the Jetson is decent (~20 FPS), but the postprocessing step seems to be the bottleneck. I suspect this is related to the tracking algorithm. Here’s the verbose output from Ultralytics:

Speed: 4.9ms preprocess, 3.6ms inference, 35.5ms postprocess per image

I’ve already applied the following Jetson-side configurations:
sudo nvpmodel -m 0
sudo jetson_clocks

Is there anyway to boost performance?

Thanks a lot.

2

*** Please note that this reply is generated by LLM automatically ***

Based on the provided context, it appears that you are working on a red zone detection solution using a YOLOv11 model and the ByteTrack algorithm from Ultralytics on a Jetson AGX Orin. You are experiencing decent performance at 20 FPS, but the postprocessing step seems to be the bottleneck.

To address this issue, you have already applied some Jetson-side configurations, but you are looking for ways to further boost performance.

One potential solution is to utilize the MAXN mode on the Jetson AGX Orin. MAXN mode is an experimental setting that allows for adjusting clock settings and creating custom power profiles to balance performance and power consumption. However, it is essential to note that MAXN mode is not recommended for running heavy workloads for prolonged periods, as it may cause overheating.

Another possible solution is to optimize the postprocessing step in your Python code. Since the postprocessing step is taking around 355ms per image, it might be beneficial to investigate ways to reduce this time. This could involve optimizing the tracking algorithm, reducing the number of operations, or using more efficient data structures.

Additionally, you may want to consider using the Jetson Orin NX or Jetson Orin Nano series, which offer a MAXN_SUPER mode. This mode may provide better performance for your specific use case.

It is also worth noting that the Jetson AGX Orin has a thermal design power (TDP) budget, and hardware throttling is engaged when the total module power exceeds this budget. Therefore, it is crucial to ensure that your system is not exceeding the TDP budget, as this can impact performance.

In comparison to other Jetson modules, the Jetson AGX Orin has a more powerful GPU and CPU, which can provide better performance for compute-intensive tasks like deep learning inference. However, the Jetson Orin NX and Jetson Orin Nano series may offer better performance for specific use cases, depending on the requirements.

To further optimize performance, you may want to consider the following:

  • Use the Power Estimator tool to estimate the power consumption of your system and generate a custom power profile.
  • Investigate ways to reduce the number of operations in the postprocessing step.
  • Use more efficient data structures and algorithms in your Python code.
  • Consider using a more powerful Jetson module, such as the Jetson AGX Orin 32GB, if your system can handle the increased power consumption.

In summary, to boost performance on your Jetson AGX Orin, you may want to consider utilizing the MAXN mode, optimizing the postprocessing step, and ensuring that your system is not exceeding the TDP budget. Additionally, you may want to investigate using a more powerful Jetson module or optimizing your Python code for better performance.

*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***

5

Hi,

Could you verify if the post-processing run on the GPU with tegrastats?

$ sudo tegrastats

If not, maybe you can try to use the GPU-based tracking algorithm.
We have some in the Deepstream SDK:

Thanks.