CenterNet keypoint detector not giving good FPS in jetson xavier NX

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
,
1

I’m try to use CenterNet Resnet50 V1 FPN Keypoints 512x512 model from tensorflow object detection API.

I have converted this saved_model to TensorRT model by using tf-trt converter but the model size after conversion was around 800MB which is uncommon and while inferecing on the Xavier NX it takes more than 20GB RAM, almost 1 hour to load the model but even after i was getting only 3 FPS.

I have tried also with ONNX runtime but the result is same.

kindly help on this.

Thanks

3

Hi,

How do you measure the 20GB RAM usage?
Since NX only has 8GB physical memory, does the inference also use swap memory?

More, since you get the similar result with onnxruntime.
The high memory usage should come from model itself.

Have you tried it on a desktop GPU?
If yes, could you share the memory usage on an x86 environment with us?

Thanks.

4

Hi AstaLLL,

yes, I have extended the swap memory.

but the model loading memory is reduced to 4GB with few solutions,
however the FPS remains the same.

In my laptop with 1650max-Q GPU it gives around 13FPS.
Im using Python with Tensorflow implementation.

we have sucessfully converted the model to ONNX format and inferenced with ONNX runtime in the xavier NX but there is not much improvement in the FPS (3-4FPS)

I have tried to convert the ONNX to Nvidia tensorRT but getting the below error

mirrag@mirrag-desktop:~/Downloads$ python3 createengine.py

Unsupported ONNX data type: UINT8 (2)

Traceback (most recent call last):

File “createengine.py”, line 19, in 

engine = eng.build_engine(onnx_path, shape= shape)

File “/home/mirrag/Downloads/engine.py”, line 21, in build_engine

network.get_input(0).shape = shape

AttributeError: ‘NoneType’ object has no attribute ‘shape’

This is the link we have tried to convert https://developer.nvidia.com/blog/speeding-up-deep-learning-inference-using-tensorflow-onnx-and-tensorrt/

Thanks

5

Hi,

XavierNX is an embedded device.
It’s expected that 1650 will have a better performance compared to Jetson.
You can try to inference it with fp16 or int8 for extra acceleration.

The unsupported data type error is a known issue.
It is caused by the different default input data type between ONNX and TensorRT.
Please check below comment for the solution:

Thanks.

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.