The issue of converting onnx to trt in orinagx hardware

flaty_star · June 5, 2024, 10:15am

hi
We are currently using the Orin Agx development kit to deploy models and convert ONNX mode to TRT mode. We have no problem with YOLOv5 v7, but during the deployment of OSTRACK, we encountered a problem with model conversion because the previous model input was single input, whereas OSTRACK has two inputs. Therefore, in
Nvinfer1:: IOptimizationProfile * profile=builder ->createOptimizationProfile(); profile>setDimensions(“z”,nvinfer1::OptProfileSelector::kMIN,nvinfer1::Dims4(1,3,384,384));
profile->setDimensions(“x”,nvinfer1::OptProfileSelector::kOPT,nvinfer1::Dims4(2,3,384,384)); This section has been optimized and configured using the above method, and the model can be successfully transformed. However, I believe this is not the most correct method. How should the correct method be implemented?

And during the model conversion process, we encountered an issue with the system automatically shutting down and restarting, so we suspect it may be a problem with the model conversion. However, in the x86 system, the model conversion was successful without this issue

AastaLLL · June 6, 2024, 3:57am

Hi,

Please find our example here:

github.com

NVIDIA/TensorRT/blob/release/8.6/samples/common/sampleEngines.cpp#L836


      
              SMP_RETVAL_IF_FALSE(profile->setShapeValues(input->getName(), OptProfileSelector::kOPT,
                                      profileDims.data(), static_cast<int>(profileDims.size())),
                  "Error in set shape values OPT", false, err);
              profileDims = shapes[static_cast<size_t>(OptProfileSelector::kMAX)];
              SMP_RETVAL_IF_FALSE(profile->setShapeValues(input->getName(), OptProfileSelector::kMAX,
                                      profileDims.data(), static_cast<int>(profileDims.size())),
                  "Error in set shape values MAX", false, err);
          }
          else
          {
              profileDims = shapes[static_cast<size_t>(OptProfileSelector::kMIN)];
              SMP_RETVAL_IF_FALSE(
                  profile->setDimensions(input->getName(), OptProfileSelector::kMIN, toDims(profileDims)),
                  "Error in set dimensions to profile MIN", false, err);
              profileDims = shapes[static_cast<size_t>(OptProfileSelector::kOPT)];
              SMP_RETVAL_IF_FALSE(
                  profile->setDimensions(input->getName(), OptProfileSelector::kOPT, toDims(profileDims)),
                  "Error in set dimensions to profile OPT", false, err);
              profileDims = shapes[static_cast<size_t>(OptProfileSelector::kMAX)];
              SMP_RETVAL_IF_FALSE(
                  profile->setDimensions(input->getName(), OptProfileSelector::kMAX, toDims(profileDims)),

You can set kMIN, kOPT, and kMAX for each input tensor separately.
Thanks.

flaty_star · June 7, 2024, 9:06am

This issue has not been resolved, and an error will still be reported. I suspect it may be due to the model being too large, as follows:
ERROR] 4: [optimizer.cpp::computeCosts::3725] Error Code 4: Internal Error (Could not find any implementation for node {ForeignNode[Reshape_62 + Transpose_63…ReduceMean_640]} due to insufficient workspace. See verbose log for requested sizes.)
ERROR] 2: [builder.cpp::buildSerializedNetwork::751] Error Code 2: Internal Error (Assertion engine != nullptr failed. )

This model has no issues with RTX4090

AastaLLL · June 25, 2024, 7:15am

Hi,

Sorry for the late update.

Could you try to run your model with trtexec to see if similar errors were reported?

$ /usr/src/tensorrt/bin/trtexec --onnx=[file] --minShapes=[minShape] ...

Thanks.

system · July 17, 2024, 6:50am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Keras->Onnx->TensorRT Jetson AGX Orin tensorrt	4	65	September 25, 2024
Can not convert onnx model to TensorRT format on the Nvidia AGX Orin dev kit TensorRT tensorrt	5	591	September 28, 2023
Facing error while trying to convert onnx model using TensorRT Optimisation tool DRIVE AGX Xavier General driveos-dl	10	1411	April 3, 2023
Running a pytorch network converted to ONNX with TensorRT on the TX2 Jetson TX2	24	8804	October 18, 2021
Orin-NX 32GB convert model to tensorRT mask system crush Jetson AGX Orin tensorrt	9	508	January 5, 2024
Input length mismatch (onnx conversion to .trt) TensorRT tensorrt , onnx	4	1234	July 13, 2022
Convert onnx model using trtexec in DRIVE OS DRIVE AGX Orin General driveos-dl	2	18	August 6, 2024
Issue with ONNX to TensorRT Model Conversion Jetson Orin NX tensorrt	10	1102	March 20, 2024
How to convert .onnx to .trtmodel tensorrt using trtexec with size output optional TensorRT	2	183	September 11, 2024
Unable to convert torchvision model to TRT engine compatible with latest version of deepstream on Jetson DeepStream SDK tensorrt , pytorch , jetson , deepstream	10	854	June 6, 2023

The issue of converting onnx to trt in orinagx hardware

Related topics