YOLOv8 ONNX Model De-serialization Issue on 2080 GPU

Please provide complete information as applicable to your setup.

• Hardware Platform ( GPU)
• DeepStream Version 6.1
• TensorRT Version 8.6
• NVIDIA GPU Driver Version 515
• Issue Type (Model De-serialization)

I am currently facing an issue while running my program on an NVIDIA GeForce RTX 2080 GPU. The program involves using a YOLOv8 model (ONNX format) for object detection. Unfortunately, each time I run the program, I encounter an error during the de-serialization of the inference engine. This issue is causing the program to recreate the engine file with each run, resulting in a significant delay in the program startup, which takes approximately 2-3 minutes.

The specific warning message I receive is as follows:

WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /home/user/repos/pipeline/model_b1_gpu0_fp32.engine open error

As a result of this issue, the program needs to recreate the engine file, replacing the last one. I have ensured that the engine file exists at the specified path. However, the de-serialization process encounters an open error. I have checked the file permissions and paths, but I am still unable to resolve the issue.

Thank you for your time and support.

Best regards,
Daphna

please refer to this topic.

Here are the relevant lines from my config file:

onnx-file=best.onnx
model-engine-file=model_b1_gpu0_fp32.engine

Other pipelines I work with use the engine file without rebuilding the engine file each time.
Same thing is happening to me on Jetson NX… with this particular model/code.

please check if model-engine-file value is correct.

I think the value is correct, there is a different warning when the path to the engine file is incorrect:
here is an example of model-engine-file wrong value:

model-engine-file=/home/user/Documents/model_b1_gpu0_fp32.engine

and the correct path:

model-engine-file=/home/user/repos/model_b1_gpu0_fp32.engine

WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /home/user/repos/model_b1_gpu0_fp32.engine open error
   
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /home/user/Documents open error ```

do you mean that the app will generate engine in every run even the model-engine-file is set correctly? if yes, please check if the engine exists after the first run.

The file does exist and is rebuilt on every run.

please share the logs of the first and second run.

First run:

Now playing…
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
[NvMultiObjectTracker] Initialized
WARNING: …/nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /home/daphna/repos/pipeline_realsense/model_orange_b1_gpu0_fp32.engine open error
0:00:01.415715803 648676 0x557695151390 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1897> [UID = 1]: deserialize engine from file :/home/daphna/repos/pipeline_realsense/model_orange_b1_gpu0_fp32.engine failed
0:00:01.437907698 648676 0x557695151390 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2002> [UID = 1]: deserialize backend context from engine from file :/home/daphna/repos/pipeline_realsense/model_orange_b1_gpu0_fp32.engine failed, try rebuild
0:00:01.437923403 648676 0x557695151390 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
WARNING: [TRT]: Tensor DataType is determined at build time for tensors not marked as input or output.

Building the TensorRT Engine

Building complete

WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
0:00:51.529727601 648676 0x557695151390 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1955> [UID = 1]: serialize cuda engine to file: /home/daphna/repos/pipeline_realsense/model_b1_gpu0_fp32.engine successfully
WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
INFO: …/nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 4
0 INPUT kFLOAT input 3x640x640
1 OUTPUT kFLOAT boxes 8400x4
2 OUTPUT kFLOAT scores 8400x1
3 OUTPUT kFLOAT classes 8400x1

0:00:51.557422359 648676 0x557695151390 INFO nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus: [UID 1]: Load new model:config_infer_primary_yoloV8_orange.txt sucessfully

Second run:

Now playing…
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
gstnvtracker: Batch processing is ON
gstnvtracker: Past frame output is OFF
~~ CLOG[include/modules/NvMultiObjectTracker/NvTrackerParams.hpp, getConfigRoot() @line 52]: [NvTrackerParams::getConfigRoot()] !!![WARNING] Invalid low-level config file caused an exception, but will go ahead with the default config values
[NvMultiObjectTracker] Initialized
WARNING: …/nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /home/daphna/repos/pipeline_realsense/model_orange_b1_gpu0_fp32.engine open error
0:00:01.442730039 648001 0x55ea66402f90 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1897> [UID = 1]: deserialize engine from file :/home/daphna/repos/pipeline_realsense/model_orange_b1_gpu0_fp32.engine failed
0:00:01.464427195 648001 0x55ea66402f90 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger: NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2002> [UID = 1]: deserialize backend context from engine from file :/home/daphna/repos/pipeline_realsense/model_orange_b1_gpu0_fp32.engine failed, try rebuild
0:00:01.464441764 648001 0x55ea66402f90 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1923> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
WARNING: [TRT]: onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
WARNING: [TRT]: Tensor DataType is determined at build time for tensors not marked as input or output.

Building the TensorRT Engine

Building complete

WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
0:00:50.506209154 648001 0x55ea66402f90 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger: NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1955> [UID = 1]: serialize cuda engine to file: /home/daphna/repos/pipeline_realsense/model_b1_gpu0_fp32.engine successfully
WARNING: [TRT]: The getMaxBatchSize() function should not be used with an engine built from a network created with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag. This function will always return 1.
INFO: …/nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 4
0 INPUT kFLOAT input 3x640x640
1 OUTPUT kFLOAT boxes 8400x4
2 OUTPUT kFLOAT scores 8400x1
3 OUTPUT kFLOAT classes 8400x1

0:00:50.533644184 648001 0x55ea66402f90 INFO nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus: [UID 1]: Load new model:config_infer_primary_yoloV8_orange.txt sucessfully

The logs look identical except for the timestamp…

deserialize path: /home/daphna/repos/pipeline_realsense/model_orange_b1_gpu0_fp32.engine
serialize path: /home/daphna/repos/pipeline_realsense/model_b1_gpu0_fp32.engine
model-engine-file is wrong.

Thank you for your assistance! However, I’m wondering why the engine file isn’t being built with the name I specified in the configuration?

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.