Usability of the exported onnx file

I’ve been working on training a YOLOv4 network using my own dataset and I know that I can export an onnx file using the command “tao model yolo_v4 export”.

However, I wonder if an exported onnx file can be corrrectly used by the onnxruntime module?
I’d like to use an exported onnx file to label the unannotated data so that I can increase my training data faster.

A simple code snippet looks like this:

import onnxruntime as ort
detector_onnx = "exported_file.onnx"
detector = ort.InferenceSession(detector_onnx)

I encountered the following error message as a result:

onnxruntime.capi.onnxruntime_pybind11_state.InvalidGraph: [ONNXRuntimeError] : 10 : INVALID_GRAPH : Load model from pretrained_model\Smoke/Face_and_hand.onnx failed:This is an invalid model. In Node, ("BatchedNMS_N", BatchedNMSDynamic_TRT, "", -1) : ("box": tensor(float),"cls": tensor(float),) -> ("BatchedNMS": tensor(int32),"BatchedNMS_1": tensor(float),"BatchedNMS_2": tensor(float),"BatchedNMS_3": tensor(float),) , Error No Op registered for BatchedNMSDynamic_TRT with domain_version of 12

Refer to Errors while reading ONNX file produced by TAO 5

1 Like

Thanks for your help, @Morganh .

I wrote a tool to succesfully trim the model and have been spending some time on trying to do inference using the trimmed model via onnxruntime. There are still some things going on, however.

I traced the source code of and managed to know how images are preprocessed whenever I run tao yolov4 inference and would like to write a tool using onnxruntime for inference.

    import cv2
    import numpy as np
    import os
    import onnxruntime as rt
    img_path = os.path.join(INPUT_ROOT_PATH, img_name)
    original_image = cv2.imread(img_path)
    if original_image is None:
    original_image_BGR2RGB = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB)
    original_image_size = original_image_BGR2RGB.shape[:2]
    image_data = image_preprocess(np.copy(original_image_BGR2RGB), [input_size, input_size])
    image_data = image_data[np.newaxis, ...].astype(np.float32) # NxCxHxW (1x3x416x416)
    # print("Preprocessed image shape:",image_data.shape) # shape of the preprocessed input
    sess = rt.InferenceSession(MODEL)

    outputs = sess.get_outputs()
    output_names = list(map(lambda output:, outputs))
    input_name = sess.get_inputs()[0].name

    detections =, {input_name: image_data})

The output of the function image_preprocess(), i,e, image_data, is exactly in the NxCxHxW dimension.
I can feed the preprocessed image into and it does give me results.
But I keep getting the following warning messages:

Is it a sign of something going wrong?

[W:onnxruntime:, onnxruntime::ExecutionFrame::VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,10647,1,4} for output box
[W:onnxruntime:, onnxruntime::ExecutionFrame::VerifyOutputSizes] Expected shape from model of {} does not match actual shape of {1,10647,2,1} for output cls
Output shape: [(1, 10647, 1, 4), (1, 10647, 2, 1)]

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.