# Different output value between TensorRT and Darknet **URL:** \ **Category:** TensorRT\ **Created:** [November 22, 2018, 6:00am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693 "2018-11-22T06:00:32Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![hotaekhans1m6t](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hotaekhans1m6t](https://forums.developer.nvidia.com/u/hotaekhans1m6t)\ **Post date:** [November 22, 2018, 6:00am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/1 "2018-11-22T06:00:32Z")
TensorRT version : 5.0.2.10 CUDA version : 9.0 cuDNN version : 7.3.1 Driver version : 384.145 GPU : V100 Hello, I have tested ‘yolov3\_onnx’ sample in ‘TensorRT5.0.2/samples/python’. but output value is different with darknet detector. double checked ‘nms threshold’ and ‘obj threshold’. and there are no quantization(fp32). but output values are different. is it ok? I don’t know why output of model is different between ‘tensorrt model’ and ‘original model’ ---
**Author:** ![hotaekhans1m6t](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hotaekhans1m6t](https://forums.developer.nvidia.com/u/hotaekhans1m6t)\ **Post date:** [November 22, 2018, 2:38pm UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/2 "2018-11-22T14:38:56Z")
i found some caution in developer guide. ‘yolo v3’ example has to use onnx-tensorrt because tensorrt not support ‘leakyrelu’ and ‘upsample’. is it true? but i found also support operation list for onnx in developer guide contain ‘leakyrelu’ and ‘upsample’. yes, i’m confused :( ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [November 26, 2018, 11:23pm UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/3 "2018-11-26T23:23:04Z")
Hello, can you provide a specific example where Darknet detector produces a different result compared to TRT? regards, NVES ---
**Author:** ![hotaekhans1m6t](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hotaekhans1m6t](https://forums.developer.nvidia.com/u/hotaekhans1m6t)\ **Post date:** [November 28, 2018, 7:54am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/4 "2018-11-28T07:54:37Z")
Linux : Ubuntu 16.04 Python version 2.7.15 Hello, I downloaded ‘Darknet’ using ‘git clone [https://github.com/pjreddie/darknet.git](https://github.com/pjreddie/darknet.git)’. And I downloaded ‘TensorRT-5.0.2.6’ from download page and I used python sample code for yolov3 to tensorrt. I didn’t any change the code. and I used the same ‘.cfg file’ and ‘.weights file’(these are from ‘yolov3\_to\_onnx.py’) I ran two detector(Darknet, TensorRT) for same image. But I got different outputs. from Darknet: dog: 85% dog: 84% dog: 53% person: 50% from TensorRT: [[180.32643636 6.89801149 107.12263617 162.63818179] [189.34186364 57.27135054 94.68533009 115.64401249] [74.13400486 107.20060694 55.11068005 70.82084432]] [0.63056643 0.92838418 0.71350949] [0 16 16] Darknet detect three dogs and one person but TensorRT detect two dogs and one person. Also there are different output values(the probability of object). e.g, TensorRT detect person as 63% confidence but Darknet detect person as 50% confidence(it is huge gap). I thought if there are no quantization, then output values are identical. is it right? please, clarify why output values are different after convert to TensorRT. Thanks. p.s, I uploaded test image, TensorRT sample code and result images. ![Darknet_predictions.jpg](https://global.discourse-cdn.com/nvidia/original/2X/3/3de95ee71cffdcaab69313017e33a1280a827748.jpeg) ![TensorRT_predictions.png](https://global.discourse-cdn.com/nvidia/original/2X/b/b55cc9c2852c32d010a625335ea029e7db324562.png) ![dog.jpg](https://global.discourse-cdn.com/nvidia/original/2X/c/c7d508dfbb8123ad9bda372fa1e5641bf72b5d21.jpeg) [yolov3\_TensorRT.zip](https://forums.developer.nvidia.com/uploads/short-url/2U2so31n5c164L018QSyS5ZLYKj.zip) (15.2 KB) ![Darknet_predictions.jpg](https://global.discourse-cdn.com/nvidia/original/2X/3/3de95ee71cffdcaab69313017e33a1280a827748.jpeg) ![dog.jpg](https://global.discourse-cdn.com/nvidia/original/2X/c/c7d508dfbb8123ad9bda372fa1e5641bf72b5d21.jpeg) ---
**Author:** ![hotaekhans1m6t](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hotaekhans1m6t](https://forums.developer.nvidia.com/u/hotaekhans1m6t)\ **Post date:** [December 3, 2018, 5:45am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/5 "2018-12-03T05:45:53Z")
any updates…? ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [December 4, 2018, 3:41pm UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/6 "2018-12-04T15:41:36Z")
Just to confirm, i think you have your file names reversed? TensorRT\_predictions.png shows 3 dogs. Darknet\_predictions.jpg shows 2 dogs 1 person. ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [December 4, 2018, 5:08pm UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/7 "2018-12-04T17:08:00Z")
Hello, Per engineering: This appears to be an issue with using light-weight PIL (trt example) instead of OpenCV (darknet) for loading + pre-processing the input . The image you provided has a resolution of 177x284 px and has to be upscaled by a huge factor to 608x608. The assumption is that using OpenCV (darknet) vs. using PIL in the sample may make a big difference. Also the interpolation method will have an influence - we use PIL.Image.BICUBIC. The original dog image for this sample has a resolution of 576x786 px. Here the difference between DarkNet and TensorRT for the original dog.jpg image. DarkNet: dog: 100% truck: 92% bicycle: 99% TensorRT: [0.99854713 0.99880403 0.93829263] [16 1 7] (16=dog, 1=bicycle, 7=truck) The recommendation is either to supply higher resolution input image into sample, or modify TRT sample to use opencv instead of PIL. regards, NVES ---
**Author:** ![hotaekhans1m6t](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hotaekhans1m6t](https://forums.developer.nvidia.com/u/hotaekhans1m6t)\ **Post date:** [December 7, 2018, 12:48am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/8 "2018-12-07T00:48:04Z")
Hello, Thanks for your comment. I have tried high-resolution image but I got same problem. And I have modified both Darknet and TRT sample to use OpenCV for image resize. I used L2 norm to check both resized images are same. And I have exactly the same L2 norm value. [TRT sample] L2 norm of input : 804.7459466025994 [Darknet] L2 norm of input : 804.7459466 But, sadly…I got different output values. [TRT sample] dog: 0.9275025682158862 dog: 0.6527860108775222 [Darknet] dog: 93% dog: 57% one dog has similar confidence(probability). but the other dog has huge different confidence(about 10%). well…I also checked parsed weight value for both Darknet and TRT sample and these are same. where came from error? Could you clarify this problem? And I attach the modified files for TRT sample and Darknet. [yolov3\_onnx.zip](https://forums.developer.nvidia.com/uploads/short-url/9IX4dbRirGBPSqBpT3UEoZCTddk.zip) (15.4 KB) ![dog.jpg](https://global.discourse-cdn.com/nvidia/original/2X/c/cebb80d0b50a2917bf8345505ed307ff5dc8e052.jpeg) [modified\_darknet.zip](https://forums.developer.nvidia.com/uploads/short-url/qgyca1K0e7L5OkzHcV7tvh8LKZM.zip) (16.4 KB) ![dog.jpg](https://global.discourse-cdn.com/nvidia/original/2X/c/cebb80d0b50a2917bf8345505ed307ff5dc8e052.jpeg) ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [December 7, 2018, 1:00am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/9 "2018-12-07T01:00:41Z")
Hello, seems TRT has higher confidence than Darknet for the “2nd” dog ? 0.65 vs. .57? or is it reversed? NVES ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [December 7, 2018, 5:46pm UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/10 "2018-12-07T17:46:42Z")
In #8 reply, I’m seeing better inference result for TRT than darknet. or is the data reversed? In general, we cannot guarantee bit-correctness to open-source repos. ---
**Author:** ![hotaekhans1m6t](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@hotaekhans1m6t](https://forums.developer.nvidia.com/u/hotaekhans1m6t)\ **Post date:** [December 10, 2018, 2:52am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/11 "2018-12-10T02:52:12Z")
Hello, no, it’s not reversed. but…that’s not my point of this thread. my point is that same input and same weights make the same output. okay, thanks :) ---
**Author:** ![NVES](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nves/32/14043_2.png) [@NVES](https://forums.developer.nvidia.com/u/NVES)\ **Post date:** [December 10, 2018, 2:58am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/12 "2018-12-10T02:58:24Z")
@hotaekhans1m6t, In general, we cannot guarantee bit-correctness to open-source repos that are constantly updated by the community (the YOLO layer implementation in this sample is from ~July 2018) ---
**Author:** ![so-chin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@so-chin](https://forums.developer.nvidia.com/u/so-chin)\ **Post date:** [February 7, 2019, 9:57am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/13 "2019-02-07T09:57:40Z")
Hello As I excute the yolo3 to onnx sample code,it gave me some error message like this: ```auto Traceback (most recent call last): File "yolov3_to_onnx.py", line 761, in main() File "yolov3_to_onnx.py", line 754, in main onnx.checker.check_model(yolov3_model_def) File "/usr/local/lib/python2.7/dist-packages/onnx/checker.py", line 86, in check_model C.check_model(model.SerializeToString()) onnx.onnx_cpp2py_export.checker.ValidationError: Node (086_upsample) has input size 1 not in range [min=2, max=2]. ==> Context: Bad node spec: input: "085_convolutional_lrelu" output: "086_upsample" name: "086_upsample" op_type: "Upsample" attribute { name: "mode" s: "nearest" type: STRING } attribute { name: "scales" floats: 1 floats: 1 floats: 2 floats: 2 type: FLOATS } ``` did anyone have this problem? ---
**Author:** ![so-chin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@so-chin](https://forums.developer.nvidia.com/u/so-chin)\ **Post date:** [February 7, 2019, 10:18am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/14 "2019-02-07T10:18:27Z")
By the way, my environment is: ```auto TensorRT version : 5.0.2.10 CUDA version : 9.0 cuDNN version : 7.4.2 GPU : TITAN V ``` ---
**Author:** ![xding](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@xding](https://forums.developer.nvidia.com/u/xding)\ **Post date:** [February 13, 2019, 8:08am UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/15 "2019-02-13T08:08:45Z")
i have the same problem and i replaced the version of onnx (1.2.1). ---
**Author:** ![meetdave06](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@meetdave06](https://forums.developer.nvidia.com/u/meetdave06)\ **Post date:** [April 3, 2019, 4:57pm UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/16 "2019-04-03T16:57:55Z")
Hi, I am trying the same repo to convert YOLO to ONNX to TensorRT. I can successfully get the .trt file but getting a segmentation error on inference. TensorRT version : 5.1.2.2 CUDA version : 10.1 cuDNN version : 7.4.2 GPU : V100 (AWS) Error Dump :- ```auto Loading ONNX file from path yolov3.onnx... Beginning ONNX file parsing Completed parsing of ONNX file Building an engine from file yolov3.onnx; this may take a while... Completed creating Engine Running inference on image dog.jpg... Fatal Python error: Segmentation fault Current thread 0x00007fbb1850b700 (most recent call first): File "/workspace/tensorrt/samples/python/yolov3_onnx/../common.py", line 145 in do_inference File "onnx_to_tensorrt.py", line 160 in main File "onnx_to_tensorrt.py", line 183 in Segmentation fault (core dumped) ``` Below is the function where it throws the error:- ```auto def do_inference(context, bindings, inputs, outputs, stream, batch_size=1): start = time.time() # Transfer input data to the GPU. [cuda.memcpy_htod_async(inp.device, inp.host, stream) for inp in inputs] # Run inference. context.execute_async(batch_size=batch_size, bindings=bindings, stream_handle=stream.handle) # Transfer predictions back from the GPU. [cuda.memcpy_dtoh_async(out.host, out.device, stream) for out in outputs] # Synchronize the stream stream.synchronize() # Return only the host outputs. print("=> time: %.4f" %(time.time()-start)) return [out.host for out in outputs] ``` Any help is appreciated. ---
**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\ **Post date:** [October 12, 2021, 7:48pm UTC](https://forums.developer.nvidia.com/t/different-output-value-between-tensorrt-and-darknet/67693/17 "2021-10-12T19:48:04Z")
Closed on October 12, 2021, 7:48pm UTC