Pytorch Conv2d operation with stride=2 after ONNX conversion and TensorRT inference shows accuracy mismatch against PyTorch inference. This is not seen for stride=1 . The conv2D signature is nn.Conv2d(64, 1, 3, stride=STRIDE, bias=False, padding=1) . I have posted this issue on the TRT-OSS github page too, but haven’t received any further response: Github issue
. Below I have attached the polygraphy code snippet used for TRT vs PYT comparison. Note that the comparison of accuracy fails with TRT vs ONNX-RT too.
Environment
TensorRT Version : 8.2.1.8 NVIDIA GPU : NVIDIA GeForce RTX 3080 Laptop GPU NVIDIA Driver Version : 470.86 CUDA Version : 11.1 CUDNN Version : 8 Operating System : Ubuntu 20.04 Python Version (if applicable) : 3.7.11 PyTorch Version (if applicable) : 1.9.0+cu111 Baremetal or Container (if so, version) : Docker image based off nvidia/cuda:11.1-cudnn8-devel-ubuntu20.04
Install Torch and Polygraphy with the versions shown above, and run the code snippet with STRIDE=1 for no mismatch, and STRIDE=2 to see the mismatches between PYT and TRT results.
Hi, I have attached the github gist link for the polygraphy test code in the original post, It does a dump of ONNX model as well. Please let me know if thats not sufficient. And yes, ONNX-RT vs PYT matches… but not TRT vs ONNX-RT or PYT vs TRT
For some reason, we are unable to run your code successfully.
Could you please check and share us working code to try from our end for better debugging.
[I] trt-runner-N0-02/11/22-08:56:57
---- Inference Output(s) ----
{2 [dtype=float16, shape=(1, 1, 9, 9)]}
[I] trt-runner-N0-02/11/22-08:56:57 | Completed 1 iteration(s) in 0.5744 ms | Average inference time: 0.5744 ms.
[I] pytorch-runner-N0-02/11/22-08:56:57 | Activating and starting inference
Traceback (most recent call last):
File “test.py”, line 56, in
results = Comparator.run(runners)
File “/usr/local/lib/python3.8/dist-packages/polygraphy/comparator/comparator.py”, line 223, in run
run_results.append((runner.name, execute_runner(runner, loader_cache)))
File “/usr/local/lib/python3.8/dist-packages/polygraphy/comparator/comparator.py”, line 97, in execute_runner
with runner as active_runner:
File “/usr/local/lib/python3.8/dist-packages/polygraphy/backend/base/runner.py”, line 59, in enter
self.activate()
File “/usr/local/lib/python3.8/dist-packages/polygraphy/backend/base/runner.py”, line 94, in activate
self.activate_impl()
File “/usr/local/lib/python3.8/dist-packages/polygraphy/backend/pyt/runner.py”, line 52, in activate_impl
self.model, _ = util.invoke_if_callable(self._model)
File “/usr/local/lib/python3.8/dist-packages/polygraphy/util/util.py”, line 579, in invoke_if_callable
ret = func(*args, **kwargs)
File “/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py”, line 1102, in _call_impl
return forward_call(*input, **kwargs)
TypeError: forward() missing 1 required positional argument: ‘input’