Description
Cuda crash dump on trtexec after upgrade to LT4.4 from LT4.3.
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
Cuda failure: initialization error
Aborted (core dumped)
Followed these steps:
https://docs.nvidia.com/jetson/jetpack/install-jetpack/index.html#upgrade-jetpack
Environment
TensorRT Version: 7.1.0.16
GPU Type: Nvidia Xavier
Nvidia Driver Version:
CUDA Version: 10.2.89
CUDNN Version: 8.0.0.145
Operating System + Version: Ubuntu 18.04.4 LTS
Python Version (if applicable): 3.6
TensorFlow Version (if applicable):(tf)1.15.2+nv20.4, (tf-gpu) 1.15.0+nv20.1.tf1,
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag): Baremetal
Relevant Files
Steps To Reproduce
nvidia@nvidia-desktop:~/butterfly$ /usr/src/tensorrt/bin/trtexec --explicitBatch --onnx=frozen_graph2.onnx
&&&& RUNNING TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --explicitBatch --onnx=frozen_graph2.onnx
[06/17/2020-21:20:08] [I] === Model Options ===
[06/17/2020-21:20:08] [I] Format: ONNX
[06/17/2020-21:20:08] [I] Model: frozen_graph2.onnx
[06/17/2020-21:20:08] [I] Output:
[06/17/2020-21:20:08] [I] === Build Options ===
[06/17/2020-21:20:08] [I] Max batch: explicit
[06/17/2020-21:20:08] [I] Workspace: 16 MB
[06/17/2020-21:20:08] [I] minTiming: 1
[06/17/2020-21:20:08] [I] avgTiming: 8
[06/17/2020-21:20:08] [I] Precision: FP32
[06/17/2020-21:20:08] [I] Calibration:
[06/17/2020-21:20:08] [I] Safe mode: Disabled
[06/17/2020-21:20:08] [I] Save engine:
[06/17/2020-21:20:08] [I] Load engine:
[06/17/2020-21:20:08] [I] Builder Cache: Enabled
[06/17/2020-21:20:08] [I] NVTX verbosity: 0
[06/17/2020-21:20:08] [I] Inputs format: fp32:CHW
[06/17/2020-21:20:08] [I] Outputs format: fp32:CHW
[06/17/2020-21:20:08] [I] Input build shapes: model
[06/17/2020-21:20:08] [I] Input calibration shapes: model
[06/17/2020-21:20:08] [I] === System Options ===
[06/17/2020-21:20:08] [I] Device: 0
[06/17/2020-21:20:08] [I] DLACore:
[06/17/2020-21:20:08] [I] Plugins:
[06/17/2020-21:20:08] [I] === Inference Options ===
[06/17/2020-21:20:08] [I] Batch: Explicit
[06/17/2020-21:20:08] [I] Input inference shapes: model
[06/17/2020-21:20:08] [I] Iterations: 10
[06/17/2020-21:20:08] [I] Duration: 3s (+ 200ms warm up)
[06/17/2020-21:20:08] [I] Sleep time: 0ms
[06/17/2020-21:20:08] [I] Streams: 1
[06/17/2020-21:20:08] [I] ExposeDMA: Disabled
[06/17/2020-21:20:08] [I] Spin-wait: Disabled
[06/17/2020-21:20:08] [I] Multithreading: Disabled
[06/17/2020-21:20:08] [I] CUDA Graph: Disabled
[06/17/2020-21:20:08] [I] Skip inference: Disabled
[06/17/2020-21:20:08] [I] Inputs:
[06/17/2020-21:20:08] [I] === Reporting Options ===
[06/17/2020-21:20:08] [I] Verbose: Disabled
[06/17/2020-21:20:08] [I] Averages: 10 inferences
[06/17/2020-21:20:08] [I] Percentile: 99
[06/17/2020-21:20:08] [I] Dump output: Disabled
[06/17/2020-21:20:08] [I] Profile: Disabled
[06/17/2020-21:20:08] [I] Export timing to JSON file:
[06/17/2020-21:20:08] [I] Export output to JSON file:
[06/17/2020-21:20:08] [I] Export profile to JSON file:
[06/17/2020-21:20:08] [I]
NVMAP_IOC_QUERY_HEAP_PARAMS failed [Inappropriate ioctl for device]
Cuda failure: initialization error
Aborted (core dumped)
Please include:
- Exact steps/commands to build your repro
- Exact steps/commands to run your repro
- Full traceback of errors encountered