API question

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hi,
In TensorRT 5.1, after I deserialize the plan file - is there a way to know whether the inference (as a result of how the .plan file was generated) would use INT8 or FP16/32?

thanks
Eyal

3

Hi,

You can deserialize the engine with trtexec directly.
It will show you the detail engine information on the console.

For example, in TensorRT-7.1:

$ /usr/src/tensorrt/bin/trtexec --loadEngine=out.plan

[10/14/2020-15:31:56] [I] === Model Options ===
[10/14/2020-15:31:56] [I] Format: *
[10/14/2020-15:31:56] [I] Model:
[10/14/2020-15:31:56] [I] Output:
[10/14/2020-15:31:56] [I] === Build Options ===
[10/14/2020-15:31:56] [I] Max batch: 1
[10/14/2020-15:31:56] [I] Workspace: 16 MB
[10/14/2020-15:31:56] [I] minTiming: 1
[10/14/2020-15:31:56] [I] avgTiming: 8
[10/14/2020-15:31:56] [I] Precision: FP32
[10/14/2020-15:31:56] [I] Calibration:
[10/14/2020-15:31:56] [I] Safe mode: Disabled
[10/14/2020-15:31:56] [I] Save engine:
[10/14/2020-15:31:56] [I] Load engine: out.plan
[10/14/2020-15:31:56] [I] Builder Cache: Enabled
[10/14/2020-15:31:56] [I] NVTX verbosity: 0
[10/14/2020-15:31:56] [I] Inputs format: fp32:CHW
[10/14/2020-15:31:56] [I] Outputs format: fp32:CHW
[10/14/2020-15:31:56] [I] Input build shapes: model
[10/14/2020-15:31:56] [I] Input calibration shapes: model
[10/14/2020-15:31:56] [I] === System Options ===
[10/14/2020-15:31:56] [I] Device: 0
[10/14/2020-15:31:56] [I] DLACore:
[10/14/2020-15:31:56] [I] Plugins:
[10/14/2020-15:31:56] [I] === Inference Options ===
[10/14/2020-15:31:56] [I] Batch: 1
[10/14/2020-15:31:56] [I] Input inference shapes: model
[10/14/2020-15:31:56] [I] Iterations: 10
[10/14/2020-15:31:56] [I] Duration: 3s (+ 200ms warm up)
[10/14/2020-15:31:56] [I] Sleep time: 0ms
[10/14/2020-15:31:56] [I] Streams: 1
[10/14/2020-15:31:56] [I] ExposeDMA: Disabled
[10/14/2020-15:31:56] [I] Spin-wait: Disabled
[10/14/2020-15:31:56] [I] Multithreading: Disabled
[10/14/2020-15:31:56] [I] CUDA Graph: Disabled
[10/14/2020-15:31:56] [I] Skip inference: Disabled
[10/14/2020-15:31:56] [I] Inputs:
[10/14/2020-15:31:56] [I] === Reporting Options ===
…

Thanks.

4

Thanks @AastaLLL. I wasn’t making myself clear :(
I need it via the C++ code. how do I get this via my code?

5

Hi,

Sorry that we don’t have a public API to query such information.
Please use the trtexec for checking instead.

Thanks.