More information about "safety restricted mode"

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I can’t found a clear documentation about “safety restricted mode” of TensorRT. What type of operation the network can performs in this mode?

I have tested a network using trtexec and without --safe the network work. With --safe the program gives me this error:
`[08/07/2020-19:51:21] [E] [TRT] (Unnamed Layer* 140) [Matrix Multiply]: MatrixMultiplyLayer not supported in safety restricted mode.

[08/07/2020-19:51:21] [E] [TRT] Layer (Unnamed Layer* 140) [Matrix Multiply] failed validation

It’s seems that TensorRT can’t perform matrix multiplication in safe mode. There is a documentation in which is listed usable operation in this mode?

Thanks

Environment

TensorRT Version: 7.0.0
GPU Type: 2080
Nvidia Driver Version: 440.100
CUDA Version: 10.2
CUDNN Version: 7.6.5
Operating System + Version: Ubuntu 18.04
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered
2

Hi @pippomuzzo,
Your query has been noted. Please allow me sometime to check on this.

Thanks!

1 Like
3

There really isn’t any safety restricted mode in TensorRT 7.0. Safety restricted mode is part of TensorRT released with DriveOS and is documented in the DriveOS SDK.

4

Thank You for reply.
So what is the purpose of the --safe argument of trtexec present in the TensorRT standard release (not DriveOS)?

In the documentation of trtexec (TensorRT/samples/opensource/trtexec at release/7.0 · NVIDIA/TensorRT · GitHub) it is wrote
" Specifying the --safe parameter turns the safety mode switch ON . By default, the --safe parameter is not specified; the safety mode switch is OFF . The layers and parameters that are contained within the --safe subset are restricted if the switch is set to ON . The switch is used for prototyping the safety restricted flows until the TensorRT safety runtime is made available."

It seems that there is a “safety restricted mode” in the TensortRT standard release. But it can be a my wrong interpretation.

5

Hi there, I’ve encountered the same issue. Is there any docs on how to parse MatrixMultiplyLayer when using --safe option? Or more generally, how to deal with unsupported layers when using --safe option with trtexec?