Can multiple engines be created from the same tensorrt engine

[trivedi.nagaraj]

Hi all, I want know two informations.

1. How to find out how many cores that an engine is being executed
2. Is it possible to create multiple tensorrt engines
3. Is it possible to create multiple execution contexts from the same engine

If it is possible then please let me know how this can be done. Please clarify these doubts.

Thanks and Regards

Nagaraj Trivedi

[AastaLLL]

Hi,

1. No such info but you can monitor GPU utilization with tegrastats.

2. Yes.

3. Yes, you can create multiple contexts from the same serialized engine model.

Examples can be found in trtexec:
https://github.com/NVIDIA/TensorRT/blob/release/8.5/samples/trtexec/README.md#example-6-tune-throughput-with-multi-streaming

$ /usr/src/tensorrt/bin/trtexec --help
  ...
  --streams=N                 Instantiate N engines to use concurrently (default = 1)
  ...

Thanks.