thread safety of IPluginRegistry

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

According to https://github.com/NVIDIA/TensorRT/blob/release/7.0/include/NvInferRuntimeCommon.h

//! \brief Single registration point for all plugins in an application. It is
//! used to find plugin implementations during engine deserialization.
//! Internally, the plugin registry is considered to be a singleton so all
//! plugins in an application are part of the same global registry.
//! Note that the plugin registry is only supported for plugins of type
//! IPluginV2 and should also have a corresponding IPluginCreator implementation.

It appears that there is a global, mutable IPluginRegistry (clients cannot create their own for some reason), and this does not give great confidence for thread-safety.

Is the IPluginRegistry returned by getPluginRegistry truly global or is it perhaps thread-local? If it is truly global, since it exposes mutable methods and gives access to all registered creators mutably, how can clients ensure it is used safely in multithreaded applications? Which API functions touch the creator list (since it is global, we clients have no idea) of the registry?

Regards,
Tom Peters

PS I had commented on the IPluginRegistry in another thread: https://devtalk.nvidia.com/default/topic/1064980/tensorrt/opposite-of-registercreator-/post/5408624/#5408624 where I again made the point that having a single global registry is arguably a poor design choice.

2

Singleton global instance of IPluginRegistry

  • registerCreator and getPluginCreator methods are protected by a registry mutex, as both of these interact with the creator list
  • getPluginCreatorList (not guarded by mutex) returns a const pointer to the creator list
3

getPluginCreatorList (not guarded by mutex) returns a const pointer to the creator list

The creator list may be protected, but the creators it holds are not: the return type is IPluginCreator* const* so we’re free to mutate the creators themselves in an unsynchronized manner (the TensorRT lib may be reading them as I write to them). And I might actually want to mutate creators on a per-thread basis – I may need to communicate something about how my plugins are created on a particular thread.

Several of the interface functions on IPluginCreator are non-const (for example createPlugin), so we may have data races or race conditions as TensorRT calls these virtuals in multiple threads.

Allowing clients to own their own IPluginRegistry (like the old IPluginFactory) would simplify these issues a lot.

4

Thanks for the feedback, Tom. We will deliberate over the options for next release and get back to you.

5

Cheers, thanks!