I am trying to train the Mask2Former model using the container from the tao_pytorch_backend repository: nvcr.io/nvidia/tao/tao-toolkit:5.5.0-pyt-base. The training runs on a multi-instance GPU setup in an AWS EC2 instance of type g6.12xlarge.
During the forward function of MSDeformAttnFunction, I encountered the following error:
AttributeError: _OpNamespace object has no attribute ‘MultiscaleDeformableAttnPlugin_TRT’.
It seems that the operation MultiscaleDeformableAttnPlugin_TRT is not registered under the “nvidia” namespace.