Modifying gst-nvinfer plugin

gst-nvinfer is open source. DeepStream SDK FAQ - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums. You can do what you want.

gst-nvinferserver support GRPC from DS 6.0: Gst-nvinferserver — DeepStream 6.1.1 Release documentation