You can try and use torch2trt to conver the pytorch model to a TRT engine, and then use it with DeepStream.
I am not sure if all network configurations work successfully with this though, but most off the shelf models like ResNet etc do. There are sample usage instructions given as well in the repo README.
To save you a bit of time, here are some differences from their example usage given for some additional features:
# Create example data (As far as I can tell, this is just to ascertain the network input dimensions)
x = torch.ones((1, 3, 224, 224)).cuda()
# Convert to TensorRT feeding sample data as input (I specified the max_batch_size parameter quite randomly)
model_trt = torch2trt(model, [x], fp16_mode=True, max_batch_size=16)
# Save the TensorRT model i.e. the Cuda engine
with open($ENGINE_FILE_PATH, "wb") as f:
You can then modify one of the sample configuration files and specify the path for the generated TRT engine and use it with the