Configuring Riva with AWS API Gateway

AI & Data Science Deep Learning (Training & Inference) Riva
1

Hardware - GPU (T4)
Operating System: AWS AMI
Riva Version: 2.2.1

We are trying to configure the Riva APIs with AWS API Gateway. It would let us unify the Riva API with our other APIs and would make it easy for us to implement authentication at par with other modules, SSL termination etc.

To get an inference successfully from Riva, consists of possibly two server calls.

  1. Creating gRPC channel (probably this is happening with the second call
  2. Actual call to the inference method

Above poses two challenges. First, I am not able to pinpoint the HTTP method. And second is what to configure for the backend. I’m not able to use Fiddler or any other interceptors to debug the gRPC requests. In fact that also would not work as API gateway itself is not supporting gRPC for it’s backend. Or I’ve not found any document regarding that yet.

All the tutorials are using Lambda functions. From the API gateway they are actually sending HTTP requests towards Lambda. And from Lamda responding with Binary data. One such tutorial mimicking gRPC from Lamda gRPC to AWS Lambda: Is it Possible? | by Paul Henry | The Coinbase Blog.

There are some examples of using intermediate proxy hosted on Lamda, between gRPC service and API Gateway. In fact there is a software/service grpc-gatewa to accomplish that.

Has anyone configured AWS API Gateway with Riva gRPC backends successfully?

2

Hi @arnab2

Thanks for your interest in Riva,

I will check regarding this request with the team and provide updates

Thanks for your patience

3

Hi @arnab2

Apologies for the delay

I have checked with the team, Currently we do not support or have definite answers

Even thought we do not support/recommend, maybe you can trying using grpc gateways available in the internet

Thanks