Is there any configuration to limit the maximum number of concurrent requests processed in riva?

AI & Data Science Deep Learning (Training & Inference) Riva
1

If GPU/memory resource is enough, is there any configuration to limit the maximum number of concurrent requests processed in one Riva ASR streaming recognition server instance?

2

HI @174362510

Thanks for your interest in Riva

I will check regarding this query with Riva team and update you

Thanks

3

Hello rvinobha,
Is there any update for this issue? It seems that we can config triton related parameters such as PREFERRED_BATCH_SIZE in config.sh under riva_quickstart folder, right? Could you explain it in detail?

4

Hello @rvinobha , sorry for disturbing you, is there any update for this issue?

5

Hi @174362510 and @whz796100

Apologies for the delay,

I will check again today with internal team and get back

Thanks