TImestamped output from a file very similar to transcribe_file.py:
[ 7.73s] anti
[ 9.15s] berta and
[ 9.88s] ablupur
[ 10.56s] both
[ 10.94s] transformer
[ 11.26s] basedd
[ 11.96s] language
[ 12.68s] models are
[ 13.41s] examples of the
[ 14.48s] emerging work in
[ 15.15s] using graph
[ 15.87s] neural
[ 16.57s] networks to
[ 16.70s] design protein sequences for particular target antigens
[ 16.70s]
Transcription completed in 16.70 seconds
[ 16.70s] Transcription completed
You can see that after I executed the command it takes around 7 seconds to get the first transcribed word. Is this normal behavior?
Please do help me out with understanding how to remedy this. Thank you!
I deployed the parakeet RNNT with Nvidia RIVA NIM (as shown here Getting Started — NVIDIA NIM Riva ASR ) on AWS/GCP machine with GPU A10.
When I call the server with a client (nvidia-riva/python-clients) I also have these huge delays in final segments arrival. The intermediate segments (is_final = False) are getting so fast it feels real time, but the is_final=True are like few seconds (5±) after the actual speech.
Are you using the latest NIM?
Final transcript is generated only when the ASR sees 800ms silence. Until then, it will generate partial transcripts.
You can change the parameter value (800 ms default) from client side by using —stop_history parameter from client
Yes. I execute the “docker run” command with nim…:latest
Specifically:
docker run -it --rm --name=$CONTAINER_ID --runtime=nvidia --gpus ‘“device=0”’ --shm-size=28GB -e NGC_API_KEY -e NIM_HTTP_API_PORT=9000 -e NIM_GRPC_API_PORT=50051 -p 9000:9000 -p 50051:50051 -e NIM_TAGS_SELECTOR nvcr.io/nim/nvidia/$CONTAINER_ID:latest
with CONTAINER_ID=parakeet-1-1b-rnnt-multilingual
and NIM_TAGS_SELECTOR=name=parakeet-1-1b-rnnt-multilingual,mode=str
I saw the stop_history: 800 param and tried to decrease it (from the client side with StreamingRecognitionConfig) but it seems to have no affect on the delay, even with extremely low values.
I still can see the final transcription as intermediate for few seconds, before it returns as a final. Also, I thought that using VAD might help, but as I saw it is not supported with RNNT through NIM (here https://docs.nvidia.com/nim/riva/asr/latest/customization.html )
The digest did not match the one you showed, every time I executed “docker run” with “latest” version it used the image I pulled about two months ago.
Now I pruned the old image and pulled the actual new one (which matches the digest), and the ASR model behaves as expected (pretty fast and configurable with stop_history etc).
