I need help… I trained / finetuned/deployed a qa model using the notebook provided in NGC. I used SQuAD v1.1 and 2.0 dataset.
“nvidia/tao/questionanswering_notebook:v1.2”
How to use this model such that I dont have to provide the context separately. The end goal is a chatbot.
I need to train a model on a certain domain and if I ask a question, I should get the answer without passing the context.
How to go about this?
I need some pointers.
For QA inference, see Question Answering — TAO Toolkit 3.22.05 documentation . User just need to set input_batch with the list of examples to run inference on. Does it meet your requirement?
!tao question_answering infer
-e /specs/nlp/question_answering/infer.yaml
-m trained-model.tlt \
my query here is… when we implement a chatbot using the trained model, how are we catering for the context? How does the IR system (information retrieval) work with trained model so that we just ask a question and not provide context for it?
In the examples we are using wiki for this.
In the models we provided context. How are we using it?
To infer we are again providing context and set of questions based on the context.
Can you please share any code sample for such implementation.
Can you have a try with " https://ngc.nvidia.com/catalog/resources/nvidia:tao:questionanswering_notebook/version/v1.2/files/question-answering-deployment.ipynb "?
Once the Riva server is up and running with your models, you can send inference requests querying the server.
Please download the notebook from " https://ngc.nvidia.com/catalog/resources/nvidia:tao:questionanswering_notebook/files?version=v1.2 "
Currently, the TAO QA model cannot do answering given a question without a context.
There is a sample in Riva user guide. You can have a look.
https://docs.nvidia.com/deeplearning/riva/user-guide/docs/samples/weather.html
I have done this already. Thats what I am working on. Thank you.
So this is what I understand…
If I trained the model with Squad dataset and deployed to riva server and then start the service again, I cant directly ask a question like " When did the 1973 oil crisis begin?" Which is a part of v2.0 dataset which has been used to finetune and infer.
Context is still a mandate. Please correct me if I am wrong.
Can this question work via “tao question_answering infer xxx”?
In input_batch I will anyways be providing the context again right? What are we training the model for? Should it not be able to answer, if it has already been trained on a context? The context for the question above is there in V2.0 Squad.
Do you have training log? How about the result?
I have executed both train and deploy successfully
The infer code snippet at the end of deploy notebook executes successfully
The following code sample shows how you can perform inference using Riva Python API gRPC bindings:
import grpc
import riva_api.riva_nlp_pb2 as rnlp
import riva_api.riva_nlp_pb2_grpc as rnlp_srv
grpc_server = “localhost:50051”
channel = grpc.insecure_channel(grpc_server)
riva_nlp = rnlp_srv.RivaLanguageUnderstandingStub(channel)
req = rnlp.NaturalQueryRequest()
test_context = “In 2010 the Amazon rainforest experienced another severe drought, in some ways more extreme than the 2005 drought.”
“The affected region was approximate 1,160,000 square miles (3,000,000 km2) of rainforest, compared to 734,000 square miles (1,900,000 km2)”
" in 2005. The 2010 drought had three epicenters where vegetation died off, whereas in 2005 the drought was focused on the southwestern part."
" The findings were published in the journal Science. In a typical year the Amazon absorbs 1.5 gigatons of carbon dioxide; during 2005"
“instead 5 gigatons were released and in 2010 8 gigatons were released.”
req.query = “How many tons of carbon are absorbed the Amazon in a typical year?”
req.context = test_context
resp = riva_nlp.NaturalQuery(req)
print(resp)
results {
answer: “1.5 gigatons”
score: 0.5661579966545105
}
Thanks. So, could you share similar log when you mention “cant directly ask a question like " When did the 1973 oil crisis begin?” " ?
The following code sample shows how you can perform inference using Riva Python API gRPC bindings:
import grpc
import riva_api.riva_nlp_pb2 as rnlp
import riva_api.riva_nlp_pb2_grpc as rnlp_srv
grpc_server = “localhost:50051”
channel = grpc.insecure_channel(grpc_server)
riva_nlp = rnlp_srv.RivaLanguageUnderstandingStub(channel)
req = rnlp.NaturalQueryRequest()
req.query = “When did Beyonce start becoming popular?”
_InactiveRpcError Traceback (most recent call last)
in
14
15 #req.context = test_context
—> 16 resp = riva_nlp.NaturalQuery(req)
17 print(resp)
~/.local/lib/python3.6/site-packages/grpc/_channel.py in call(self, request, timeout, metadata, credentials, wait_for_ready, compression)
944 state, call, = self._blocking(request, timeout, metadata, credentials,
945 wait_for_ready, compression)
→ 946 return _end_unary_response_blocking(state, call, False, None)
947
948 def with_call(self,
~/.local/lib/python3.6/site-packages/grpc/_channel.py in _end_unary_response_blocking(state, call, with_call, deadline)
847 return state.response
848 else:
→ 849 raise _InactiveRpcError(state)
850
851
_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INVALID_ARGUMENT
details = “Either the query or the context were empty.”
debug_error_string = “{“created”:”@1631009634.983369142",“description”:“Error received from peer ipv6:[::1]:50051”,“file”:“src/core/lib/surface/call.cc”,“file_line”:1066,“grpc_message”:“Either the query or the context were empty.”,“grpc_status”:3}
If we want to use data for a specific domain … lets say ‘My Company’… and build a chatbot which should answer user queries like FAQs. What is the way to provide context (IR system). Kindly suggest.
To build a chatbot for QA, there are indeed multiple solutions.
The question-answering model is one of it. But it needs question and a context string as the input.
So, for above error, you need to set test_context.
For other methods, for example, using joint intent/slot model + dialogue manager(domain controller + dialogue policy manager + policy state tracker + fulfillment engine ) . Anyway, this way is a system instead of one model.
Thank you Morganh, for prompt responses and your time.