NIM Tool Calling

AI & Data Science NVIDIA NIM Models
,
1

I face this error when NIM LLM call Opensearh MCP tools (Using Llama 3 8B - instruct):

raise self._make_status_error_from_response(err.response) from None openai.InternalServerError: Error code: 500 - {‘object’: ‘error’, ‘message’: “Could not translate the instance {‘description’: ‘Request body’} to a\n regular expression. Make sure it is valid to the JSON Schema specification. If\n it is, please open an issue on the Outlines repository”, ‘type’: ‘InternalServerError’, ‘param’: None, ‘code’: 500} During task with name ‘agent’ and id ‘28de3b01-5d2a-283d-c4d0-2de64343a808’

File “/opt/nim/llm/.venv/lib/python3.12/site-packages/outlines/fsm/json_schema.py”, line 487, in to_regex raise NotImplementedError( NotImplementedError: Could not translate the instance {‘description’: ‘Request body’} to a regular expression. Make sure it is valid to the JSON Schema specification. If it is, please open an issue on the Outlines repository

2

Hi @k.eid ,

The Model you are using doesn’t support tool calling. You can see the supported features on the model’s system card here: llama3-8b-instruct Model by Meta | NVIDIA NIM

Best,

Sophie

3

Well I am actually using this one and it support tool calling

4

Thanks for the NIM info - I’ll reach out to the right team to look into this.

Best,

Sophie