Hello, I am going to use tools/function call of LLM in my project. However, it shows the error that :
openai.BadRequestError: Error code: 400 - {‘object’: ‘error’, ‘message’: ‘“auto” tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set’, ‘type’: ‘BadRequestError’, ‘param’: None, ‘code’: 400}
After that, I check the NIM document and re-build the container with NIM_ENABLE_AUTO_TOOL_CHOICE=1 and NIM_TOOL_CALL_PARSER=llama3_json like below:
docker run -d --name nim
–gpus all --restart unless-stopped
-p 8000:8000
-e NGC_API_KEY=xxxxxxxxxxxxxxxxxxx
-e NIM_ENABLE_AUTO_TOOL_CHOICE=1
-e NIM_TOOL_CALL_PARSER=llama3_json
nvcr.io/nim/meta/llama-3.1-8b-instruct-dgx-spark:latest
Unfortunately, it still get error :
openai.BadRequestError: Error code: 400 - {‘object’: ‘error’, ‘message’: ‘“auto” tool choice requires --enable-auto-tool-choice and --tool-call-parser to be set’, ‘type’: ‘BadRequestError’, ‘param’: None, ‘code’: 400}