Nvidia/llama-3.1-nemotron-nano-4b-v1.1 Tool Calling Issue
I am using the nvidia/llama-3.1-nemotron-nano-4b-v1.1 model within an n8n AI Agent workflow. The model is not correctly executing the tools connected to the agent. Instead of triggering the tool, it outputs the tool call information as a plain text string directly in the chat response.
I have attempted to use this model in two different ways, both with issues:
Using NVIDIA NIM via the OpenAI Node:
When I connect to the NVIDIA NIM’s OpenAI-compatible endpoint, the model correctly identifies which tool to use but fails to execute it. It returns a string like “OOLCALL>[{“name”: “Database_FastAPI”, “arguments”: {“input”: “your-policy-number-here”}}]</TOOLCALL” as a message instead of triggering the actual tool node. This is the primary issue, as shown in the attached screenshots.
Using Ollama:
I pulled a community version of the model (avil/nvidia-llama-3.1-nemotron-nano-4b-v1.1:thinking-off) and used the Ollama Chat node. This method has partial success:
- It can successfully call a single RAG tool when it’s the only tool.
- However, when multiple tools are connected (e.g., RAG + a custom tool), it exhibits the same failure as the NIM method, outputting the tool call as a string.
- Additionally, despite the model version name (thinking-off) and explicit instructions in the prompt to be concise, it still outputs “thinking” or “reasoning” steps before its final answer.
The expected behavior is for the n8n agent to recognize the model’s intent to use a tool, parse the tool name and arguments, and execute the corresponding tool in the workflow.
My workflow with NIM credentials in openai node:
{
"nodes": [
{
"parameters": {
"public": true,
"options": {}
},
"id": "68a70ed0-6cee-4be8-b8b9-206a37088446",
"name": "When chat message received",
"type": "@n8n/n8n-nodes-langchain.chatTrigger",
"typeVersion": 1.1,
"position": [
-220,
380
],
"webhookId": "e985d15f-b2f6-456d-be15-97e0b1544a40"
},
{
"parameters": {
"promptType": "define",
"text": "=this is the `information_requested`: \" {{ $json.chatInput }}\"\n\ndetailed thinking off\nYou are a helpful and friendly insurance assistant.\n\nYour primary goal is to assist users with both **personal policy inquiries** and **general insurance-related questions** by leveraging the following tools and data sources:\n\n\n\n**🔹 For Personal Policy Questions:**\n\nWhen users ask about their **own insurance details** (e.g., policy number, premium, coverage, start/end date, insurer name, etc.), call the `Database_FastAPI` tool.\n- After using the tool, use the feteched data to summarize the most relevant information in a friendly and concise manner. \n Example: \n _\"Your policy number is POL123456. It is a Family Floater Health Insurance policy with a premium of ₹12,000 per year, covering 4 family members.\"_\n- If no data is returned from the tool, kindly inform the user: \n _\"Sorry, we couldn’t find any policy details associated with your phone number.\"_\n\n---\n\n \n---\n\n**🔹 For General Insurance Information:**\n\nWhen users ask questions like:\n- “What are the types of health insurance?”\n- “What does a family floater plan cover?”\n- “Benefits of critical illness insurance?”\n- “Can I get insurance after 60?”\n\nUse the **Knowledge_base** to retrieve the most relevant answer. \nthe databse has content like:-\nIndividual Health Insurance,Family Floater Health Insurance: etc\n\n---\n\n \n\n**🔸 Important Instructions:**\n\n- Always be polite, clear, and informative.\n - Use **Knowledge_base** for **general policy questions**.\n- Use **Database_FastAPI** for **personal policy questions**.\n- Inform users if specific information isn’t available via either source.\n- Your response should not exceed 30 words\n\n---\n \n",
"options": {
"systemMessage": ""
}
},
"type": "@n8n/n8n-nodes-langchain.agent",
"typeVersion": 1.9,
"position": [
280,
380
],
"id": "8885521d-128b-482e-88c0-f65dd7d34e1f",
"name": "AI Agent"
},
{
"parameters": {
"mode": "raw",
"jsonOutput": "={\n \"sessionId\": \"{{ $json.sessionId }}\",\n \"action\": \"{{ $json.action }}\",\n \"chatInput\":\"{{ $json.chatInput }}\",\n \"phone_number\":\"7607165189\"\n}\n",
"options": {}
},
"type": "n8n-nodes-base.set",
"typeVersion": 3.4,
"position": [
20,
380
],
"id": "c0f229f9-3cb2-43fe-a727-e2dc5ba16600",
"name": "Edit Fields"
},
{
"parameters": {
"model": {
"__rl": true,
"value": "nvidia/llama-3.1-nemotron-nano-4b-v1.1",
"mode": "list",
"cachedResultName": "nvidia/llama-3.1-nemotron-nano-4b-v1.1"
},
"options": {
"frequencyPenalty": 0,
"maxTokens": 300,
"responseFormat": "text",
"presencePenalty": 0,
"temperature": 0,
"topP": 1
}
},
"type": "@n8n/n8n-nodes-langchain.lmChatOpenAi",
"typeVersion": 1.2,
"position": [
280,
660
],
"id": "224138dc-6f94-4f9c-a936-37791186a13e",
"name": "OpenAI Chat Model",
"credentials": {
"openAiApi": {
"id": "qwfegN5Wt6dGEJQW",
"name": "OpenAi account 2"
}
}
},
{
"parameters": {
"modelName": "models/text-embedding-004"
},
"type": "@n8n/n8n-nodes-langchain.embeddingsGoogleGemini",
"typeVersion": 1,
"position": [
680,
780
],
"id": "b6495205-c1fd-4925-b60b-ff7355a12437",
"name": "Embeddings Google Gemini",
"credentials": {
"googlePalmApi": {
"id": "XUqw3Br4zaICN2uR",
"name": "Google Gemini(PaLM) Api account 4"
}
}
},
{
"parameters": {
"mode": "retrieve-as-tool",
"toolName": "knowledge_base",
"toolDescription": "use this tool when asked questions about general insurance policies.",
"tableName": {
"__rl": true,
"value": "n8n1",
"mode": "list",
"cachedResultName": "n8n1"
},
"options": {}
},
"type": "@n8n/n8n-nodes-langchain.vectorStoreSupabase",
"typeVersion": 1.1,
"position": [
520,
600
],
"id": "153a0de0-e109-4d38-8b7c-e797b4d999d8",
"name": "Knowledge_base",
"credentials": {
"supabaseApi": {
"id": "pryW7yRa6WZMN7qx",
"name": "Supabase account"
}
}
},
{
"parameters": {
"toolDescription": "Call this tool when asked about policy number or policy expiration date",
"url": "=http://localhost:8000/customer/{{ $json.phone_number }}",
"options": {}
},
"type": "n8n-nodes-base.httpRequestTool",
"typeVersion": 4.2,
"position": [
860,
600
],
"id": "dca51513-9e5d-4700-8fae-48300d544f18",
"name": "Database_FastAPI"
}
],
"connections": {
"When chat message received": {
"main": [
[
{
"node": "Edit Fields",
"type": "main",
"index": 0
}
]
]
},
"Edit Fields": {
"main": [
[
{
"node": "AI Agent",
"type": "main",
"index": 0
}
]
]
},
"OpenAI Chat Model": {
"ai_languageModel": [
[
{
"node": "AI Agent",
"type": "ai_languageModel",
"index": 0
}
]
]
},
"Embeddings Google Gemini": {
"ai_embedding": [
[
{
"node": "Knowledge_base",
"type": "ai_embedding",
"index": 0
}
]
]
},
"Knowledge_base": {
"ai_tool": [
[
{
"node": "AI Agent",
"type": "ai_tool",
"index": 0
}
]
]
},
"Database_FastAPI": {
"ai_tool": [
[
{
"node": "AI Agent",
"type": "ai_tool",
"index": 0
}
]
]
}
},
"pinData": {},
"meta": {
"templateCredsSetupCompleted": true,
"instanceId": "a864eb19f7f1637b4b403bc454fc4fca283355f863a50633ea7f242ae286953f"
}
}
