Config.yml enable or disable LLM thinking

AI & Data Science NeMo Agent toolkit
1

Hi all, does anyone know if I can define in config.yml llm section to enable or disable thinking to the LLM model?

2

The way you do this will depend a bit on how the llm is being invoked inside of your code, and which LLM you are using.

For example, for nemotron super 49b, thinking is enabled by modifying the system prompt. The approach would be:

  1. Define a function or workflow with a configuration, eg “thinking_enabled: true/false”
  2. In your workflow, access that configuration value and use it to dynamically create the system prompt
 if (config.thinking_enabled):
  system_prompt = "detailed thinking on" +  system_prompt

Feel free to open an issue on the GitHub repository if this is not clear, and you can include more details on your current code and we can work on a reproducible example. GitHub · Where software is built

1 Like
3

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.