The user asks if ToolUniverse can support a fully local LLM deployment instead of remote API calls, citing data privacy and compliance reasons for medical and research environments when using LLMs like Qwen Code.
Hello ToolUniverse team, I’m currently building a AI agent using the ToolUniverse framework together with Qwen Code as the LLM backend. By default, Qwen Code connects to the cloud using qwen-oauth (remote API calls). However, for medical and research environments, we need to run everything locally — including the LLM — for data privacy and compliance reasons. I would like to confirm the following: 1.Can ToolUniverse work with a locally deployed LLM (e.g., Qwen2.5-Coder-7B running on GPU) instead of a remote API model? 2. What configuration changes are needed in the ~/.qwen/settings.json (or equivalent) to make ToolUniverse connect to a local model rather than using OAuth / API endpoints? 3.Does ToolUniverse automatically detect the local inference backend (like vLLM, lmdeploy, or transformers), or should it be manually specified? 4.Is there an example or documentation showing how to set up ToolUniverse + local LLM for a fully offline workflow?