Loading request...
A user wants to have several LLMs available for different departments in their company, allowing them to choose and use the model that best fits their needs. They find switching between LLMs cumbersome and seek a more seamless experience.
Guys, I have a question. At my workplace we bought a 5060 Ti with 16GB to test local LLMs. I was using Ollama, but I decided to test vLLM and it seems to perform better than Ollama. However, the fact that switching between LLMs is not as simple as it is in Ollama is bothering me. I would like to have several LLMs available so that different departments in the company can choose and use them. Which do you prefer, Ollama or vLLM? Does anyone use either of them in a corporate environment? If so, which one?