The user requests an OpenAI compatible endpoint for the vLLM backend in Triton Inference Server, similar to the one that ships with vLLM itself.
**Is your feature request related to a problem? Please describe.** vLLM backend works well and is easy to set up, compared to TensorRT which had me pulling my hair. However it lacks the OpenAI compatible endpoint that ships with vLLM itself. The `/generate` endpoint on its own requires work to setup for chat applications (that I honestly don't know how to do). In essence, just by adopting vLLM triton instead of vLLM, you have to develop classes and interfaces for all these things. Not to mention that LangChain has no LLM implementation and LlamaIndex's is a bit primitive, undocumented and bugs out. **Describe the solution you'd like** Include vLLM's OpenAI compatible endpoint as an endpoint while using Triton. **Additional context** Pros: - Better integration with Langchain (through `ChatOpenAI`) and LlamaIndex - Triton becomes orders of magnitude easier to setup, run and migrate to (i.e you don't have to rebuild your whole toolset to accommodate Triton) - Be