Enable the vLLM router to route requests to external providers like OpenAI or Anthropic.
## Router - [ ] (P0) XpYd disaggregated prefill support - [ ] (P1) Routing to external providers like OpenAI or Anthropic (#883 #902 ) - [ ] (P1) Router-side request queuing support (#876 ) - [ ] (P2) Implement predictive routing based on future workloads - [ ] (P2) Implement priority routing - [ ] (P2) Improve router performance - [ ] Compile current router with Nuitka - [ ] Prototype Rust/Go/Nginx-based router frontend ## Advanced Deployment - [ ] (P0) Advanced autoscaling based on modern inference metrics - [ ] (P0) Implement KV-cache–aware and prefix-aware routing logic in Gateway Inference extension and CRD (#881 ) - [ ] (P0) Implement disaggregated prefill support in CRD (#841 ) - [ ] (P0) Integrate KEDA-based scaling in CRD (#903 ) - [ ] (P1) Make existing deployment methods (Helm/CRD/Gateway Inference) more Kubernetes-native - [ ] (P2) Support automatic configuration of engine parameters ## Support for Modern Workloads - [ ] (P0) Support deployment with vLLM Omni - [