RouteLLM

RouteLLM is a framework for serving and evaluating LLM routers. [Blog ] [Paper ] Our core features include: From PyPI From source Let's walkthrough replacing an existing OpenAI client to route queries between LLMs instead of using only a single model.