Evidently An open-source framework to evaluate, test and monitor ML and LLM-powered systems. Documentation Evidently is an open-source Python library to evaluate, test, and monitor ML and LLM systems—from experiments to production.

In a nutshell, we aim to generate polyphonic music of multiple tracks (instruments). The proposed models are able to generate music either from scratch, or by accompanying a track given a priori by the user.

The LLM ecosystem has amazing infrastructure (LoRAX, PEFT, vLLM), but lacks standardized, high-quality capability adapters. Problem: Base models limited to 32K context, need 2M tokens for large repositories Solution: Progressive curriculum learning with vLLM + Unsloth hybrid approach Key Innovation: Hybrid optimization combining vLLM's inference speed with Unsloth's training efficiency - achieving 61x context extension with minimal compute!

LLM Finetuning toolkit is a config-based CLI tool for launching a series of LLM fine-tuning experiments on your data and gathering their results. From one single yaml config file, control all elements of a typical experimentation pipeline - prompts, open-source LLMs, optimization strategy and LLM testing.

RouteLLM is a framework for serving and evaluating LLM routers. [Blog ] [Paper ] Our core features include: From PyPI From source Let's walkthrough replacing an existing OpenAI client to route queries between LLMs instead of using only a single model.

MASFactory is a graph-centric framework for orchestrating Multi-Agent Systems with Vibe Graphing: Start from intent, generate a graph design, preview and refine it in a visual environment, compile it into an executable workflow, and trace node states, messages, and shared state at runtime. Turn natural-language intent into a structural design, then iteratively converge to an executable, reusable workflow.

Fine‑tune, evaluate, and run private, personalized LLMs xTuring makes it simple, fast, and cost‑efficient to fine‑tune open‑source LLMs (e.g., GPT‑OSS, LLaMA/LLaMA 2, Qwen3, MiniMax M2, GPT‑J, GPT‑2, DistilGPT‑2, Mamba) on your own data — locally or in your private cloud. Why xTuring: Run a small, CPU‑friendly example first: Want bigger models and reasoning controls?

Website • Docs • Community Slack NannyML is an open-source python library that allows you to estimate post-deployment model performance (without access to targets), detect data drift, and intelligently link data drift alerts back to changes in model performance. Built for data scientists, NannyML has an easy-to-use interface, interactive visualizations, is completely model-agnostic and currently supports all tabular use cases, classification and regression.

Cloud | Documentation | Roadmap | Discord 📕 Table of Contents RAGFlow is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs. It offers a streamlined RAG workflow adaptable to enterprises of any scale.

GeoCalib is an algorithm for single-image calibration: it estimates the camera intrinsics and gravity direction from a single image only. By combining geometric optimization with deep learning, GeoCalib provides a more flexible and accurate calibration compared to previous approaches.