This document is the authoritative reference for how PluginEval measures plugin and skill quality. It covers the three evaluation layers, all ten scoring dimensions, the composite formula, badge thresholds, anti-pattern flags, Elo ranking, and actionable improvement tips. Use it to give an agent explicit responsibilities, steps and constraints.
llm-evaluation — detailed patterns and worked examples captures reusable agent playbook guidance for agent design & orchestration. Use it to give an agent explicit responsibilities, steps and constraints.
You MUST follow these rules exactly. Violating any of them is a failure. Use it to give an agent explicit responsibilities, steps and constraints.
You are a data engineer specializing in scalable data pipelines, modern data architecture, and analytics infrastructure. Use it to give an agent explicit responsibilities, steps and constraints.
PluginEval is a three-layer quality evaluation framework for Claude Code plugins and skills. It combines deterministic static analysis, LLM-based semantic judging, and Monte Carlo simulation to produce calibrated quality scores with confidence intervals. Use it to ground design choices in named patterns, trade-offs and examples.
This document defines the JSON schemas used by skill-creator. Use it to navigate the topic and choose relevant methods, papers or tools.
This document provides guidance on creating comprehensive evaluations for MCP servers. Evaluations test whether LLMs can effectively use your MCP server to answer realistic, complex questions using only the tools provided. Use it to navigate the topic and choose relevant methods, papers or tools.
Create MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. The quality of an MCP server is measured by how well it enables LLMs to accomplish real-world tasks. Use it to make implementation decisions and avoid common dead ends.
This repository centralizes and summarizes practical and proposed defenses against prompt injection. Use it to navigate the topic and choose relevant methods, papers or tools.
Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on Large Language Models and exploring the boundaries and limits of Generative AI. Use it to build a structured path from fundamentals to hands-on practice.
Policy enforcement, identity, sandboxing, and SRE for autonomous AI agents. One pip install, any framework. Use it as a repeatable review, validation or hardening pass.
First open-source project — fully-local on a single RTX 3090 (Qwen3.6-27B) — to report 95% SimpleQA (n=500) and 77% xbench-DeepSearch (n=100) on local hardware. See the r/LocalLLaMA announcement and the benchmark dataset. Use it as a repeatable review, validation or hardening pass.
An open-source visual environment for battle-testing prompts to LLMs. ChainForge is a data flow prompt engineering environment for analyzing and evaluating LLM responses.
A self-evolving librarian agent for document collections. AutoRAG searches your PDFs, wikis, notes, research papers, and knowledge bases — then curates the results into clean, numbered knowledge units.
中文   ｜   English   📖 中文文档   ｜   📖 English Documentation EvalScope is a one-stop LLM evaluation framework built by the ModelScope Community. Just one command to start — it supports model capability evaluation, inference performance stress testing, and result visualization.
Just like a compass guides us on our journey, OpenCompass will guide you through the complex landscape of evaluating large language models. With its powerful algorithms and intuitive interface, OpenCompass makes it easy to assess the quality and effectiveness of your NLP models.
This document is the authoritative reference for how PluginEval measures plugin and skill quality. It covers the three evaluation layers, all ten scoring dimensions, the composite formula, badge thresholds, anti-pattern flags, Elo ranking, and actionable improvement tips. Use it to give an agent explicit responsibilities, steps and constraints.
llm-evaluation — detailed patterns and worked examples captures reusable agent playbook guidance for agent design & orchestration. Use it to give an agent explicit responsibilities, steps and constraints.
You MUST follow these rules exactly. Violating any of them is a failure. Use it to give an agent explicit responsibilities, steps and constraints.
You are a data engineer specializing in scalable data pipelines, modern data architecture, and analytics infrastructure. Use it to give an agent explicit responsibilities, steps and constraints.
PluginEval is a three-layer quality evaluation framework for Claude Code plugins and skills. It combines deterministic static analysis, LLM-based semantic judging, and Monte Carlo simulation to produce calibrated quality scores with confidence intervals. Use it to ground design choices in named patterns, trade-offs and examples.
This document defines the JSON schemas used by skill-creator. Use it to navigate the topic and choose relevant methods, papers or tools.
This document provides guidance on creating comprehensive evaluations for MCP servers. Evaluations test whether LLMs can effectively use your MCP server to answer realistic, complex questions using only the tools provided. Use it to navigate the topic and choose relevant methods, papers or tools.
Create MCP (Model Context Protocol) servers that enable LLMs to interact with external services through well-designed tools. The quality of an MCP server is measured by how well it enables LLMs to accomplish real-world tasks. Use it to make implementation decisions and avoid common dead ends.
This repository centralizes and summarizes practical and proposed defenses against prompt injection. Use it to navigate the topic and choose relevant methods, papers or tools.
Awesome-LLM-Eval: a curated list of tools, datasets/benchmark, demos, leaderboard, papers, docs and models, mainly for Evaluation on Large Language Models and exploring the boundaries and limits of Generative AI. Use it to build a structured path from fundamentals to hands-on practice.
Policy enforcement, identity, sandboxing, and SRE for autonomous AI agents. One pip install, any framework. Use it as a repeatable review, validation or hardening pass.
First open-source project — fully-local on a single RTX 3090 (Qwen3.6-27B) — to report 95% SimpleQA (n=500) and 77% xbench-DeepSearch (n=100) on local hardware. See the r/LocalLLaMA announcement and the benchmark dataset. Use it as a repeatable review, validation or hardening pass.
An open-source visual environment for battle-testing prompts to LLMs. ChainForge is a data flow prompt engineering environment for analyzing and evaluating LLM responses.
A self-evolving librarian agent for document collections. AutoRAG searches your PDFs, wikis, notes, research papers, and knowledge bases — then curates the results into clean, numbered knowledge units.
中文   ｜   English   📖 中文文档   ｜   📖 English Documentation EvalScope is a one-stop LLM evaluation framework built by the ModelScope Community. Just one command to start — it supports model capability evaluation, inference performance stress testing, and result visualization.
Just like a compass guides us on our journey, OpenCompass will guide you through the complex landscape of evaluating large language models. With its powerful algorithms and intuitive interface, OpenCompass makes it easy to assess the quality and effectiveness of your NLP models.