# mcp-eval

## Docs

- [Agent Evaluation](https://mcp-eval.ai/agent-evaluation.md): Treat your agent as the system under test. Define scenarios, assert behavior, and measure efficiency and quality.
- [API: Catalog](https://mcp-eval.ai/api-catalog.md): Master the Expect API for powerful assertions on content, tools, performance, and more.
- [API: Config](https://mcp-eval.ai/api-config.md): Complete guide to mcp-eval configuration: settings, agents, providers, and programmatic control.
- [API: Core](https://mcp-eval.ai/api-core.md): Essential decorators and primitives for writing mcp-eval tests: @task, @with_agent, @parametrize, and more.
- [API: Session](https://mcp-eval.ai/api-session.md): Master TestSession and TestAgent for orchestrating tests, assertions, and metrics collection.
- [Assertions](https://mcp-eval.ai/assertions.md): Unified assertion API with the Expect catalog: content, tools, performance, judges, path.
- [Best Practices](https://mcp-eval.ai/best-practices.md): Learn proven patterns and anti-patterns for testing MCP servers and agents. Write maintainable, reliable, and efficient tests that scale with your project.
- [Building with Claude](https://mcp-eval.ai/building-with-claude.md): Use Claude subagents to accelerate mcp-eval test development with specialized AI assistants
- [Changelog](https://mcp-eval.ai/changelog.md): High-level changes and user‑visible updates.
- [CI/CD](https://mcp-eval.ai/ci-cd.md): Run mcp-eval in GitHub Actions, publish artifacts, post PR comments, and add badges.
- [CLI Reference](https://mcp-eval.ai/cli-reference.md): Complete reference for MCP-Eval command-line interface, including commands and flags
- [Common Workflows](https://mcp-eval.ai/common-workflows.md): Step-by-step guides for typical mcp-eval tasks, from writing your first test to CI/CD integration.
- [Core Concepts](https://mcp-eval.ai/concepts.md): Understand the fundamental concepts and architecture of mcp-eval for effective testing.
- [Configuration Guide](https://mcp-eval.ai/configuration.md): Master mcp-eval configuration. Learn file structures, precedence rules, environment variables, and programmatic APIs for complete control over your testing environment.
- [Connect MCP Servers](https://mcp-eval.ai/connect-servers.md): Configure and validate MCP servers via mcp.json, DXT manifests, or programmatic settings.
- [Datasets](https://mcp-eval.ai/datasets.md): Define Cases and Datasets for systematic evaluation; run programmatically or from files.
- [Detailed Guide](https://mcp-eval.ai/detailed_guide.md): A comprehensive, single-page guide to `mcp-eval`: concepts, setup, styles, assertions, metrics, CLI, and best practices.
- [Complete Examples](https://mcp-eval.ai/examples.md): Learn by example! Complete, runnable test suites for MCP servers and agents, covering all testing patterns and real-world scenarios.
- [Example: mcp_server_fetch](https://mcp-eval.ai/examples-mcp-server-fetch.md): Walkthrough of the fetch server example: datasets, golden paths, and assertions.
- [FAQ](https://mcp-eval.ai/faq.md): Frequently asked questions about mcp-eval, servers, agents, and configuration.
- [mcp-eval Documentation](https://mcp-eval.ai/index.md): The comprehensive testing framework for MCP servers and tool-using agents.
- [Integrations](https://mcp-eval.ai/integrations.md): Patterns for connecting external MCP servers and multi‑server agents.
- [Metrics & Tracing](https://mcp-eval.ai/metrics-tracing.md): OTEL traces as source of truth — metrics, span tree, coverage, and where to find artifacts.
- [mcp-eval Overview](https://mcp-eval.ai/overview.md): Learn about mcp-eval, the comprehensive framework for testing MCP servers and tool-using agents in production-like environments.
- [Pytest](https://mcp-eval.ai/pytest.md): Use native pytest with mcp-eval fixtures and markers.
- [Quickstart Guide](https://mcp-eval.ai/quickstart.md): Get `mcp-eval` up and running in 5 minutes. Learn to install, configure, and run your first tests for MCP servers and agents.
- [Reports](https://mcp-eval.ai/reports.md): Console, Markdown, and HTML reports with metrics, assertion outcomes, and traces.
- [Security Guide](https://mcp-eval.ai/security.md): Secure your mcp-eval testing environment. Learn about API key management, secure configurations, compliance, and security best practices.
- [MCP Server Evaluation](https://mcp-eval.ai/server-evaluation.md): Connect your server to an agent and validate correctness, robustness, performance, and path efficiency.
- [Generating Tests with LLMs](https://mcp-eval.ai/test-generation.md): Use the generator to discover tools, create scenarios, refine assertions, and emit tests or datasets.
- [Troubleshooting Guide](https://mcp-eval.ai/troubleshooting.md): Diagnose and fix common issues with `mcp-eval`. Learn debugging techniques, understand error messages, and get your tests running smoothly.

## Optional

- [GitHub](https://github.com/lastmile-ai/mcp-eval)
- [mcp-agent](https://mcp-agent.com)