Skip to main content
Connect your Model Context Protocol (MCP) servers so mcp‑eval can exercise real tools during evaluation.

Options at a glance

  • Add from mcp.json (project‑scoped)
  • Import a DXT manifest
  • Programmatic defaults in config or per Agent
  • Mix local stdio with remote SSE/HTTP servers

Add servers from mcp.json

If your project already defines servers in a mcp.json file (or a path you specify), you can import them:
Reference: mcp.json format and conventions at https://gofastmcp.com/integrations/mcp-json-configuration.

Add servers from a DXT manifest

DXT manifests describe an MCP server (command/args/env or URL). Import directly:
Reference: DXT documentation at https://github.com/anthropics/dxt/blob/main/README.md.

Programmatic configuration (global defaults)

You can set default servers for all tests via the evaluation config. In mcpeval.yaml:
Or programmatically:

Per‑Agent servers

If you are constructing an Agent or AgentSpec, you can override servers at the agent level (recommended for multi‑server agents):

Validate connectivity

Before running tests, verify configuration and connectivity:

Tips

  • Prefer SSE/HTTP for hosted SaaS servers; use stdio for local CLIs.
  • Keep secrets in your secrets file/env rather than hardcoding in mcp.json/DXT.
  • For multi‑server agents, name servers clearly and use Expect.tools.* assertions to verify the correct server/tool was used.