​
Does server language matter?

No. Any language works as long as your server implements MCP. mcp-eval drives it via the agent and MCP transport.

​
How do I add servers quickly?

Use mcp-eval server add or import from mcp.json / DXT. See Quickstart and CLI Reference.

​
Decorator order?

Place @with_agent(...) above @task(...) when used together.

​
Can I run without an LLM?

Yes for tool/structural checks that don’t require generation. LLM judge assertions need an LLM.

​
How do I run a specific test or dataset?

mcp-eval run tests/test_file.py::test_func or mcp-eval run dataset path.yaml.

​
Where are configs discovered?

config.py documents discovery for mcpeval.yaml and secrets.

​
Parallel datasets?

Dataset.evaluate(..., max_concurrency=N) enables parallel case execution.