> ## Documentation Index
> Fetch the complete documentation index at: https://mcp-eval.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# CI/CD

> Run mcp-eval in GitHub Actions, publish artifacts, post PR comments, and add badges.

### Run action

The recommended approach is to use the reusable workflow which handles all the setup, testing, and deployment. For the complete workflow configuration, visit [mcpeval.yml](https://github.com/lastmile-ai/mcp-eval/blob/main/.github/workflows/mcpeval.yml).

```yaml theme={null}
name: MCP-Eval CI

on:
  push:
    branches: [main, master, trunk]
  workflow_dispatch:

jobs:
  call-mcpeval:
    uses: lastmile-ai/mcp-eval/.github/workflows/mcpeval-reusable.yml
    with:
      deploy-pages: true
    permissions:
      contents: read
      pages: write
      id-token: write
      pull-requests: write
    secrets: inherit
```

Alternatively, you can directly use the action in your workflow:

```yaml theme={null}
steps:
  - name: Checkout
    uses: actions/checkout@v4

  - name: Run MCP-Eval (uv)
    id: mcpeval
    uses: lastmile-ai/mcp-eval/.github/actions/mcp-eval/run
    with:
      python-version: "3.11"
      working-directory: .
      run-args: "-v"
      tests: tests/
      reports-dir: mcpeval-reports
      json-report: mcpeval-results.json
      markdown-report: mcpeval-results.md
      html-report: mcpeval-results.html
      artifact-name: mcpeval-artifacts
      pr-comment: "true"
      set-summary: "true"
      upload-artifacts: "true"
    env:
      ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

  - name: Generate and upload badges
    uses: lastmile-ai/mcp-eval/.github/actions/mcp-eval/badges
    with:
      report-path: ${{ steps.mcpeval.outputs.results-json-path }}
      output-dir: badges
      format: "both"
      upload-artifacts: "true"
      artifact-name: mcpeval-badges
```

Sources:

* Action: [action.yaml](https://github.com/lastmile-ai/mcp-eval/blob/main/.github/actions/mcp-eval/run/action.yaml)
* README: [run/README.md](https://github.com/lastmile-ai/mcp-eval/blob/main/.github/actions/mcp-eval/run/README.md)
* Workflows: [mcpeval.yml](https://github.com/lastmile-ai/mcp-eval/blob/main/.github/workflows/mcpeval.yml), [mcpeval-reusable.yml](https://github.com/lastmile-ai/mcp-eval/blob/main/.github/workflows/mcpeval-reusable.yml)

### Publish HTML and Badges via Pages

The workflow automatically deploys both the HTML report and badges to GitHub Pages when pushing to main/master branches. After deployment, badges are accessible at `https://<username>.github.io/<repo>/badges/`.

To enable GitHub Pages deployment:

1. Enable GitHub Pages in your repository settings
2. The workflow will automatically deploy on pushes to main/master
3. Badges and reports will be available at your Pages URL

### Badges

After deployment to GitHub Pages, reference badges using your Pages URL:

```markdown theme={null}
[![mcp-tests](https://img.shields.io/endpoint?url=https://YOUR_USERNAME.github.io/YOUR_REPO/badges/mcp-tests.json&cacheSeconds=300)](https://YOUR_USERNAME.github.io/YOUR_REPO/)
[![mcp-cov](https://img.shields.io/endpoint?url=https://YOUR_USERNAME.github.io/YOUR_REPO/badges/mcp-cov.json&cacheSeconds=300)](https://YOUR_USERNAME.github.io/YOUR_REPO/)
```

For example,
[![mcp-tests](https://img.shields.io/endpoint?url=https://lastmile-ai.github.io/mcp-eval/badges/mcp-tests.json\&cacheSeconds=300)](https://lastmile-ai.github.io/mcp-eval/)
[![mcp-cov](https://img.shields.io/endpoint?url=https://lastmile-ai.github.io/mcp-eval/badges/mcp-cov.json\&cacheSeconds=300)](https://lastmile-ai.github.io/mcp-eval/)