CI/CD integration

1. Add your API key as a secret

In your GitHub repository go to Settings → Secrets → Actions and add a secret named EVALKIT_API_KEY. Generate a key in your API keys page.

2. Add the workflow file

Create .github/workflows/eval.yml in your repository:

name: EvalKit

on:
  pull_request:
    branches: [main]

jobs:
  eval:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install evalkit
        run: npm install -g evalkit

      - name: Run evals
        env:
          EVALKIT_API_KEY: ${{ secrets.EVALKIT_API_KEY }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
        run: |
          evalkit login --token $EVALKIT_API_KEY
          evalkit run @your-org/your-eval \
            --model anthropic/claude-sonnet-4-6 \
            --record \
            --fail-below 0.80

3. Trigger via API directly

POST to the CI run endpoint from any CI system:

curl -X POST https://api.evalkit.io/v1/ci/run \
  -H "Authorization: Bearer $EVALKIT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "package": "@your-org/your-eval",
    "model": "anthropic/claude-sonnet-4-6",
    "public": false
  }'

# Response:
# {
#   "runId": "run_ci_abc123",
#   "score": 0.91,
#   "status": "completed",
#   "usageAfter": { "used": 12, "limit": 1000 }
# }

CI/CD integration is available on the Developer plan and above. View pricing