io.github.cyanheads/evals-mcp-server
Author verifiable eval records through a draft→review→revise→submit loop with enforced graders.
MCP server · by cyanheads
About
io.github.cyanheads/evals-mcp-server is an MCP server published by cyanheads. Marketplaces we read that list it: 1 (Official MCP Registry). We first saw it on 2026-10-11 and have recorded no change since. As of our marketplace pass on 2026-10-12.
What it asks for
- Runs on your machine
- Settings: 10
What it declares, asks for and runs, in full
Only install plugins from developers you trust. We list what marketplaces publish and do not test plugins.
Package
npm @cyanheads/evals-mcp-server (@cyanheads/evals-mcp-server on npm)
- npm downloads/week: 266 (2026-10-12)
- npm downloads/month: 655 (2026-10-12)
- Latest version on npm: 0.1.5, published 2026-10-07
npm facts come from the public npm registry (registry.npmjs.org, api.npmjs.org). Each figure is the registry's own and is never added to another. A registry that publishes no figure shows none: no number is not zero.
Works with
- Claude Code
- Codex
- Cursor
- VS Code
- Copilot CLI
A one-click link opens the client, which asks you to confirm before it adds the server.
From the marketplaces that list it and the MCP transport it declares, as each client documents them. We do not test plugins.
Connect
Commands and configs built only from what its marketplaces declare. We do not run them for you and do not test plugins.
As a local MCP server: npm package
Built from what Official MCP Registry declares.
claude mcp add --transport stdio evals-mcp-server -- npx -y @cyanheads/evals-mcp-server[mcp_servers.evals-mcp-server]
command = "npx"
args = ["-y", "@cyanheads/evals-mcp-server"]{
"mcpServers": {
"evals-mcp-server": {
"command": "npx",
"args": [
"-y",
"@cyanheads/evals-mcp-server"
]
}
}
}Optional settings it also lists: EVALS_CAPTURE_DIR, EVALS_DATA_DIR, EVALS_DEFAULT_LICENSE, EVALS_REQUIRE_CONFIRMATION, MCP_AUTH_MODE, MCP_HTTP_ENDPOINT_PATH, MCP_HTTP_HOST, MCP_HTTP_PORT, MCP_LOG_LEVEL, MCP_TRANSPORT_TYPE.
We do not store launch arguments. If the server needs any, its own documentation names them.
Where it is published
| Marketplace | Status | Version | Figures | Verified | First seen |
|---|---|---|---|---|---|
| Official MCP Registry (Open the registry record) | Listed | 0.1.5 | Downloads/month (npm): 655 Downloads/week (npm): 266 Forks (GitHub): 0 Stars (GitHub): 1 |
Not marked |
Each figure is the marketplace's own and is never added up across marketplaces. A marketplace that publishes no figure shows none: no number is not zero.
What changed
Nothing has moved on this plugin since we started watching it.
Similar plugins
Products whose own names, descriptions, tools and skills are closest in meaning to this one, on every marketplace we read. Closeness is ranked within this list only.
-
mcp-eval-runner io.github.dbsectrainer/mcp-eval-runner
A standardized testing harness for MCP servers and agent workflows
-
LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.
-
codegworkspace-mcp com.clauxel.codegworkspace/codegworkspace-mcp
A paid remote MCP for CodeG, built to return verdicts, receipts, usage logs, and audit-ready JSON.
-
futureagievals-mcp com.clauxel.futureagievals/futureagievals-mcp
A paid remote MCP for AI SDK eval dashboard, built to return verdicts, receipts, usage logs, and aud
-
server io.mcpskills/server
Trust scoring for MCP servers, AI skills & npm packages — 15 signals + safety scanning.
-
Git/CI MCP server with the full protocol surface (tools, resources, prompts, roots, sampling).
-
mcp io.github.DojoGenesis/mcp
Practice library for coding agents: skill search/invoke, seed patches, scout, reflect, ADR writer
-
Agentic code review, no signup to try: reality gates + frontier-model review, with veto.
-
rubrkit com.rubrkit/rubrkit
MCP-native AI evaluation: rubric audits, eval suites, and proof reports for AI/LLM output.
-
evalgate io.github.ipezygj/evalgate
Statistical checks an agent runs before trusting an AI eval number (is #1 real, judge bias, more).
Similar plugins by marketplace
The same question asked of one marketplace at a time: is there an analog of this plugin there?
Has a listing on Claude Code: claude-plugins-community (5)
-
evalview-mcp io.github.hidai25/evalview-mcp
Regression testing for AI agents. Golden baselines, CI/CD, LangGraph, CrewAI, OpenAI, Claude.
-
Generate MCP server scaffold and tool implementation code from a natural language description. Describe what tools you want in plain language, choose TypeScript or Python, and get…
-
Convert MCP servers to token-efficient CLI tools with auto-generated SKILL.md files for Claude Code agents
-
mcp-server io.github.cap-js/mcp-server
Model Context Protocol (MCP) server for AI-assisted development of CAP applications.
-
vynn-mcp io.github.beee003/vynn-mcp
Self-improving AI workflows with backtesting, prompt optimization, and scheduling
Has a listing on Claude Code: claude-plugins-official (5)
-
mcp-server io.github.cap-js/mcp-server
Model Context Protocol (MCP) server for AI-assisted development of CAP applications.
-
Skills for designing and building MCP servers that work seamlessly with Claude. Guides you through deployment models (remote HTTP, MCPB, local), tool design patterns, auth, and…
-
mdk-mcp-server io.github.SAP/mdk-mcp-server
Model Context Protocol (MCP) server for AI-assisted development of MDK applications.
-
Skills for adding DeepEval evaluations, tracing, datasets, Confident AI reports, and iterative improvement loops to AI applications.
-
firebase-mcp io.github.firebase/firebase-mcp
Gives AI development tools Firebase-specific capabilities and expertise.
Has a listing on Claude: knowledge-work-plugins (5)
-
MCP server for terminal commands, file operations, and process management
-
Figma design platform integration. Access design files, extract component information, read design tokens, and translate designs into code. Bridge the gap between design and…
-
gk-cli com.gitkraken/gk-cli
The GitKraken MCP Server for managing repos, PRs, issues across GitHub, GitLab, Bitbucket and more.
-
Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.
-
Hosted MCP server for AI-assisted Grafana Cloud observability — no local installation required.
Has a listing on Codex: openai-curated (openai/plugins) (5)
-
Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking.
-
Skills for NVIDIAs ecosystem spans GPU acceleration, CUDA, AI agents, inference, robotics, Physical AI, Omniverse, and simulation. This plugin helps you understand the pieces…
-
Inspect repositories, triage pull requests and issues, debug CI, and publish changes through a hybrid GitHub connector and CLI workflow.
-
Build with OpenAI APIs, Agents SDK, and ChatGPT Apps, and create and save OpenAI API keys from Codex.
-
Cloudflare platform plugin with curated skills for Workers, Wrangler, and Agents SDK plus the official Cloudflare API MCP server.
Has a listing on Community GitHub marketplaces (5)
-
4da-mcp-server io.github.4DA-Systems/4da-mcp-server
Upgrade intelligence for AI agents: what an upgrade changes, where it touches your code, CVE scans.
-
Self-hosted GitHub MCP server. PAT auth, stdio transport, transport-agnostic so it works with…
-
Skills for adding DeepEval evaluations, tracing, datasets, Confident AI reports, and iterative improvement loops to AI applications.
-
bettermemory io.github.0Mattias/bettermemory
Memory for coding agents, checked against the filesystem and git before it is believed.
-
inquisitor io.github.1111111111111111111114oLvT2/inquisitor
Hypothesis-driven problem solving for AI agents: probe, falsify, escalate.
Has a listing on Copilot CLI: github/awesome-copilot (5)
-
Complete toolkit for building Model Context Protocol (MCP) servers in Kotlin using the official io.modelcontextprotocol:kotlin-sdk library. Includes instructions for best…
-
Complete toolkit for building Model Context Protocol (MCP) servers in Go using the official github.com/modelcontextprotocol/go-sdk. Includes instructions for best practices, a…
-
Complete toolkit for building Model Context Protocol (MCP) servers in TypeScript/Node.js using the official SDK. Includes instructions for best practices, a prompt for generating…
-
Safe BLE diagnostics, evidence workflows, and guarded automation for AI agents.
-
Figma design platform integration. Access design files, extract component information, read design tokens, and translate designs into code. Bridge the gap between design and…
Has a listing on Copilot CLI: github/copilot-plugins (5)
-
mcp-apps
Generate MCP App widgets for MCP tools. Produces self-contained HTML widgets using the MCP Apps protocol.
-
Toolkit for building Microsoft 365 Copilot declarative agents — scaffolding, JSON manifest development, and capability configuration.
-
Build Power Apps Canvas Apps using the Canvas Authoring MCP server.
-
Advanced Security plugin for GitHub Copilot.
-
Navigate and interact with your C++ code using the Microsoft C++ Language Server.
Has a listing on Cursor: cursor/plugins (5)
-
Build apps, scripts, and automations with the TypeScript SDK.
-
Fan large tasks out across parallel cloud agents with planners, workers, verifiers, and structured handoffs.
-
CLI-backed repo compatibility scans plus agents that audit startup, validation, and docs against reality.
-
playwright-mcp io.github.microsoft/playwright-mcp
Playwright Tools for MCP
-
Skills for building on the Cursor Origin API. Fetch the live spec first, then follow the Origin rules for credentials, scopes, webhooks, paging, and errors. Includes a skill that…
Has a listing on Docker MCP Catalog (5)
-
testkube-mcp io.github.kubeshop/testkube-mcp
MCP server for Testkube - Manage test workflows, executions, and artifacts via AI assistants
-
Giving Claude ability to run code with E2B via MCP (Model Context Protocol).
-
Dynatrace-mcp io.github.dynatrace-oss/Dynatrace-mcp
[DEPRECATED] Dynatrace MCP Server. Migrate to dynatrace-for-ai or Dynatrace Remote MCP Server.
-
Read and score Opik traces, experiments, datasets and prompts from your AI assistant.
-
Access to Rube's catalog of remote MCP servers.
Has a listing on Official MCP Registry (5)
-
mcp-eval-runner io.github.dbsectrainer/mcp-eval-runner
A standardized testing harness for MCP servers and agent workflows
-
LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.
-
codegworkspace-mcp com.clauxel.codegworkspace/codegworkspace-mcp
A paid remote MCP for CodeG, built to return verdicts, receipts, usage logs, and audit-ready JSON.
-
futureagievals-mcp com.clauxel.futureagievals/futureagievals-mcp
A paid remote MCP for AI SDK eval dashboard, built to return verdicts, receipts, usage logs, and aud
-
server io.mcpskills/server
Trust scoring for MCP servers, AI skills & npm packages — 15 signals + safety scanning.
Found by meaning, not by shared code or a shared repository: similar plugins stay separate products, and this is not a review or a recommendation.
Is this your plugin?
Add it to your projects and prove you maintain it with a file in its repository, a file on your domain or a DNS record. You get alerts when a marketplace lists it, drops it, changes its verified mark or what it asks for, and its remote endpoint goes on monitoring as a web service.
Data comes from the Official MCP Registry (registry.modelcontextprotocol.io), CC0 1.0. Figures come from our own regular passes over the marketplaces, last read on . We publish no contact details for plugin publishers.