ML Referee
Evaluate ML features beyond a baseline and test bounded feature repairs. Invite-only beta.
MCP server · by scarrico
About
ML Referee is an MCP server published by scarrico. Marketplaces we read that list it: 1 (Official MCP Registry). We first saw it on 2026-10-27 and have recorded no change since. As of our marketplace pass on 2026-11-03.
What it asks for
- Runs remotely
None of the marketplaces we read publishes its tools or the settings it asks for.
What it declares, asks for and runs, in full
Only install plugins from developers you trust. We list what marketplaces publish and do not test plugins.
Works with
- Claude Code
- Codex
- Cursor
- VS Code
- Copilot CLI
A one-click link opens the client, which asks you to confirm before it adds the server.
From the marketplaces that list it and the MCP transport it declares, as each client documents them. We do not test plugins.
Connect
Commands and configs built only from what its marketplaces declare. We do not run them for you and do not test plugins.
As a remote MCP server
Built from what Official MCP Registry declares.
claude mcp add --transport http mlreferee https://api.mlreferee.com/mcp[mcp_servers.mlreferee]
url = "https://api.mlreferee.com/mcp"{
"mcpServers": {
"mlreferee": {
"url": "https://api.mlreferee.com/mcp"
}
}
}Where it is published
| Marketplace | Status | Version | Figures | Verified | First seen |
|---|---|---|---|---|---|
| Official MCP Registry (Open the registry record) | Listed | 1.1.0 | No figure published | Not marked |
Each figure is the marketplace's own and is never added up across marketplaces. A marketplace that publishes no figure shows none: no number is not zero.
What changed
- Became a product we track
- Official MCP Registry: appeared
Similar plugins
Products whose own names, descriptions, tools and skills are closest in meaning to this one, on every marketplace we read. Closeness is ranked within this list only.
-
Sports-analytics research assistant that grades its own model and reports the losses. $9.99/mo.
-
Read-only US public-data screening tools with free Basic and Evidence tiers.
-
Batru — Competitive Game Intelligence for AI Agents
Live competitive-game predictions, meta data and decision tools for AI agents.
-
Predictive models and analytics for motorsport — race intelligence delivered to your endpoint.
-
mcp-server-lol io.github.ebrunet001/mcp-server-lol
League of Legends AI coaching: player profiles, match history, ranked stats & performance analysis
-
LLM benchmark rankings, drift status, incidents and Smart Router previews from AI Stupid Level.
-
Neutral tie-breaks between AI agents and sealed review of an agent's output, with signed receipts.
-
Check AI work against requirements and return structured verdicts, findings, and repair steps.
-
Evaluate state with typed choice, score, and probability questions.
-
Score and improve AI prompts with Cue Free and Pro account access.
Similar plugins by marketplace
The same question asked of one marketplace at a time: is there an analog of this plugin there?
Has a listing on Claude Code: claude-plugins-community (5)
-
AI agent evaluation sandbox for Claude Code. Holdout scenario testing with Doer/Judge/Adversary/Observer roles, probabilistic satisfaction scoring, and append-only JSONL audit…
-
Context engineering plugin for Claude Code — real-time prompt scoring, RLM adaptive weights, and token optimization. Scores every tool use across 5 dimensions (token utilization…
-
Research sprint orchestrator for Claude Code. Ask a question, get a decision-ready brief — every finding is tracked as a typed claim, adversarially challenged, confidence-graded…
-
DataRobot Agent Skills - Agentic skills for DataRobot enterprise AI and ML workflows. Covers model training, deployment, predictions, feature engineering, monitoring…
-
Public AI coding agents, skills, and MCP servers for AccelByte.
Has a listing on Claude Code: claude-plugins-official (5)
-
Skills for adding DeepEval evaluations, tracing, datasets, Confident AI reports, and iterative improvement loops to AI applications.
-
Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.
-
DataRobot skills for AI/ML workflows — model training, deployment, predictions, feature engineering, monitoring, explainability, data preparation, App Framework CI/CD, and…
-
Find the right NVIDIA skill for GPU acceleration, CUDA, AI agents, data loading, training, inference, robotics, Physical AI, Omniverse, simulation, and other NVIDIA-powered…
-
Skills for tracing, evaluating, and improving AI agents with MLflow. Supports the full agent improvement loop: instrument → trace → evaluate → iterate → validate.
Has a listing on Claude: knowledge-work-plugins (5)
-
Skills for working with Langfuse, the open-source LLM engineering platform for tracing, prompt management, and evaluation.
-
DataRobot skills for AI/ML workflows — model training, deployment, predictions, feature engineering, monitoring, explainability, data preparation, App Framework CI/CD, and…
-
AI-first project auditor and re-engineer based on the 9 design principles and 7 design patterns from the TechWolf AI-First Bootcamp
-
Connects Claude Code to the Synthflow AI voice-agent platform through its hosted MCP server, with skills for reviewing calls and auditing agent prompts, plus a docs-search…
-
Figma design platform integration. Access design files, extract component information, read design tokens, and translate designs into code. Bridge the gap between design and…
Has a listing on Codex: openai-curated (openai/plugins) (5)
-
Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking.
-
Skills for NVIDIAs ecosystem spans GPU acceleration, CUDA, AI agents, inference, robotics, Physical AI, Omniverse, and simulation. This plugin helps you understand the pieces…
-
Build with OpenAI APIs, Agents SDK, and ChatGPT Apps, and create and save OpenAI API keys from Codex.
-
PostHog gives your AI agent direct access to your product analytics, feature flags, experiments, error tracking, surveys, logs, and LLM analytics. Ask questions about your data…
-
Available for US1 customers only. Analyze, investigate, and act on your Datadog telemetry directly from ChatGPT using natural language. Ask questions about your production…
Has a listing on Community GitHub marketplaces (4)
-
Connect Baizhi Agent Toolkit through an authenticated remote MCP server, with an optional skill for bounded research and source verification. Requires a Baizhi account and API…
-
system-one
Design, integrate, evaluate, self-host, and troubleshoot typed System One decision models including TypeSafe Jev, Convai Innovations Laya, CLM, and experimental Strands Decider…
-
Skills for adding DeepEval evaluations, tracing, datasets, Confident AI reports, and iterative improvement loops to AI applications.
-
LLM evaluation methodology and eval-suite design, based on Anthropic's evaluation guidance. /evals:methodology answers questions on success criteria, eval design, and grading…
Has a listing on Copilot CLI: github/awesome-copilot (5)
-
Visualizes live session AI-credit usage as a Pac-Man board with pellets, ghosts, fruit milestones, and game-over limits.
-
Safe BLE diagnostics, evidence workflows, and guarded automation for AI agents.
-
Run a lightweight, role-separated AI development team with flexible tools, developer-selected models, proportionate planning, and optional QA.
-
Three-layer verification pipeline for AI output. Extracts claims, finds sources, and flags hallucination risks so humans can verify before acting.
-
See your local Copilot token usage by day, model, and session, and get tips to use it more efficiently.
Has a listing on Copilot CLI: github/copilot-plugins (5)
-
Toolkit for building Microsoft 365 Copilot declarative agents — scaffolding, JSON manifest development, and capability configuration.
-
workiq
Query Microsoft 365 data with natural language — emails, meetings, documents, Teams messages, and more.
-
workiq-productivity
WorkIQ productivity skills powered by the local WorkIQ CLI — read-only insights across email, meetings, calendar, Teams, SharePoint, projects, and people.
-
Advanced Security plugin for GitHub Copilot.
-
model-apps
Build and deploy Power Apps generative pages for model-driven apps with specialist agents for planning, entity creation, and parallel code generation. Requires PAC CLI >= 2.7.0…
Has a listing on Cursor: cursor/plugins (5)
-
Iterative self-referential AI loops using the Ralph Wiggum technique.
-
Work with PostHog from chat: query product analytics and run SQL, manage feature flags and experiments, triage error tracking issues, and review session replays, surveys, and…
-
Ask Databricks Genie questions about your workspace's data in plain language.
-
Inspect and manage feature gates, experiments, dynamic configs, and metrics.
-
Incremental transcript-driven memory updates for AGENTS.md using high-signal bullet points only.
Has a listing on Docker MCP Catalog (5)
-
Read and score Opik traces, experiments, datasets and prompts from your AI assistant.
-
Wayfound’s MCP server allows business users to govern, supervise, and improve AI Agents.
-
The Official Model Context Protocol (MCP) server for Kagi Search & other tools.
-
Discover and connect to a curated marketplace of MCP servers for extending AI agent capabilities.
-
A unified MCP proxy that aggregates multiple MCP servers into one interface, enabling seamless tool discovery and management across all your AI interactions. Manage all your MCP…
Has a listing on Official MCP Registry (5)
-
Sports-analytics research assistant that grades its own model and reports the losses. $9.99/mo.
-
Read-only US public-data screening tools with free Basic and Evidence tiers.
-
Batru — Competitive Game Intelligence for AI Agents
Live competitive-game predictions, meta data and decision tools for AI agents.
-
Predictive models and analytics for motorsport — race intelligence delivered to your endpoint.
-
mcp-server-lol io.github.ebrunet001/mcp-server-lol
League of Legends AI coaching: player profiles, match history, ranked stats & performance analysis
Has a listing on PyPI (MCP servers) (5)
-
Read and score Opik traces, experiments, datasets and prompts from your AI assistant.
-
Read, measure and explain LLM eval runs of a digline suite. Cannot promote a baseline.
-
Judgment calls on small, cheap models: is this spam, which team, how urgent. YES, NO or UNSURE.
-
air-blackbox-mcp io.github.airblackbox/air-blackbox-mcp
EU AI Act compliance scanner for Python AI agents. 14 tools across 6 articles.
-
Self-hosted Intervals.icu MCP server: training analytics, Garmin data, wellness, workout planning
Has a listing on npm registry (MCP servers) (5)
-
Typed AI judgments for agents: choice, score, and yes/no probabilities instead of prose.
-
140 deterministic tools for your AI: logs, tests, SQL, diagrams, SPARQL. Local, no network.
-
whoop-mcp io.github.davidmosiah/whoop-mcp
Privacy-first, unofficial WHOOP MCP server for AI health, sleep, recovery, and performance agents.
-
Local MCP server for AI coding safety: risk checks, sessions, checkpoints, and fix prompts.
-
Git content governance, 23 deterministic MCP tools. Hosted: docs.contentrain.io/developer/mcp-cloud
Found by meaning, not by shared code or a shared repository: similar plugins stay separate products, and this is not a review or a recommendation.
Want this mark on your plugin in our catalog?
Prove you maintain it with a file in its repository, a file on your domain or a DNS record, and you can show this mark on the card, with live banners for your site and README. How verification works
You also get alerts when a marketplace lists it, drops it, changes its verified mark or what it asks for, and its remote endpoint goes on monitoring as a web service.
Data comes from the Official MCP Registry (registry.modelcontextprotocol.io), CC0 1.0. Figures come from our own regular passes over the marketplaces, last read on . We publish no contact details for plugin publishers.