io.github.ryuamarines/robots-txt-checker
Checks if a URL path is allowed by robots.txt. x402 payment required (testnet USDC).
Servidor MCP · de ryuamarines
Resumen
io.github.ryuamarines/robots-txt-checker es un servidor MCP publicado por ryuamarines. Marketplaces que leemos y que lo publican: 1 (Official MCP Registry). Lo vimos por primera vez el 2026-10-11 y desde entonces no hemos registrado cambios. Según nuestra pasada por los marketplaces del 2026-10-13.
Qué pide
- Se ejecuta en remoto
Ninguno de los marketplaces que leemos publica sus herramientas ni los ajustes que pide.
Lo que declara, pide y ejecuta, completo
Instale solo plugins de desarrolladores en los que confíe. Mostramos lo que publican los marketplaces y no probamos plugins.
Funciona con
- Claude Code
- Codex
- Cursor
- VS Code
- Copilot CLI
Un enlace de un clic abre el cliente, que pide confirmación antes de añadir el servidor.
Según los marketplaces que lo listan y el transporte MCP que declara, tal como lo documenta cada cliente. No probamos plugins.
Conectar
Comandos y configuraciones generados solo a partir de lo que declaran sus marketplaces. No los ejecutamos por usted ni probamos plugins.
Como servidor MCP remoto
Generado a partir de lo que declara Official MCP Registry.
claude mcp add --transport http robots-txt-checker https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp[mcp_servers.robots-txt-checker]
url = "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"{
"mcpServers": {
"robots-txt-checker": {
"url": "https://faceted-napped-debtless.ngrok-free.dev/robots-txt-checker/mcp"
}
}
}Dónde está publicado
| Marketplace | Estado | Versión | Cifras | Verificado | Visto por primera vez |
|---|---|---|---|---|---|
| Official MCP Registry (Abrir el registro) | Publicado | 1.0.93057 | Sin cifra publicada | Sin marca |
Cada cifra es del propio marketplace y nunca se suma entre marketplaces. Un marketplace que no publica cifras no muestra ninguna: sin número no es cero.
Qué cambió
Nada ha cambiado en este plugin desde que lo seguimos.
Plugins similares
Productos cuyos propios nombres, descripciones, herramientas y skills son los más cercanos a este por su significado, en todos los marketplaces que leemos. La cercanía se ordena solo dentro de esta lista.
-
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
-
Analyse robots.txt content and report whether a path may be crawled.
-
robotstxt-ai io.github.sharozdawa/robotstxt-ai
Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.
-
robots-txt-ai-crawler-audit io.github.jmshinhwa/robots-txt-ai-crawler-audit
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
-
ActableSite AI Crawler Monitor
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
-
ai-visibility com.lastminutedealshq/ai-visibility
AI-search-visibility tools: check crawler access, generate llms.txt, lint robots.txt, census data.
-
Fetch a public robots.txt and return group counts only.
-
Can GPTBot, ClaudeBot and PerplexityBot reach a site? Real fetch per crawler, firewall/CDN blocks.
-
AI visibility checker: GPTBot vs OAI-SearchBot, robots.txt audit, raw-HTML schema. 165 crawlers.
-
ai-crawler-readiness com.lumnika/ai-crawler-readiness
Can AI crawlers read this site? GPTBot/ClaudeBot verdicts, robots vs edge blocking, 47k-domain index
Plugins similares por marketplace
La misma pregunta a cada marketplace por separado: ¿hay allí un equivalente de este plugin?
Publicado en Claude Code: claude-plugins-community (5)
-
Inspect, debug, and test Machine Payments Protocol (HTTP 402) endpoints directly from Claude Code. Parses spec-compliant WWW-Authenticate challenges, discovers MPP endpoints on…
-
Skyvern browser automation skills for web scraping and interaction via remote MCP.
-
Data freshness and endpoint liveness verification for AI agents. Probes external APIs and endpoints before agents act on their data, returning deterministic FRESH/STALE/UNKNOWN…
-
Web scraping, Google search, structured data extraction, and MCP server integration powered by Bright Data. Includes 7 skills: scrape any webpage as markdown (with bot…
-
Agent safety infrastructure (RiskOracle, NoLeak, MemGuard, SecureExec, FlowCore) and real-time DELPHI Oracle intelligence signals. x402 USDC micropayments on Base Mainnet. 19…
Publicado en Claude Code: claude-plugins-official (5)
-
Web scraping, Google search, structured data extraction, and MCP server integration powered by Bright Data. Includes 7 skills: scrape any webpage as markdown (with bot…
-
Give Claude a real browser — your Chrome or a Browser Use Cloud browser. Use it whenever a task involves a website or web app: browsing, scraping and data extraction, filling…
-
playwright-mcp io.github.microsoft/playwright-mcp
Playwright Tools for MCP
-
Automate API security directly in Claude Code with 42Crunch - automatically audit OpenAPI specs, detect vulnerabilities aligned with OWASP API Security risks (including…
-
Web scraping skills for Claude Code powered by the Zyte API — scrape sites, generate and run Scrapy spiders, define extraction schemas, and ship to Scrapy Cloud.
Publicado en Claude: knowledge-work-plugins (5)
-
Web scraping, Google search, structured data extraction, and MCP server integration powered by Bright Data. Includes 7 skills: scrape any webpage as markdown (with bot…
-
Give Claude a real browser — your Chrome or a Browser Use Cloud browser. Use it whenever a task involves a website or web app: browsing, scraping and data extraction, filling…
-
The complete web toolkit for your agent. Search the web and get answers in milliseconds. Fetch any URL and get clean markdown content back. Send a browser agent to navigate sites…
-
Exa AI web search, deep research, and content extraction. Provides MCP tools and research skills for comprehensive web search, people discovery, company research, academic papers…
-
Configure, run, and interpret HawkScan DAST results inside Claude Code. Generates stackhawk.yml configs, runs scans via CLI or Docker, and transforms security findings into…
Publicado en Codex: openai-curated (openai/plugins) (5)
-
Cloudflare platform plugin with curated skills for Workers, Wrangler, and Agents SDK plus the official Cloudflare API MCP server.
-
Evaluate Codex skills and plugins from chat with a beginner-friendly start command, local-first reports, token budget explanations, and guided benchmarking.
-
Codex Security workflows for security scans, analysis, and investigation.
-
Build with OpenAI APIs, Agents SDK, and ChatGPT Apps, and create and save OpenAI API keys from Codex.
-
Skills for NVIDIAs ecosystem spans GPU acceleration, CUDA, AI agents, inference, robotics, Physical AI, Omniverse, and simulation. This plugin helps you understand the pieces…
Publicado en Community GitHub marketplaces (5)
-
Browser automation and web page analysis skills using playwright-cli: connect via CDP, probe bot protection, dismiss overlays, capture DOM trees, reduce pages to skeletons…
-
HTTP/REST client with saved collections, env secrets, and SSRF-safe host allowlisting.
-
Web scraping, Google search, structured data extraction, and MCP server integration powered by Bright Data. Includes 7 skills: scrape any webpage as markdown (with bot…
-
Give Claude a real browser — your Chrome or a Browser Use Cloud browser. Use it whenever a task involves a website or web app: browsing, scraping and data extraction, filling…
-
agentforge io.github.acarchidi/agentforge
DeFi safety layer for AI agents: wallet safety, token risk, tx decode/simulate. 20 tools.
Publicado en Copilot CLI: github/awesome-copilot (5)
-
Helps devs extract data from any website with thousands of trusted scrapers, crawlers, and automations from Apify Store. Integrate ready-made Actors to access all popular social…
-
Safe BLE diagnostics, evidence workflows, and guarded automation for AI agents.
-
Three-layer verification pipeline for AI output. Extracts claims, finds sources, and flags hallucination risks so humans can verify before acting.
-
Visualizes live session AI-credit usage as a Pac-Man board with pellets, ghosts, fruit milestones, and game-over limits.
-
MCP server for Chrome DevTools
Publicado en Copilot CLI: github/copilot-plugins (5)
-
Advanced Security plugin for GitHub Copilot.
-
Toolkit for building Microsoft 365 Copilot declarative agents — scaffolding, JSON manifest development, and capability configuration.
-
Build Power Apps Canvas Apps using the Canvas Authoring MCP server.
-
mcp-apps
Generate MCP App widgets for MCP tools. Produces self-contained HTML widgets using the MCP Apps protocol.
-
Bring Azure DevOps work-item, repository, and pull-request workflows into GitHub Copilot.
Publicado en Cursor: cursor/plugins (5)
-
Browse and automate the web with TinyFish: run multi-step web automations, search the web, extract clean page content, and drive remote stealth browser sessions.
-
playwright-mcp io.github.microsoft/playwright-mcp
Playwright Tools for MCP
-
Build apps, scripts, and automations with the TypeScript SDK.
-
CLI-backed repo compatibility scans plus agents that audit startup, validation, and docs against reality.
-
Fan large tasks out across parallel cloud agents with planners, workers, verifiers, and structured handoffs.
Publicado en Docker MCP Catalog (5)
-
🔥 Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor, Claude and any other LLM clients.
-
Apify is the world's largest marketplace of tools for web scraping, data extraction, and web automation. You can extract structured data from social media, e-commerce, search…
-
oxylabs-mcp io.oxylabs/oxylabs-mcp
Fetch and process content from specified URLs & sources using the Oxylabs Web Scraper API.
-
mcp-server-browserbase io.github.browserbase/mcp-server-browserbase
MCP server for AI web browser automation using Browserbase and Stagehand
-
Give your AI agents trusted access to the full Postman platform.
Publicado en Official MCP Registry (5)
-
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}} free no key
-
Analyse robots.txt content and report whether a path may be crawled.
-
robotstxt-ai io.github.sharozdawa/robotstxt-ai
Analyze and generate robots.txt with AI crawler detection for 20+ bots like GPTBot and ClaudeBot.
-
robots-txt-ai-crawler-audit io.github.jmshinhwa/robots-txt-ai-crawler-audit
Checks robots.txt line by line and says what each AI crawler token actually controls - training, AI
-
ActableSite AI Crawler Monitor
Check AI crawler robots.txt policy and monitor public-site policy, sitemap, and llms.txt changes.
Encontrados por significado, no por código ni repositorio compartidos: los plugins similares siguen siendo productos distintos, y esto no es una reseña ni una recomendación.
¿Es tu plugin?
Añádelo a tus proyectos y demuestra que lo mantienes con un archivo en su repositorio, un archivo en tu dominio o un registro DNS. Recibirás avisos cuando un marketplace lo publique, lo retire, cambie su marca verificada o lo que pide, y su endpoint remoto quedará monitorizado como un servicio web.
Datos de Official MCP Registry. Las cifras proceden de nuestros propios recorridos periódicos por los marketplaces, leída por última vez el . No publicamos datos de contacto de los editores de plugins.