mcp-probeView on GitHub

Scripted checks for MCP servers

Write the calls you care about in YAML. mcp-probe runs them against your server and exits non-zero on failure.

git clone https://github.com/Kelvis123456/mcp-probe
cd mcp-probe && npm ci && npm run build
node dist/cli.js examples/everything.probe.yaml
Output of that last command
PASS  initialize handshake (1807 ms)PASS  tools/list answers (95 ms)PASS  every tool has a description and an object input schema (0 ms)PASS  required tools exist (echo, get-sum, get-structured-content) (0 ms)PASS  unknown tool is rejected, not ignored (16 ms)PASS  call: echo returns the message (13 ms)PASS  call: get-sum adds two numbers (6 ms)PASS  call: get-sum rejects a string for a number (42 ms)PASS  call: echo without its required argument fails (3 ms)PASS  call: get-structured-content rejects an unknown location (24 ms)PASS  server still responds after all calls (14 ms)All 11 checks passed

A refactor can change a tool without failing anything

An MCP tool can start returning different text after a refactor, or stop rejecting bad input. Nothing breaks until a client trips on it.

I wrote mcp-probe after shipping a few MCP tools with no cheap way to notice that. You describe the server and the calls you care about once. The tool starts the server, talks to it the way a client would, and prints a pass or fail line per check.

Declare the calls, read the report

Both files below are unedited copies from the repo’s examples/ folder. The probe targets the official reference server, pinned to 2.2.1, and all 11 checks pass.

examples/everything.probe.yaml
# Probe for the official reference server (@modelcontextprotocol/server-everything, pinned in examples/package.json).# Run from the repo root: node dist/cli.js examples/everything.probe.yamlserver:  command: node  args: [examples/node_modules/@modelcontextprotocol/server-everything/dist/index.js, stdio]  timeoutMs: 8000 tools:  require: [echo, get-sum, get-structured-content] calls:  - name: echo returns the message    tool: echo    args: { message: hello }    expect: { equals: "Echo: hello" }   - name: get-sum adds two numbers    tool: get-sum    args: { a: 40, b: 2 }    expect: { text: { contains: "42" } }   - name: get-sum rejects a string for a number    tool: get-sum    args: { a: "forty", b: 2 }    expect: { error: true }   - name: echo without its required argument fails    tool: echo    args: {}    expect: { error: true }   - name: get-structured-content rejects an unknown location    tool: get-structured-content    args: { location: Atlantis }    expect: { error: true }
examples/report.md
Report written by mcp-probe for the reference server
ResultCheckms
passinitialize handshake475
passtools/list answers22
passevery tool has a description and an object input schema0
passrequired tools exist (echo, get-sum, get-structured-content)0
passunknown tool is rejected, not ignored4
passcall: echo returns the message4
passcall: get-sum adds two numbers3
passcall: get-sum rejects a string for a number2
passcall: echo without its required argument fails1
passcall: get-structured-content rejects an unknown location2
passserver still responds after all calls2

That result says the reference server behaves the way the probe expects. It is not a claim that the probe finds bugs in it.

A changed expectation turns one line red

To see a failure, I changed the first golden value in that probe from Echo: hello to Hello: hello and ran it again. This is the real output. The process exited with code 1.

Same probe, one golden value changed
PASS  initialize handshake (948 ms)PASS  tools/list answers (48 ms)PASS  every tool has a description and an object input schema (0 ms)PASS  required tools exist (echo, get-sum, get-structured-content) (0 ms)PASS  unknown tool is rejected, not ignored (8 ms)FAIL  call: echo returns the message (8 ms)text differs from golden (expected "Hello: hello", got "Echo: hello")PASS  call: get-sum adds two numbers (2 ms)PASS  call: get-sum rejects a string for a number (2 ms)PASS  call: echo without its required argument fails (2 ms)PASS  call: get-structured-content rejects an unknown location (3 ms)PASS  server still responds after all calls (4 ms)1 of 11 checks failed

Exit codes

0
Every check passes.
1
A check fails.
2
The probe file is invalid, or the tool itself crashed.

A call that never answers is reported as a timeout. It does not count as an expected failure, because a hung tool is exactly what you want the probe to catch.

expect: { error: true } accepts both a thrown protocol error and a result with isError: true, since correct servers do either.

What it checks

Always, with no configuration

initialize
The handshake completes within the timeout.
tools/list
The server answers it. Pagination is followed, up to 100 pages. If it fails, the checks that need the list fail too instead of passing on an empty one.
tool metadata
Every tool has a non-empty description and an input schema of type object.
unknown tool
Calling a tool that does not exist fails instead of being accepted.
ping
The server still answers after all the calls have run.

From your probe file

require
The tools you list exist.
calls
Each call succeeds or fails, as you declare. Optionally its first text block must contain a string, match a regex, or equal a golden value. A call to a tool that is not in tools/list fails whatever you expect, so a typo cannot pass as an expected error. If the result has no text block, equals, contains and matches fail.
strict files
An unknown key in the probe file is an error that names the key. A matches pattern that is not a valid regex fails when the file is loaded.
environment
The server only gets the SDK's small default environment plus what you list under server.env. A relative server.cwd is resolved against the probe file's directory.
CLI flags
Unknown or incomplete flags exit with code 2 instead of being ignored.

What it does not do

  • stdio only. No HTTP or SSE servers yet.
  • Tools only. Resources, prompts and sampling are not probed.
  • Golden values compare the first text block. Other text blocks and structuredContent are not looked at.
  • Not on npm. Clone the repo to use it.
  • Behavior, not strict conformance. It does not validate every message against the spec's JSON schema.

Tested, and audited once

The suite has 32 tests in 3 files. It starts a small real MCP server and runs the probe against it, including a server that hangs, one whose tools/list errors, one that pages its tool list, and one that throws protocol errors on every call.

An independent audit found checks that could pass when they should fail: probe files that silently dropped unknown keys, typo’d tool names satisfying an expected error, timeouts counted as expected failures, and the whole shell environment leaking into the server. All four are fixed. The reasoning is in docs/decisions.md. An audit is not a guarantee that no others remain.