AgentSidekick

Iris

MCP serverby iris-evalSRC

Stop shipping agents on vibes. Score every agent output for quality, safety, and cost.

SRC Publisher-stated: these are the publisher's own words, not an assessment.

WebsiteRepositorySource listing

Key facts

Type
MCP server · mcp-localSRC
Publisher
iris-eval · organizationSRC
Licence
MITSRC
Open source
YesDER
Hosting
Local (runs on your machine)SRC
Latest version
0.21.0SRC
Languages
typescriptSRC
First published
SRC
In this catalogue since
DER
Last verified
Not verified by a researcher. Last seen in a source on 2026-11-21.

Capabilities and categories

Assigned by keyword rules from the description unless marked otherwise. The matched word is shown so you can judge the match.

CapabilityEvidenceKind
Agent Evaluationmatched agent evaluationDER
Application Monitoringmatched observabilityDER
Tracingmatched tracingDER

Filed underAgent InfrastructureDeveloper ToolsAgent ObservabilityDevOps & SREAgent Monitoring

Integrations DER

No third-party service is named in the description or package metadata.

Industries HEU

Platform compatibility

No pairing has been tested by AgentSidekick. Each status says what kind of evidence exists, not whether it works for you.

Protocol-level29

Each platform listed officially documents support for MCP. This specific pairing has not been tested.

Platform discontinued1
PlatformBasis
Roo CodeDiscontinuedRoo Code documented support for MCP but is discontinued. Project shut down and repository archived.
Unknown17
PlatformBasis
OpenClawCould not confirm whether OpenClaw supports MCP.
OpenHandsOpenHands documents MCP support, but it runs only on web, api, slack surfaces and this server runs locally as a process on your machine. Whether OpenHands can reach a local server is not established.
AiderCould not confirm whether Aider supports MCP.
Bolt (bolt.new)Bolt (bolt.new) documents MCP support, but it runs only on web surfaces and this server runs locally as a process on your machine. Whether Bolt (bolt.new) can reach a local server is not established.
Amazon Q DeveloperWinding downCould not confirm whether Amazon Q Developer supports MCP.
Amazon Bedrock Agents (Agents Classic)Maintenance onlyCould not confirm whether Amazon Bedrock Agents (Agents Classic) supports MCP.
GPTs (custom GPTs)Winding downCould not confirm whether GPTs (custom GPTs) supports MCP.
Gemini (app)Gemini (app) documents MCP support, but it runs only on web, mobile surfaces and this server runs locally as a process on your machine. Whether Gemini (app) can reach a local server is not established.
Gemini Enterprise Agent Platform (formerly Vertex AI / Vertex AI Agent Builder)Gemini Enterprise Agent Platform (formerly Vertex AI / Vertex AI Agent Builder) documents MCP support, but it runs only on web, api surfaces and this server runs locally as a process on your machine. Whether Gemini Enterprise Agent Platform (formerly Vertex AI / Vertex AI Agent Builder) can reach a local server is not established.
GitHub Copilot ExtensionsDiscontinuedCould not confirm whether GitHub Copilot Extensions supports MCP.
Google AI StudioCould not confirm whether Google AI Studio supports MCP.
JulesWinding downCould not confirm whether Jules supports MCP.
Microsoft CopilotCould not confirm whether Microsoft Copilot supports MCP.
Microsoft Copilot StudioMicrosoft Copilot Studio documents MCP support, but it runs only on web surfaces and this server runs locally as a process on your machine. Whether Microsoft Copilot Studio can reach a local server is not established.
Replit AgentReplit Agent documents MCP support, but it runs only on web surfaces and this server runs locally as a process on your machine. Whether Replit Agent can reach a local server is not established.
Sourcegraph CodyCould not confirm whether Sourcegraph Cody supports MCP.
v0v0 documents MCP support, but it runs only on web, api surfaces and this server runs locally as a process on your machine. Whether v0 can reach a local server is not established.

Install

Read from registry metadata. Follow the publisher's instructions; we have not run this.

Packages
RegistryPackageTransportRuntime
mcpbhttps://github.com/iris-eval/mcp-server/releases/download/v0.21.0/iris-eval.mcpbstdionot stated
npm@iris-eval/mcp-serverstdionot stated

Environment variables SRC

NameRequiredSecretPublisher's description
IRIS_API_KEYNoSecretAPI key for HTTP authentication. Required to bind the HTTP transport or the dashboard beyond loopback (0.0.0.0, a LAN address, a container): without it the server refuses to start
IRIS_API_KEY_FILENoNoPath to a file whose trimmed contents are the API key (the secret-file pattern Docker and Kubernetes mount). Set this or IRIS_API_KEY, not both; further keys and rotation are security.apiKeys in config.json
IRIS_ALLOW_UNAUTHENTICATEDNoNoSet to 1 to run a non-loopback bind with NO API key on purpose (lifts the startup refusal; the network is then your boundary)
IRIS_DB_PATHNoNoSQLite database path
IRIS_SQLITE_DRIVERNoNoWhich SQLite driver holds the database: native (better-sqlite3, the default) or node (Node's built-in node:sqlite, Node 22.13+). Unset: native, falling back to node when the native module cannot load
IRIS_SEARCH_BUDGET_MSNoNoHow long one trace search may read before it answers with the matches found so far, in milliseconds (50-60000, default 1000)
IRIS_SEARCH_INDEXNoNoon (default) or off. off keeps no full-text index of the traces: writes are cheaper, and a search reads the traces within its time budget
IRIS_LOG_LEVELNoNoLog level: debug, info, warn, error
IRIS_HOMENoNoDirectory for the database, custom rules and preferences (default ~/.iris)
IRIS_ANTHROPIC_API_KEYNoSecretEnables evaluate_with_llm_judge and verify_citations with an Anthropic model. Optional: the deterministic rules never need a key
IRIS_OPENAI_API_KEYNoSecretEnables evaluate_with_llm_judge and verify_citations with an OpenAI model. Optional: the deterministic rules never need a key
IRIS_LLM_JUDGE_MAX_COST_USD_PER_EVALNoNoHard cap on the worst-case cost of one LLM judge call (default 0.25); max_cost_usd can lower it for a call, never raise it
IRIS_RELEVANCE_JUDGE_MODELNoNoA priced judge model; answers_the_ask then asks it on every evaluation with input, sending that input and output to the model's provider, and gates on its verdict (off by default)
IRIS_LLM_JUDGE_DAILY_BUDGET_USDNoNoWhat every judge call together may spend per UTC day, per tenant, in USD (default 1): the relevance judge, evaluate_with_llm_judge and verify_citations. Past it no judge call is made
IRIS_RELEVANCE_JUDGE_DAILY_BUDGET_USDNoNoThe name of IRIS_LLM_JUDGE_DAILY_BUDGET_USD before 0.21.0. Read when the new name is unset, and it then limits every judge call
IRIS_RELEVANCE_JUDGE_MAX_CALLS_PER_REQUESTNoNoRelevance judge calls one request may make (default 20)
IRIS_RELEVANCE_JUDGE_REDACTNoNoon (default) replaces the PII and credentials no_pii flags before the relevance judge sends input and output to its provider; off sends them as they are
IRIS_CITATION_ALLOW_FETCHNoNoLet verify_citations fetch URLs (off by default); a tool argument can turn it off for a call, never on
IRIS_DASHBOARDNoNoStart the dashboard alongside the server (same as --dashboard)
IRIS_DASHBOARD_PORTNoNoDashboard port (default 6920); IRIS_PORT is the MCP HTTP transport port (default 3000)
IRIS_PORTNoNoMCP HTTP transport port (default 3000); only used with --transport http
IRIS_OTEL_ENDPOINTNoNoOTLP/HTTP collector URL; when set, every stored trace is also exported as OpenTelemetry spans (best-effort, asynchronous)
IRIS_CITATION_DOMAINSNoNoComma-separated domain allowlist for verify_citations fetches (only consulted when IRIS_CITATION_ALLOW_FETCH is on); a tool argument can narrow it, never add to it
IRIS_CITATION_MAX_COST_USD_TOTALNoNoThe most one verify_citations call may spend on judge calls, in USD (default 1); max_cost_usd_total can lower it, never raise it
IRIS_TRANSPORTNoNoTransport: stdio (default) or http; same as --transport
IRIS_HOSTNoNoBind host for the HTTP transport (default 127.0.0.1); only used with --transport http
IRIS_ALLOWED_ORIGINSNoNoComma-separated Origin allowlist for the HTTP transport and dashboard CORS (default http://localhost:*)
IRIS_MAX_INPUT_ELEMENTSNoNoThe most array elements and object keys one input may hold before it is validated (default 100000): a tool call's arguments, and a dashboard or OTLP JSON request body. A tool call over it gets an error result; a body, 413. OTLP protobuf has its own ceiling, one element per 5 bytes of the request size limit
IRIS_MAX_TOOL_INPUT_ELEMENTSNoNoThe name of IRIS_MAX_INPUT_ELEMENTS before 0.21.0, read when the new name is unset
IRIS_DASHBOARD_HOSTNoNoBind host for the dashboard (default 127.0.0.1); same as --dashboard-host
IRIS_OTEL_HEADERSNoNoComma-separated key=value headers sent with every OTLP export (for example an auth token for your collector)
IRIS_OTEL_SERVICE_NAMENoNoservice.name resource attribute on exported spans (default iris-eval)
IRIS_OTEL_TIMEOUT_MSNoNoTimeout in milliseconds for each OTLP export request
IRIS_WEBHOOK_URLNoNoThe receiver of the webhook that fires on a moment (docs/webhooks.md); merged over notify.webhook in config.json
IRIS_WEBHOOK_SECRETNoSecretThe signing key of that webhook (any string, or whsec_ + base64); the iris format refuses to run without one

Permissions and signals

Signals, not a safety verdict. They describe what the product's metadata references. Security review: not reviewed.

SignalEvidenceHow observedObserved
Runs a local processruns locally via mcpb,npmregistry metadata 2026-11-21
Asks for a secretdeclares secret env var(s): IRIS_API_KEY, IRIS_ANTHROPIC_API_KEY, IRIS_OPENAI_API_KEY, IRIS_WEBHOOK_SECRETregistry metadata 2026-11-21

Known advisories for the published package SRC

An advisory existing does not mean the listed version is affected: compare the vulnerable range with the version you install. None found is not evidence of safety.

No advisories found

No published advisory matches a package this product declares. Products without a declared package cannot be matched at all. This is not evidence of safety.

Evaluation

Separate dimensions, each with its evidence. There is no overall score: the dimensions do not add up to one number honestly.

Source authority

Publisher-submitted

Published to the official MCP Registry by the namespace owner. Field values are the publisher's own claims.

Publisher verification

Namespace verified by registry

The MCP Registry verifies namespace ownership (GitHub login or DNS) before publishing. This confirms who published it, not what it does.

Maintenance activity

Active

Last push 2026-11-19 (2 days before this page was built).

Licence clarity

MIT

Licence identifier as reported by GitHub repository metadata.

Installation clarity

Machine-readable

2 package(s) and 0 remote endpoint(s) declared in registry metadata.

Documentation

Repository / website only

No dedicated documentation URL on record. AgentSidekick has not assessed documentation quality.

Compatibility evidence

Protocol-level only

Compatibility is inferred from the standard it implements. No pairing has been tested.

Security review

Not reviewed

AgentSidekick has not audited this product. 2 automated signal(s) are listed under Permissions & signals — they describe what it references, not whether it is safe.

Pricing

Not published in any source we track. The code is open source, which says nothing about the cost of any hosted service.

Repository activity

iris-eval/mcp-server. Read from the GitHub API on 2026-11-21.

10SRCGitHub stars
5SRCForks
33SRCOpen issues
2d agoSRCLast push2026-11-19
v0.21.0SRCLatest release2026-10-25

Version history SRC

  1. 0.21.0
  2. 0.20.0
  3. 0.19.0
  4. 0.18.0
  5. 0.17.0
  6. 0.16.0
  7. 0.15.0
  8. 0.14.0
27 earlier versions
  1. 0.13.0
  2. 0.12.1
  3. 0.12.0
  4. 0.11.0
  5. 0.10.0
  6. 0.9.0
  7. 0.8.2
  8. 0.8.1
  9. 0.8.0
  10. 0.7.0
  11. 0.6.0
  12. 0.5.1
  13. 0.5.0
  14. 0.4.5
  15. 0.4.4
  16. 0.4.2
  17. 0.4.0
  18. 0.3.0
  19. 0.2.4
  20. 0.2.3
  21. 0.2.2
  22. 0.2.1
  23. 0.1.8
  24. 0.1.7
  25. 0.1.6
  26. 0.1.4
  27. 0.1.3

Change history

What changed between two ingests of the same source, and when the product was first seen. Registry releases are under Version history.

  1. First seen

Competitors and alternativesDER

Paired by shared capabilities and integrations. Similar on paper; not tested side by side.

Direct

Same type, overlapping capabilities.

SharesApplication MonitoringTracing

SharesApplication MonitoringTracingAgent Evaluation

Adjacent

A different form or a partial overlap.

prompty

Infrastructureby microsoftHEU

1.3k GitHub starspushed 12d agoMIT

SharesApplication MonitoringAgent Evaluation

SharesApplication MonitoringTracing

SharesTracingAgent Evaluation

coding.sh

Infrastructureby jackulauHEU

149 GitHub starspushed todayMIT

SharesApplication MonitoringTracingAgent Evaluation

SharesApplication MonitoringTracingAgent Evaluation

MLflow

Developer toolby Databricks (Linux Foundation)CUR

29k GitHub starspushed todayApache-2.0

SharesApplication MonitoringTracingAgent Evaluation

Provenance

Which source set each field, what kind of claim it is, and when.

FieldSourceKindSet on
Canonical URLOfficial MCP RegistrySRC2026-11-21
PublisherOfficial MCP RegistrySRC2026-11-21
DescriptionOfficial MCP RegistrySRC2026-11-21
HostingOfficial MCP RegistrySRC2026-11-21
IconOfficial MCP RegistrySRC2026-11-21
Integration typeOfficial MCP RegistrySRC2026-11-21
LanguagesGitHub repository metadataSRC2026-11-21
Latest versionOfficial MCP RegistrySRC2026-11-21
LicenceGitHub repository metadataSRC2026-11-21
NameOfficial MCP RegistrySRC2026-11-21
Open sourceGitHub repository metadataDER2026-11-21
TypeOfficial MCP RegistrySRC2026-11-21
PublishedOfficial MCP RegistrySRC2026-11-21
RepositoryOfficial MCP RegistrySRC2026-11-21
source_updated_atOfficial MCP RegistrySRC2026-11-21
Source URLOfficial MCP RegistrySRC2026-11-21
WebsiteOfficial MCP RegistrySRC2026-11-21

Sources

Not verified by a researcher. Last seen in a source on . Report incorrect information

Page data last updated .