Agent Monitoring
Tracing, logging and tracking cost and health of LLM apps. As of 2026-11-21, 515 indexed products match this subcategory (262 with full profiles); 166 were published in the last 90 days, against 118 in the 90 days before, slower than the catalogue as a whole. By GitHub stars, iFixAi leads, holding 11% of the stars of the 414 products here with their own repository. The most common form is MCP servers (63% of indexed products).
SubcategorySegment label: Lagging the catalogue DER
Segment statistics
Whole catalogue (all 515 indexed products, not only profiled ones). Calculated 2026-11-21.
Lagging the catalogue. Mid-sized for its level, with launches growing more slowly than the catalogue’s. Size is ranked against the other subcategory nodes; momentum compares this node’s launches in the last 90 days with the 90 before, relative to the whole catalogue’s ratio (score 62 or more = faster, 38 or less = slower; below 5 dated launches it is not measured and the label uses size alone).
Concentration. The Herfindahl–Hirschman index (HHI) adds up each project’s squared share of stars. At 0.045, attention is spread as if across about 22 equally popular projects. Using the common competition-policy thresholds (0.15 and 0.25), that is unconcentrated. Stars measure attention, not usage or quality.
Which product forms serve it
The same need is met by MCP servers, agent skills, plugins, extensions and frameworks. This counts each form across the whole catalogue.
| Product type | Indexed | Share | With a profile |
|---|---|---|---|
| MCP servers | 322 | 62.5% | 83 |
| Infrastructure | 100 | 19.4% | 94 |
| Agent skills | 33 | 6.4% | 33 |
| Plugins | 24 | 4.7% | 19 |
| Developer tools | 9 | 1.7% | 9 |
| Libraries | 6 | 1.2% | 5 |
| Agent projects | 5 | 0.97% | 5 |
| SaaS products | 4 | 0.78% | 4 |
| Frameworks | 4 | 0.78% | 4 |
| Agent SDKs | 2 | 0.39% | 2 |
| Rules packs | 2 | 0.39% | 0 |
| IDE extensions | 2 | 0.39% | 2 |
| Protocols & standards | 1 | 0.19% | 1 |
| AI APIs | 1 | 0.19% | 1 |
5 capabilities
Indexed = whole catalogue; profiled = products with their own page. A product can sit in several, so the rows do not add up to the total.
| Capability | Indexed | Profiled | New, 90 days | Label |
|---|---|---|---|---|
| TracingFollowing each step and tool call of an agent run. | 177 | 106 | 60 | Lagging the catalogue |
| Cost MonitoringTracking token usage and spend. | 152 | 58 | 53 | Lagging the catalogue |
| Failure DetectionCatching hallucinations, errors and drift. | 85 | 36 | 21 | Lagging the catalogue |
| Performance MonitoringTracking latency and throughput of model calls. | 26 | 13 | 11 | Emerging |
| LoggingCapturing prompts, responses and agent events. | 4 | 4 | 3 | Thin |
Top 40 products by GitHub stars
Search the directoryProfiled products only (262 of 515), ranked by the stars of their own repository. Products that live in a subfolder of a larger repository (common for skills) show that whole repository's stars and are ranked after them; products without a repository come last. Stars are popularity, not quality or installs.
| # | Product | Type | Publisher | Capabilities | GitHub stars | Last push | Licence | Save and compare |
|---|---|---|---|---|---|---|---|---|
| 1 | iFixAiIndependent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds. | Infrastructure | ifixai-ai | 47k | 1d ago | Apache-2.0 | ||
| 2 | LangfuseOpen-source LLM engineering platform for tracing, prompt management, evaluations and datasets, self-hosted or Langfuse Cloud. | Developer tool | ClickHouse (Langfuse) | 38k | today | MIT | ||
| 3 | OpenAI Agents SDKLightweight SDK for multi-agent workflows with agents, handoffs, guardrails, sessions and built-in tracing; works with OpenAI and other model providers. | SDK | openai | 31k | today | MIT | ||
| 4 | MLflowOpen-source ML/GenAI lifecycle platform whose GenAI features include OpenTelemetry-compatible tracing, evaluation and prompt registry for agents. | Developer tool | Databricks (Linux Foundation) | 29k | today | Apache-2.0 | ||
| 5 | OpikOpen-source platform for tracing, evaluating and monitoring LLM apps and agents, with LLM-as-judge metrics and prompt optimisation. | Developer tool | Comet | 23k | today | Apache-2.0 | ||
| 6 | serve☁️ Build multimodal AI applications with cloud-native stack | Infrastructure | jina-ai | 22k | 1y ago | Apache-2.0 | ||
| 7 | deepevalSkills for adding DeepEval evaluations, tracing, datasets, Confident AI reports, and iterative improvement loops to AI applications. | Plugin | confident-ai | 20k | 1d ago | Apache-2.0 | ||
| 8 | RagaAI-CatalystPython SDK for Agent AI Observability, Monitoring and Evaluation Framework. Includes features like agent, llm and tools tracing, debugging multi-agentic system, self-hosted dashboard and advanced analytics with timeline and execution graph view | Infrastructure | raga-ai-hub | 16k | 9mo ago | Apache-2.0 | ||
| 9 | Arize PhoenixSelf-hostable tracing and evaluation tool built on OpenTelemetry/OpenInference for inspecting LLM and agent runs, datasets and experiments. | Developer tool | Arize-ai | 12k | today | Elastic-2.0 | ||
| 10 | CodeBurnAI coding spend, tokens and plan quotas across Claude Code, Codex, Cursor and 40+ more tools | MCP server | getagentseal | 12k | 1d ago | MIT | ||
| 11 | VoltAgentTypeScript agent framework with memory, workflows, supervisor/sub-agent patterns and a companion observability console. | Framework | VoltAgent | 11k | 15d ago | MIT | ||
| 12 | failproofaiObservability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement. | Infrastructure | FailproofAI | 9.9k | today | — | ||
| 13 | BifrostGo-based LLM gateway exposing many providers through one OpenAI-compatible API with failover, load balancing, governance and an MCP gateway. | Infrastructure | Maxim AI | 9.9k | today | Apache-2.0 | ||
| 14 | evidentlyEvidently is an open-source ML and LLM observability framework. Evaluate, test, and monitor any AI-powered system or data pipeline. From tabular data to Gen AI. 100+ metrics. | Infrastructure | evidentlyai | 8.1k | 1mo ago | Apache-2.0 | ||
| 15 | llm-gatewayOpen Source LLM Gateway 🦚 | Infrastructure | mnfst | 7.7k | 4d ago | MIT | ||
| 16 | OpenLLMetryOpenTelemetry-based instrumentation for LLM providers, vector DBs and frameworks that exports traces to any OTel backend. | Library | traceloop | 7.5k | 2d ago | Apache-2.0 | ||
| 17 | greptimedbThe open-source observability database. One columnar engine for metrics, logs, and traces, on object storage. | Agent project | GreptimeTeam | 6.8k | today | Apache-2.0 | ||
| 18 | mission-controlSelf-hosted control plane for AI agents: dispatch tasks, review runs, track spend, and operate OpenClaw, Claude Code, Codex, and other runtimes. | Agent project | builderz-labs | 6.5k | 4d ago | MIT | ||
| 19 | HeliconeProxy-based LLM observability that logs requests, costs and latency with one-line integration, plus an AI gateway. | Developer tool | Mintlify (Helicone) | 6.3k | 4d ago | Apache-2.0 | ||
| 20 | tokscale🛰️ Track token usage across AI coding agents from your terminal. 🏅 Global leaderboard with trillions of tokens tracked. | Agent project | junhoyeo | 5.9k | 3d ago | MIT | ||
| 21 | coze-loopNext-generation AI Agent Optimization Platform: Cozeloop addresses challenges in AI agent development by providing full-lifecycle management capabilities from development, debugging, and evaluation to monitoring. | Infrastructure | coze-dev | 5.8k | 4d ago | Apache-2.0 | ||
| 22 | LangWatchPlatform for LLM tracing, evaluations and agent simulation testing, self-hosted or cloud. | Developer tool | langwatch | 5.0k | today | Apache-2.0 | ||
| 23 | AgentaOpen-source LLMOps platform for prompt management, evaluation and observability with a playground for comparing variants. | Developer tool | Agenta | 4.9k | today | MIT | ||
| 24 | all-agentic-architectures35 production-grade agentic AI architectures (Reflexion, LATS, GraphRAG, MemGPT, Voyager, BrowserAgent, ...) — a Python library and runnable textbook with multi-provider LLM support and a 17-task benchmark leaderboard. | Agent project | FareedKhan-dev | 4.8k | 5d ago | MIT | ||
| 25 | latitude-llmOpen-source observability for AI agents. Find where your agents fail, dispatch your coding agent to fix it, and verify the fix against real traces. | Infrastructure | latitude-dev | 4.8k | 3d ago | MIT | ||
| 26 | Pydantic LogfireOpenTelemetry-based observability service with first-class tracing for LLM calls, Pydantic AI agents and general Python apps, queryable with SQL. | SaaS product | pydantic | 4.6k | today | MIT | ||
| 27 | LaminarOpen-source observability platform for agents with tracing, browser-session recordings, evaluations and SQL over trace data. | Developer tool | Laminar | 3.5k | 1d ago | Apache-2.0 | ||
| 28 | openlitSame agent, Smaller bill. The tools and infrastructure to get more work from your AI. Fewer tokens. Less compute. Lower cost. | Agent project | openlit | 3.1k | 2d ago | Apache-2.0 | ||
| 29 | Korean Law MCPKorean law: statutes, precedents, ordinances, treaties and citation verification (법제처 API). | MCP server | chrisryugj | 2.9k | today | MIT | ||
| 30 | hamiltonApache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does. | Infrastructure | apache | 2.6k | 5d ago | Apache-2.0 | ||
| 31 | uptrainUpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them. | Infrastructure | uptrain-ai | 2.4k | 2y ago | Apache-2.0 | ||
| 32 | inspectorTesting and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps. | MCP server | MCPJam | 2.3k | today | — | ||
| 33 | future-agiOpen-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0. | Infrastructure | future-agi | 2.3k | today | Apache-2.0 | ||
| 34 | gortexHigh-performance code-intelligence engine for AI agents and IDE, supports 257 languages, multi repositories, based on graph, with access via CLI, MCP Server, and API. AI coding agents teammate - expose only needed information, cutting token usage up to 50x. 100% local. Discord: https://discord.gg/39MFHu3J5d | MCP server | zzet | 2.1k | today | Apache-2.0 | ||
| 35 | Stripe Agent ToolkitLibraries and an MCP server that let agent frameworks call Stripe APIs, plus helpers for metering and billing LLM token usage. | SDK | Stripe | 2.1k | today | MIT | ||
| 36 | trpc-agent-goA Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability. | MCP server | trpc-group | 1.9k | today | Apache-2.0 | ||
| 37 | SuperlogOpen-source agent that observes and fixes your application. Query logs, traces, metrics, incidents. | MCP server | superlog.sh | 1.5k | 2d ago | Apache-2.0 | ||
| 38 | traceroottraceroot: the open-source platform that continuously improves your ai agents. | Infrastructure | traceroot-ai | 1.4k | today | — | ||
| 39 | AutoTSAutomated time series forecasting with model search, anomaly detection, and event risk analysis | MCP server | winedarksea | 1.4k | 2d ago | MIT | ||
| 40 | reverifyStop your AI from making things up — it proposes, deterministic tools decide, every claim checked against ground truth with evidence. Grounded facts and context survive resets. Reverse engineering is the proving ground. MCP server + CLI. | MCP server | 2akouwu | 1.4k | 20d ago | MIT |
Most common integrations
Third-party services named by products in this node, whole catalogue.
| Integration | Products | Share |
|---|---|---|
| Google Cloud | 8 | 1.6% |
| Azure | 6 | 1.2% |
| AWS | 5 | 1.0% |
| PostgreSQL | 5 | 1.0% |
| Datadog | 3 | 0.6% |
| GitHub | 3 | 0.6% |
| Grafana | 3 | 0.6% |
| Kubernetes | 3 | 0.6% |
| Terraform | 2 | 0.4% |
| Android | 1 | 0.2% |
| Blender | 1 | 0.2% |
| Discord | 1 | 0.2% |
| Google Ads | 1 | 0.2% |
| Meta Ads | 1 | 0.2% |
| Microsoft Excel | 1 | 0.2% |
Launches per month
New Agent Monitoring products per monthSRC
Whole catalogue, by publication month, 2024-12 to 2026-11. The last month is partial; undated products are excluded.
Show the data as a table
| Products published (products) | |
|---|---|
| 2024-12 | 2 |
| 2025-01 | 2 |
| 2025-02 | 1 |
| 2025-03 | 3 |
| 2025-04 | 2 |
| 2025-05 | 4 |
| 2025-06 | 1 |
| 2025-07 | 2 |
| 2025-09 | 6 |
| 2025-10 | 4 |
| 2025-11 | 2 |
| 2025-12 | 6 |
| 2026-01 | 2 |
| 2026-02 | 18 |
| 2026-03 | 31 |
| 2026-04 | 36 |
| 2026-05 | 33 |
| 2026-06 | 30 |
| 2026-07 | 40 |
| 2026-08 | 57 |
| 2026-09 | 44 |
| 2026-10 | 64 |
| 2026-11 | 44 |
Related opportunities
Opportunity radarScored gaps for this node or anything beneath it. Scores rank hypotheses; they do not predict demand.
| Opportunity | Competition | Confidence | Score |
|---|---|---|---|
| Coverage gap: LoggingLogging | Medium | Low | 59.1 |
| Coverage gap: Performance MonitoringPerformance Monitoring | High | Low | 55.5 |
Limits of these numbers
- Products are placed here by whole-word keyword rules on their name, description and repository topics, and inherit every ancestor of a matched node. Products that do this without naming it are missed; ambiguous words cause false matches. Methodology
- About half of the catalogue matches no node at all, so absolute counts understate every category.
- Launch counts use the publication date each source reports; for auto-discovered repositories that is the repository’s creation date.
- Stars and concentration cover only products with their own repository (not a subfolder of a larger one). They measure attention, not adoption.
Page data last updated .