AgentSidekick

Data & Knowledge

Retrieval, databases, analytics, research and files that agents read and write. As of 2026-11-21, 7,219 indexed products match this ecosystem (1,971 with full profiles); 3,575 were published in the last 90 days, against 1,740 in the 90 days before, slower than the catalogue as a whole. By GitHub stars, Firecrawl leads, holding 3% of the stars of the 5,051 products here with their own repository. The most common form is MCP servers (90% of indexed products).

EcosystemSegment label: Crowded, slowing DER

Segment statistics

Whole catalogue (all 7,219 indexed products, not only profiled ones). Calculated 2026-11-21.

7,219DERIndexed products1,971 with full profiles
3,575DERLaunched, last 90 days1,740 in the 90 days before
30DERMomentum vs catalogue50 = same pace as the catalogue
3%DERLargest project's share of starsAcross 5,051 products with their own repository
0.011DERConcentration (HHI)About 94 equal-sized projects’ worth; unconcentrated
1SRCMedian starsPer product with its own repository
63%DEROpen sourceMarked open source; unknown counts as not

Crowded, slowing. Among the largest fifth of nodes at its level, but its launches grew more slowly than the catalogue’s. Size is ranked against the other ecosystem nodes; momentum compares this node’s launches in the last 90 days with the 90 before, relative to the whole catalogue’s ratio (score 62 or more = faster, 38 or less = slower; below 5 dated launches it is not measured and the label uses size alone).

Concentration. The Herfindahl–Hirschman index (HHI) adds up each project’s squared share of stars. At 0.011, attention is spread as if across about 94 equally popular projects. Using the common competition-policy thresholds (0.15 and 0.25), that is unconcentrated. Stars measure attention, not usage or quality.

Which product forms serve it

The same need is met by MCP servers, agent skills, plugins, extensions and frameworks. This counts each form across the whole catalogue.

Products in Data & Knowledge by product type, as of 2026-11-21
Product type IndexedShareWith a profile
MCP servers6,50890.2%1,294
Agent skills2473.4%245
Infrastructure1492.1%148
Plugins1131.6%98
Frameworks630.87%62
Agent projects530.73%52
Libraries290.40%27
Rules packs210.29%9
IDE extensions120.17%12
AI APIs90.12%9
SaaS products60.08%6
Automation platforms40.06%4
Platforms30.04%3
Developer tools20.03%2

6 categories

Indexed = whole catalogue; profiled = products with their own page. A product can sit in several, so the rows do not add up to the total.

CategoryIndexed ProfiledNew, 90 daysLabel
Public & Real-world DataMaps, weather, public datasets and other real-world information.2,0111881,113Crowded, slowing
Retrieval & RAGVector search, embeddings, RAG pipelines and knowledge graphs.1,756751598Crowded, slowing
Files & DocumentsReading, converting and processing files and documents.1,613320958Established
Research & KnowledgeAcademic research, deep research and knowledge bases.1,279422598Lagging the catalogue
DatabasesQuerying and managing SQL, NoSQL and hosted databases.805355383Lagging the catalogue
Data Engineering & AnalyticsWarehouses, pipelines and hands-on data analysis.606325271Thin

Top 40 products by GitHub stars

Search the directory

Profiled products only (1,971 of 7,219), ranked by the stars of their own repository. Products that live in a subfolder of a larger repository (common for skills) show that whole repository's stars and are ranked after them; products without a repository come last. Stars are popularity, not quality or installs.

Data & Knowledge: top profiled products by GitHub stars, 2026-11-21
#ProductTypePublisherCapabilities GitHub stars Last push LicenceSave and compare
1FirecrawlAPI that crawls and scrapes websites into clean Markdown or structured JSON, with search, map and agentic extract endpoints.APIfirecrawl206ktodayAGPL-3.0
2MarkItDownPython utility that converts PDF, Office, HTML, images and audio files into Markdown for LLM consumption.Librarymicrosoft197k14d agoMIT
3DifyPlatform for building LLM apps and agent workflows visually with RAG pipelines, model management and app publishing.Automation platformLangGenius162ktodayLicenseRef-Dify-Open-Source-License
4LangflowVisual builder for agent and RAG flows using drag-and-drop components, with flows deployable as APIs or MCP servers.Automation platformDataStax157ktodayMIT
5graphifyTurn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.Agent projectGraphify-Labs151ktodayApache-2.0
6LangChainFramework of model, tool, retriever and agent abstractions with hundreds of provider integrations; its agent API now runs on LangGraph.FrameworkLangChain150ktodayMIT
7claude-memPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + MoreAgent skillthedotmack119ktodayApache-2.0
8RAGFlowSelf-hostable RAG engine with deep document parsing, template-based chunking, citations and an agent builder.Frameworkinfiniflow94k1d agoApache-2.0
9Understand-AnythingGraphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.Agent skillegonex.ai93k3d agoMIT
10MCP Reference ServersOfficial collection of reference MCP servers (filesystem, fetch, git, memory and others) demonstrating SDK usage.LibraryAgentic AI Foundation (Linux Foundation)92k2d ago—
11Crawl4AIOpen-source async web crawler that outputs LLM-ready Markdown and structured extraction, running headless browsers locally.Libraryunclecode88k1d agoApache-2.0
12MiroFishA Simple and Universal Swarm Intelligence Engine, Predicting Anything. 简洁通用的群体智能引擎,预测万物Infrastructure666ghj86k1mo agoAGPL-3.0
13deer-flowAn open-source long-horizon SuperAgent harness that researches, codes, and creates. With the help of sandboxes, memories, tools, skill, subagents and message gateway, it handles different levels of tasks that could take minutes to hours.Agent projectbytedance86ktodayMIT
14claude-flowAI orchestration with hive-mind swarms, neural networks, and 87 MCP tools for enterprise dev.MCP serverruvnet82ktodayMIT
15ai-agents-for-beginners18 Lessons to Get Started Building AI AgentsAgent projectmicrosoft81k4d agoMIT
16Prompt-Engineering-Guide🐙 Guides, papers, lessons, notebooks and resources for prompt engineering, context engineering, RAG, and AI Agents.Agent projectdair-ai80k8mo agoMIT
17DoclingDocument conversion toolkit that parses PDF, DOCX, PPTX, HTML and images with layout and table understanding into Markdown, JSON or DoclingDocument.LibraryIBM (LF AI & Data)71ktodayMIT
18diagram-designEditorial diagram design for Claude Code, Codex, GitHub Copilot, Cursor, Factory Droid, Qoder, and Pi. 45 diagram types. Self-contained HTML + SVG. No shadows. No Mermaid slop.Agent skillcathrynlavery70k2d agoMIT
19Mem0Memory layer that extracts, stores and retrieves long-term facts about users and sessions for agents, available as an open-source library or hosted platform.LibraryMem069k3d agoApache-2.0
20anything-llmStop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience Agent projectMintplex-Labs68ktodayMIT
21TrendRadar⭐AI-driven public opinion & trend monitor with multi-platform aggregation, RSS, web search, MCP & smart alerts.🎯 告别信息过载,你的 AI 舆情监控助手与热点筛选工具!聚合多平台热点 + RSS 订阅 + 定时联网搜索,支持关键词与 AI 智能筛选。AI 翻译 + AI 分析简报直推手机,支持接入 MCP 架构,赋能 AI 自然语言对话分析、实时检索、情感洞察与趋势预测。支持 Docker,数据本地/云端自持。集成微信/飞书/钉钉/Telegram/邮件/ntfy/Bark/Slack 等渠道智能推送。MCP serversansan064ktodayGPL-3.0
22DeepTutorDeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.Agent projectHKUDS61k1d agoApache-2.0
23meilisearchA lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.Infrastructuremeilisearch60k1d ago—
24ai-agent-book《深入理解 AI Agent:设计原理与工程实践》(李博杰 著)开源主仓库:全书正文、编译版 PDF 与按章配套代码Agent projectbojieli59k1mo agoApache-2.0
25servietteA no-code RAG server that keeps answers in sync with your documents. Works with 8 vector databases. Built on Pathway.Infrastructurepathwaycom59k11d agoMIT
26scientific-agent-skillsTurn any AI agent into an AI Scientist. The #1 Agent Skills library for science, used by 250,000+ scientists worldwide. 177 ready-to-use validated skills plus 100+ scientific databases covering biology, chemistry, medicine, and drug discovery. Compatible with Cursor, Claude Code, Codex, Pi, Antigravity, and the open Agent Skills standard.Agent skillK-Dense-AI58k5d agoMIT
27cherry-studioYour AI workspace for models, agents, and productivity.Agent skillCherryHQ54ktodayAGPL-3.0
28obsidian-skillsTeach your agent to use Obsidian and open formats including Markdown, Bases, JSON Canvas.Agent skillobsidianmd53k1d agoMIT
29LlamaIndexFramework for building agents and workflows over your data, with connectors, indexing, retrieval and an event-driven Workflows engine.FrameworkLlamaIndex53k1d agoMIT
30Codebase MemoryCodebase knowledge graph for AI agents — 162 languages, sub-ms queries, 99% fewer tokens.MCP serverDeusData49ktodayMIT
31MilvusDistributed vector database for billion-scale similarity search with many index types; Zilliz Cloud is the managed version.InfrastructureZilliz (LF AI & Data)47ktodayApache-2.0
32LangGraphLow-level orchestration framework that models agents as stateful graphs with checkpointing, human-in-the-loop interrupts and durable execution.FrameworkLangChain45ktodayMIT
33mcp-serverPuter MCP enables AI tools to interact with Puter: manage files, websites, workers, and moreMCP serverputer.com44ktodayAGPL-3.0
34openhumanThe fastest, cheapest, most efficient open-source agent harness. Run more than 500 agents on a $10 VPS.Agent projecttinyhumansai43k1d agoGPL-3.0
35BettaFish微舆:人人可用的多Agent舆情分析助手,打破信息茧房,还原舆情原貌,预测未来走向,辅助决策!从0实现,不依赖任何框架。Framework666ghj43k1mo agoGPL-2.0
36OpenVikingSelf-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.Infrastructurevolcengine43ktodayAGPL-3.0
37FaissC++/Python library for efficient similarity search and clustering of dense vectors, including GPU index implementations.LibraryMeta42k1d agoMIT
38MarkerConverts PDFs, images and Office documents to Markdown, JSON or HTML using deep-learning OCR and layout models, with optional LLM refinement.LibraryDatalab41k1d agoApache-2.0
39LightRAGGraph-enhanced RAG system that indexes entities and relations alongside vectors for dual-level retrieval, with a server and web UI.LibraryHKU Data Intelligence Lab41ktodayMIT
40tidbTiDB is built for agentic workloads that grow unpredictably, with ACID guarantees and native support for transactions, analytics, and vector search. No data silos. No noisy neighbors. No infrastructure ceiling.Infrastructurepingcap41ktodayApache-2.0

Most common integrations

Third-party services named by products in this node, whole catalogue.

IntegrationProducts Share
PostgreSQL2864.0%
MySQL1121.6%
Google Maps1091.5%
Obsidian1091.5%
GitHub600.8%
Redis600.8%
Microsoft Excel590.8%
YouTube500.7%
Supabase470.7%
Google Sheets460.6%
Google Drive340.5%
MongoDB290.4%
Apache Kafka270.4%
BigQuery270.4%
Snowflake260.4%

Launches per month

New Data & Knowledge products per monthSRC

Whole catalogue, by publication month, 2024-11 to 2026-11. The last month is partial; undated products are excluded.

New Data & Knowledge products per monthWhole catalogue, by publication month, 2024-11 to 2026-11. The last month is partial; undated products are excluded.. Products published: 25 points, latest 1,074 products in 2026-11.
Show the data as a table
New Data & Knowledge products per month. Whole catalogue, by publication month, 2024-11 to 2026-11. The last month is partial; undated products are excluded.
Products published (products)
2024-116
2024-1210
2025-013
2025-027
2025-0318
2025-0419
2025-0514
2025-0616
2025-079
2025-0811
2025-0988
2025-1068
2025-1140
2025-1263
2026-0171
2026-02172
2026-03245
2026-04357
2026-05332
2026-06641
2026-07609
2026-08603
2026-09896
2026-101,418
2026-111,074

Related opportunities

Opportunity radar

Scored gaps for this node or anything beneath it. Scores rank hypotheses; they do not predict demand.

OpportunityCompetitionConfidenceScore
Coverage gap: Chat with DocumentsChat with DocumentsLowLow64.0
Coverage gap: NotebooksNotebooksHighLow61.7
Coverage gap: Entity ExtractionEntity ExtractionMediumLow60.2
Coverage gap: RerankingRerankingHighLow60.0
Coverage gap: Schema InspectionSchema InspectionHighLow58.6
Coverage gap: Database MigrationsDatabase MigrationsHighLow58.2
Coverage gap: Multimodal EmbeddingsMultimodal EmbeddingsLowLow58.0
Coverage gap: Query OptimizationQuery OptimizationHighLow57.6
Coverage gap: Text EmbeddingsText EmbeddingsMediumLow57.4
Coverage gap: SQL QueryingSQL QueryingHighLow57.0
Coverage gap: Data VisualizationData VisualizationHighLow56.7
Coverage gap: Document IngestionDocument IngestionHighLow55.6
Coverage gap: Local File AccessLocal File AccessHighLow55.2
Coverage gap: Team WikisTeam WikisHighLow54.3
Coverage gap: Fact-checkingFact-checkingHighLow54.2

Limits of these numbers

  • Products are placed here by whole-word keyword rules on their name, description and repository topics, and inherit every ancestor of a matched node. Products that do this without naming it are missed; ambiguous words cause false matches. Methodology
  • About half of the catalogue matches no node at all, so absolute counts understate every category.
  • Launch counts use the publication date each source reports; for auto-discovered repositories that is the repository’s creation date.
  • Stars and concentration cover only products with their own repository (not a subfolder of a larger one). They measure attention, not adoption.

Page data last updated .