AgentSidekick

Retrieval & RAG

Vector search, embeddings, RAG pipelines and knowledge graphs. As of 2026-11-21, 1,756 indexed products match this category (751 with full profiles); 598 were published in the last 90 days, against 420 in the 90 days before, slower than the catalogue as a whole. By GitHub stars, Firecrawl leads, holding 5% of the stars of the 1,485 products here with their own repository. The most common form is MCP servers (81% of indexed products).

CategorySegment label: Crowded, slowing DER

Segment statistics

Whole catalogue (all 1,756 indexed products, not only profiled ones). Calculated 2026-11-21.

1,756DERIndexed products751 with full profiles
598DERLaunched, last 90 days420 in the 90 days before
17DERMomentum vs catalogue50 = same pace as the catalogue
5%DERLargest project's share of starsAcross 1,485 products with their own repository
0.017DERConcentration (HHI)About 58 equal-sized projects’ worth; unconcentrated
7SRCMedian starsPer product with its own repository
77%DEROpen sourceMarked open source; unknown counts as not

Crowded, slowing. Among the largest fifth of nodes at its level, but its launches grew more slowly than the catalogue’s. Size is ranked against the other category nodes; momentum compares this node’s launches in the last 90 days with the 90 before, relative to the whole catalogue’s ratio (score 62 or more = faster, 38 or less = slower; below 5 dated launches it is not measured and the label uses size alone).

Concentration. The Herfindahl–Hirschman index (HHI) adds up each project’s squared share of stars. At 0.017, attention is spread as if across about 58 equally popular projects. Using the common competition-policy thresholds (0.15 and 0.25), that is unconcentrated. Stars measure attention, not usage or quality.

Which product forms serve it

The same need is met by MCP servers, agent skills, plugins, extensions and frameworks. This counts each form across the whole catalogue.

Products in Retrieval & RAG by product type, as of 2026-11-21
Product type IndexedShareWith a profile
MCP servers1,41980.8%427
Infrastructure1347.6%133
Agent skills512.9%49
Frameworks422.4%41
Agent projects372.1%37
Plugins231.3%18
Libraries231.3%22
AI APIs70.40%7
Rules packs60.34%3
SaaS products50.28%5
IDE extensions40.23%4
Automation platforms30.17%3
Developer tools20.11%2

4 subcategories

Indexed = whole catalogue; profiled = products with their own page. A product can sit in several, so the rows do not add up to the total.

SubcategoryIndexed ProfiledNew, 90 daysLabel
Retrieval-Augmented GenerationGrounding model answers in retrieved documents.1,193532394Crowded, slowing
Knowledge GraphsGraphs of entities and relationships for reasoning and retrieval.506221162Lagging the catalogue
Vector DatabasesStoring and searching embeddings.470271128Lagging the catalogue
EmbeddingsTurning text and media into vectors.2119869Thin

Top 40 products by GitHub stars

Search the directory

Profiled products only (751 of 1,756), ranked by the stars of their own repository. Products that live in a subfolder of a larger repository (common for skills) show that whole repository's stars and are ranked after them; products without a repository come last. Stars are popularity, not quality or installs.

Retrieval & RAG: top profiled products by GitHub stars, 2026-11-21
#ProductTypePublisherCapabilities GitHub stars Last push LicenceSave and compare
1FirecrawlAPI that crawls and scrapes websites into clean Markdown or structured JSON, with search, map and agentic extract endpoints.APIfirecrawl206ktodayAGPL-3.0
2MarkItDownPython utility that converts PDF, Office, HTML, images and audio files into Markdown for LLM consumption.Librarymicrosoft197k14d agoMIT
3DifyPlatform for building LLM apps and agent workflows visually with RAG pipelines, model management and app publishing.Automation platformLangGenius162ktodayLicenseRef-Dify-Open-Source-License
4LangflowVisual builder for agent and RAG flows using drag-and-drop components, with flows deployable as APIs or MCP servers.Automation platformDataStax157ktodayMIT
5graphifyTurn any codebase, with its docs, SQL schemas, configs, and PDFs, into a queryable knowledge graph. A /graphify skill for Claude Code, Cursor, Codex, and Gemini CLI: local deterministic AST parsing, every edge explained, no vector store.Agent projectGraphify-Labs151ktodayApache-2.0
6LangChainFramework of model, tool, retriever and agent abstractions with hundreds of provider integrations; its agent API now runs on LangGraph.FrameworkLangChain150ktodayMIT
7claude-memPersistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + MoreAgent skillthedotmack119ktodayApache-2.0
8RAGFlowSelf-hostable RAG engine with deep document parsing, template-based chunking, citations and an agent builder.Frameworkinfiniflow94k1d agoApache-2.0
9Understand-AnythingGraphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.Agent skillegonex.ai93k3d agoMIT
10Crawl4AIOpen-source async web crawler that outputs LLM-ready Markdown and structured extraction, running headless browsers locally.Libraryunclecode88k1d agoApache-2.0
11MiroFishA Simple and Universal Swarm Intelligence Engine, Predicting Anything. 简洁通用的群体智能引擎,预测万物Infrastructure666ghj86k1mo agoAGPL-3.0
12claude-flowAI orchestration with hive-mind swarms, neural networks, and 87 MCP tools for enterprise dev.MCP serverruvnet82ktodayMIT
13ai-agents-for-beginners18 Lessons to Get Started Building AI AgentsAgent projectmicrosoft81k4d agoMIT
14Prompt-Engineering-Guide🐙 Guides, papers, lessons, notebooks and resources for prompt engineering, context engineering, RAG, and AI Agents.Agent projectdair-ai80k8mo agoMIT
15DoclingDocument conversion toolkit that parses PDF, DOCX, PPTX, HTML and images with layout and table understanding into Markdown, JSON or DoclingDocument.LibraryIBM (LF AI & Data)71ktodayMIT
16Mem0Memory layer that extracts, stores and retrieves long-term facts about users and sessions for agents, available as an open-source library or hosted platform.LibraryMem069k3d agoApache-2.0
17anything-llmStop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience Agent projectMintplex-Labs68ktodayMIT
18DeepTutorDeepTutor: Lifelong Personalized Tutoring. https://deeptutor.info/.Agent projectHKUDS61k1d agoApache-2.0
19meilisearchA lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.Infrastructuremeilisearch60k1d ago—
20ai-agent-book《深入理解 AI Agent:设计原理与工程实践》(李博杰 著)开源主仓库:全书正文、编译版 PDF 与按章配套代码Agent projectbojieli59k1mo agoApache-2.0
21servietteA no-code RAG server that keeps answers in sync with your documents. Works with 8 vector databases. Built on Pathway.Infrastructurepathwaycom59k11d agoMIT
22cherry-studioYour AI workspace for models, agents, and productivity.Agent skillCherryHQ54ktodayAGPL-3.0
23LlamaIndexFramework for building agents and workflows over your data, with connectors, indexing, retrieval and an event-driven Workflows engine.FrameworkLlamaIndex53k1d agoMIT
24Codebase MemoryCodebase knowledge graph for AI agents — 162 languages, sub-ms queries, 99% fewer tokens.MCP serverDeusData49ktodayMIT
25MilvusDistributed vector database for billion-scale similarity search with many index types; Zilliz Cloud is the managed version.InfrastructureZilliz (LF AI & Data)47ktodayApache-2.0
26LangGraphLow-level orchestration framework that models agents as stateful graphs with checkpointing, human-in-the-loop interrupts and durable execution.FrameworkLangChain45ktodayMIT
27OpenVikingSelf-evolving Context Database for AI Agents. Unify Agent Memory, Knowledge RAG and Skills.Infrastructurevolcengine43ktodayAGPL-3.0
28FaissC++/Python library for efficient similarity search and clustering of dense vectors, including GPU index implementations.LibraryMeta42k1d agoMIT
29MarkerConverts PDFs, images and Office documents to Markdown, JSON or HTML using deep-learning OCR and layout models, with optional LLM refinement.LibraryDatalab41k1d agoApache-2.0
30LightRAGGraph-enhanced RAG system that indexes entities and relations alongside vectors for dual-level retrieval, with a server and web UI.LibraryHKU Data Intelligence Lab41ktodayMIT
31tidbTiDB is built for agentic workloads that grow unpredictably, with ACID guarantees and native support for transactions, analytics, and vector search. No data silos. No noisy neighbors. No infrastructure ceiling.Infrastructurepingcap41ktodayApache-2.0
32PageIndex📑 PageIndex: Document Index for Vectorless, Reasoning-based RAGAgent projectVectifyAI41ktodayMIT
33book-to-skillTurn any technical book PDF into a Claude Code skill — ready to study, reference, and use while you work.Agent skillvirgiliojr9439k1d agoMIT
34VaneVane is an AI-powered answering engine.Agent projectItzCrazyKns38k2mo agoMIT
35GraphRAGPipeline that extracts an entity knowledge graph and community summaries from text, enabling global and local graph-based retrieval.Librarymicrosoft37k2d agoMIT
36QdrantVector search engine written in Rust with payload filtering, quantisation and hybrid sparse/dense search; self-hosted or Qdrant Cloud.Infrastructureqdrant36ktodayApache-2.0
37CogneeMemory engine that turns documents and conversations into a knowledge graph plus vector index, exposed through add/cognify/search operations.Librarytopoteretes33k1d agoApache-2.0
38GraphitiPython library for building temporally aware knowledge graphs from episodes of data, with hybrid semantic, keyword and graph search for agent memory.LibraryZep AI33k1d agoApache-2.0
39RAG_TechniquesThis repository showcases various advanced techniques for Retrieval-Augmented Generation (RAG) systems. Each technique has a detailed notebook tutorial.InfrastructureNirDiamant30k19d ago—
40SimOpen-source visual canvas for building and deploying agent workflows, self-hosted or cloud.Automation platformSim30ktodayApache-2.0

Most common integrations

Third-party services named by products in this node, whole catalogue.

IntegrationProducts Share
PostgreSQL412.3%
Obsidian341.9%
YouTube191.1%
Qdrant171.0%
GitHub150.9%
Neo4j150.9%
Gmail70.4%
Microsoft Excel70.4%
Azure60.3%
Redis60.3%
Chroma50.3%
Discord50.3%
GraphQL50.3%
MySQL50.3%
Pinecone50.3%

Launches per month

New Retrieval & RAG products per monthSRC

Whole catalogue, by publication month, 2024-11 to 2026-11. The last month is partial; undated products are excluded.

New Retrieval & RAG products per monthWhole catalogue, by publication month, 2024-11 to 2026-11. The last month is partial; undated products are excluded.. Products published: 25 points, latest 188 products in 2026-11.
Show the data as a table
New Retrieval & RAG products per month. Whole catalogue, by publication month, 2024-11 to 2026-11. The last month is partial; undated products are excluded.
Products published (products)
2024-114
2024-126
2025-013
2025-024
2025-037
2025-049
2025-058
2025-0612
2025-077
2025-086
2025-0927
2025-1029
2025-1118
2025-1223
2026-0129
2026-0262
2026-0380
2026-04117
2026-05108
2026-06122
2026-07179
2026-08146
2026-09143
2026-10222
2026-11188

Related opportunities

Opportunity radar

Scored gaps for this node or anything beneath it. Scores rank hypotheses; they do not predict demand.

OpportunityCompetitionConfidenceScore
Coverage gap: Chat with DocumentsChat with DocumentsLowLow64.0
Coverage gap: Entity ExtractionEntity ExtractionMediumLow60.2
Coverage gap: RerankingRerankingHighLow60.0
Coverage gap: Multimodal EmbeddingsMultimodal EmbeddingsLowLow58.0
Coverage gap: Text EmbeddingsText EmbeddingsMediumLow57.4
Coverage gap: Document IngestionDocument IngestionHighLow55.6
Coverage gap: Graph RAGGraph RAGHighLow53.6

Limits of these numbers

  • Products are placed here by whole-word keyword rules on their name, description and repository topics, and inherit every ancestor of a matched node. Products that do this without naming it are missed; ambiguous words cause false matches. Methodology
  • About half of the catalogue matches no node at all, so absolute counts understate every category.
  • Launch counts use the publication date each source reports; for auto-discovered repositories that is the repository’s creation date.
  • Stars and concentration cover only products with their own repository (not a subfolder of a larger one). They measure attention, not adoption.

Page data last updated .