Local Inference
Running models on your own machine. As of 2026-11-21, 256 indexed products offer it (150 with full profiles), in 10 forms: 188 MCP servers, 16 IDE extensions, 15 Infrastructure, 11 Frameworks. 89 were published in the last 90 days, against 54 in the 90 days before. cherry-studio has the most GitHub stars, 16% of the total across 239 products with their own repository.
CapabilitySegment label: Crowded, slowing DER
Available as
All in the directoryProfiled products grouped by product form. Within each form: own-repository products by GitHub stars, then products in a subfolder of a larger repository (they show that repository's stars), up to 10 each.
MCP servers 188 indexed · 84 profiled
Protocol-level reach: 36 of the 45 active platforms AgentSidekick tracks document support for MCP. DER Which platforms
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | MaxKB🔥 MaxKB is an open-source platform for building enterprise-grade agents. 强大易用的开源企业级智能体平台。 | 1Panel-dev | 23k | today | ||
| 2 | docker-androidAndroid in docker solution with noVNC supported, video recording, mcp server and AI-agent | budtmo | 16k | 2d ago | ||
| 3 | osaurusOwn your AI. The open-source agent harness for macOS. Run local or cloud models with persistent memory, skills, tools, and autonomous execution. Built in Swift. | osaurus-ai | 8.2k | today | ||
| 4 | bla🛡️ BLA (bla-agents / Blind LLM Agents) — Open-source Blue Defense MCP server & CLI protecting JS/TS, Electron ASAR, WASM, Python, Native Binaries, SQLite, GGUF/Safetensors & Android DEX against AI reverse-engineering agents (REA). | qybaihe | 6.4k | today | ||
| 5 | deep-researchUse any LLMs (Large Language Models) for Deep Research. Support SSE API and MCP server. | u14app | 4.7k | 5mo ago | ||
| 6 | pipeshub-aiThe open-source context layer for AI agents and enterprise AI search. Permission-aware search and cited answers across 30+ sources, including Slack, Drive, Jira, GitHub and Microsoft 365, for people and agents through MCP, SDKs and built-in agents. Self-hosted or Cloud. | pipeshub-ai | 4.1k | today | ||
| 7 | chat-ollamaChatOllama is an open source agentic app for running AI agents across local and hosted models. | sugarforever | 3.5k | 3d ago | ||
| 8 | agency-orchestrator🚀 One sentence → your one-person company of AI experts → complete deliverable in minutes. 276 CN + 184 EN + 5 more languages (ko/ru/pt-BR/id/ar) · zero-code YAML · auto-verified acceptance · Web Studio / Desktop / Docker · 15 LLM providers (11 key-free). 一句话组建你的「一人公司」AI 专家团队,几分钟交付完整方案;验收自动核验,网页 / 桌面 / Docker 全渠道。 | jnMetaCode | 2.5k | 10d ago | ||
| 9 | blender-ai-mcp-addonAI assistant and MCP server for Blender: visual chat, scene tools, multi-agent workflows. Works with ChatGPT/GPT, Claude, Gemini, Ollama, OpenCode. Blender AI addon, copilot for 3D, bpy Python scripting, MCP integration. | bookwjitchieni1986 | 1.9k | 2d ago | ||
| 10 | DiveDive is an open-source MCP Host Desktop Application that seamlessly integrates with any LLMs supporting function calling capabilities. ✨ | OpenAgentPlatform | 1.8k | 3mo ago |
IDE extensions 16 indexed · 17 profiled
No compatibility evidence is collected for this form.
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | twinny - AI Code Completion and ChatAI code completion, chat, inline edit and code review on models you run. Private, open source, one gateway for the whole team. | twinnydotdev | 3.7k | 3d ago | ||
| 2 | llama-vscodeLocal LLM-assisted text completion using llama.cpp | ggml-org | 1.6k | 1d ago | ||
| 3 | ChatGPT CopilotAn VS Code ChatGPT Copilot Extension | feiskyer | 164 | 5mo ago | ||
| 4 | Dish AI Commit Message Gen🤖 AI-Powered VSCode extension for generating standardized Git/SVN commit messages. ✨ Supports 20+ AI services: OpenAI, Ollama, VS Code AI, Zhipu AI, DashScope, Doubao AI, Gemini, Deepseek, Anthropic, Mistral, Azure OpenAI, VertexAI, LMStudio and more. 🌍 19 languages supported. 📊 Features: AI commit generation, code review, weekly reports, branch naming, PR summaries, token statistics. | littleCareless | 163 | 1mo ago | ||
| 5 | CodeBuddy: Deep Coding Agent — Bring Your Own ModelA LangGraph/deepagents coding agent that lives in VS Code — it plans, reads your codebase, edits (AST-aware), runs commands, searches, and reviews. Bring your own model and keys: Anthropic, Gemini, OpenAI, Groq, DeepSeek, Qwen, GLM, xAI Grok, or a local model. MCP-native, security-first, MIT-licensed. | fiatinnovations | 142 | 2mo ago | ||
| 6 | GitHub Copilot API GatewayFree & open-source universal API Gateway for all VS Code language models. Auto-discovers GitHub Copilot, Gemini, Ollama & any AI extension — exposing them via local OpenAI, Anthropic, Google & Llama compatible APIs. One endpoint, every model. Use with Cursor, LangChain, Agents & more. | Suhaib Bin Younis | 134 | 4d ago | ||
| 7 | Commit SageFree AI commit messages using Gemini, OpenRouter, Groq, Claude, DeepSeek, xAI, Ollama, or any OpenAI-compatible LLM | VizzleTF | 53 | 4d ago | ||
| 8 | Bandit StealthFree, local-first coding agent for VS Code — run Ollama models (Gemma, Qwen, Devstral) with tool use, diff approvals, and voice. Optional Bandit cloud. | Burtson Labs | 9 | 3d ago | ||
| 9 | GitMind: AI Commit Message GeneratorAI commit message generator for VS Code. Write clear Conventional Commits from your diff with GitHub Copilot, local Ollama, or 24 built-in AI providers. | ShahabBahreiniJangjoo | 5 | 2d ago | ||
| 10 | Lattice AILattice AI extension for the local-first Digital Brain that keeps your knowledge durable across any AI model. | parktaesoo | 2 | 1d ago |
Infrastructure 15 indexed · 13 profiled
No compatibility evidence is collected for this form.
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | LEANN[MLsys2026 Best Paper]: https://arxiv.org/abs/2506.08276. RAG on Everything with LEANN. Enjoy 97% storage savings while running a fast, accurate, and 100% private RAG application on your personal device. | StarTrail-org | 14k | today | ||
| 2 | langchain4jLangChain4j is an idiomatic, open-source Java library for building LLM-powered applications on the JVM. It offers a unified API over popular LLM providers and vector stores, and makes implementing tool calling (including MCP support), agents and RAG easy. It integrates seamlessly with enterprise Java frameworks like Quarkus and Spring Boot. | langchain4j | 13k | today | ||
| 3 | SoupFine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. | MakazhanAlpamys | 11k | today | ||
| 4 | cube-studiocube studio开源云原生一站式机器学习/深度学习/大模型AI平台,mlops算法链路全流程,算力租赁平台,notebook在线开发,拖拉拽任务流pipeline编排,多机多卡分布式训练,超参搜索,推理服务VGPU虚拟化,边缘计算,标注平台自动化标注,deepseek等大模型sft微调/奖励模型/强化学习训练,vllm/ollama/mindie大模型多机推理,私有知识库,AI模型市场,支持国产cpu/gpu/npu 昇腾生态,支持RDMA,支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano等分布式 | tencentmusic | 5.1k | 1mo ago | ||
| 5 | cube-studiocubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台,算法全链路流程,多租户,算力租赁平台,token中转,拖拉拽任务流pipeline编排,多机多卡分布式训练,超参搜索,推理服务,VGPU虚拟化,云边端协同,边缘计算,自动化标注平台,deepseek等大模型sft微调/奖励模型/强化学习训练,vllm/ollama/mindie大模型多机推理,私有知识库llmops智能体,AI模型市场,支持国产异构算力调度,昇腾/寒武纪/海光/摩尔/沐曦等,支持ib/roce/RDMA,信创支持 | data-infra | 2.6k | 1mo ago | ||
| 6 | RAGLightRAGLight is a modular framework for Retrieval-Augmented Generation (RAG). It makes it easy to plug in different LLMs, embeddings, and vector stores, and now includes seamless MCP integration to connect external tools and data sources. | Bessouat40 | 672 | 1mo ago | ||
| 7 | LynkrStreamline your workflow with Lynkr, a CLI tool that acts as an HTTP proxy for efficient code interactions using Claude Code CLI. | Fast-Editor | 642 | 6d ago | ||
| 8 | embedJsA NodeJS RAG framework to easily work with LLMs and embeddings | llm-tools | 601 | 4mo ago | ||
| 9 | juliusSimple LLM service identification - translate IP:Port to Ollama, vLLM, LiteLLM, or 60+ other AI services in seconds | praetorian-inc | 256 | today | ||
| 10 | heronAgent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude, Codex, DeepAgents and more — deployed on the provider side, no SDK changes required. | Netis | 104 | 1mo ago |
Frameworks 11 indexed · 10 profiled
No compatibility evidence is collected for this form.
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | open-multi-agentSelf-hosted TypeScript agent runtime with durable approvals and verifiable run records. Own it, approve it, audit it. | open-multi-agent | 7.1k | 4d ago | ||
| 2 | koogKoog is a JVM (Java and Kotlin) framework for building predictable, fault-tolerant and enterprise-ready AI agents across all platforms – from backend services to Android and iOS, JVM, and even in-browser environments. Koog is based on our AI products expertise and provides proven solutions for complex LLM and AI problems | JetBrains | 4.7k | 11d ago | ||
| 3 | ObserverThe micro-agent orchestrator that monitors anything for you. Running completely locally. Open Source. | Roy3838 | 1.8k | today | ||
| 4 | Agentlas-OSThe Open-Source AI Agent OS inspired by OpenAI Dots, Meta Muse, OpenClaw, Hermes & Mem0. Autonomous 3-tier headhunting, Receipt & Verifier loops, scoped memory, and BYOM for Claude Code, Codex & Cursor. | agentlas-ai | 1.4k | today | ||
| 5 | PhoneClawPhoneClaw turns phones into local AI agent runtimes with on-device models, native mobile Skills, LiveLand, and optional Mac Gateway inference. | kellyvv | 1.3k | 3mo ago | ||
| 6 | samplesAgent samples built using the Strands Agents SDK. | strands-agents | 943 | 3d ago | ||
| 7 | pocketpawYour AI agent in 30 seconds. Not 30 hours. Self-hosted, open-source personal AI with desktop installer, multi-agent Command Center(Deep Work), and 7-layer security. Anthropic, OpenAI, or Ollama. | pocketpaw | 935 | 1d ago | ||
| 8 | Deuz-SDKTypeScript AI SDK and agent framework with zero dependencies. Build production AI agents and multi-agent swarms: durable execution, long-term memory, hybrid RAG, MCP client/server, A2A, human-in-the-loop approvals, CodeAct sandboxes, realtime voice. One streaming API, 29 providers: Claude, GPT, Gemini, Grok, DeepSeek, Ollama. Node, Bun, Deno, edge. | Deuz-AI | 508 | 1mo ago | ||
| 9 | Abu-CoworkOpen-source AI office assistant for your desktop: describe the result or file you want in one sentence, and Abu plans and does the rest. Local-first, with Claude, OpenAI, DeepSeek and Ollama. | PM-Shawn | 424 | 8d ago | ||
| 10 | animaworksOrganization-as-Code for autonomous AI agents. Brain-inspired memory that grows, consolidates, and forgets. Multi-model (Claude/Codex/Gemini/Cursor/Ollama). | xuiltul | 268 | 5d ago |
Agent projects 8 indexed · 8 profiled
No compatibility evidence is collected for this form.
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | CodewhaleAn open-source coding agent for your terminal, written in Rust. It works with any model, can split one task across several agents, and every line of it is yours to read. | codewhale-hq | 41k | today | ||
| 2 | BrowserOS🌐 The open-source Agentic browser; alternative to ChatGPT Atlas, Perplexity Comet, Dia. | browseros-ai | 14k | today | ||
| 3 | aichatAll-in-one LLM CLI tool featuring Shell Assistant, Chat-REPL, RAG, AI Tools & Agents, with access to OpenAI, Claude, Gemini, Ollama, Groq, and more. | sigoden | 11k | 9mo ago | ||
| 4 | big-AGIThe AI workspace for experts. Switch across 800+ models mid-chat, tune 40+ parameters, Beam models side by side. BYO keys, self-hostable. Built on our native-protocol engine: reliable, hyper-configurable, and fast. | enricoros | 7.2k | today | ||
| 5 | LongMemoryLocal persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc. | CaviraOSS | 4.6k | 11d ago | ||
| 6 | agentic-rag-for-dummiesA modular Agentic RAG built with LangGraph — learn Retrieval-Augmented Generation Agents in minutes. | GiovanniPasq | 4.4k | 10d ago | ||
| 7 | little-coderA harness optimized to smaller LLMs | itayinbarr | 2.7k | 2mo ago | ||
| 8 | zeroThe coding agent that answers to you, your model, your machine, your rules. | Twigpine | 1.8k | 1d ago |
Agent skills 8 indexed · 8 profiled
Protocol-level reach: 26 of the 45 active platforms AgentSidekick tracks document support for the Agent Skills format. DER Which platforms
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | cherry-studioYour AI workspace for models, agents, and productivity. | CherryHQ | 54k | today | ||
| 2 | pinvou-agentOpen-source desktop AI agent for tools, files, knowledge, workflows, and real deliverables. | Pinvou | 3.0k | today | ||
| 3 | claude-apiReference for the Claude API / Anthropic SDK — model ids, pricing, params, streaming, tool use, MCP, agents, caching, token counting, model migration. TRIGGER — read BEFORE opening the target file; don't skip because it "looks like a one-liner" — whenever: the prompt names Claude/Anthropic in any form (Claude, Anthropic, Fable, Opus, Sonnet, Haiku, `anthropic`, `@anthropic-ai`, `claude-*`, `us.anthropic.*`, `[1m]`); the user asks about an LLM (pricing/model choice/limits/caching) — never answer from memory; OR the task is LLM-shaped with provider unstated (agent/MCP/tool-definition/multi-agent/RAG/LLM-judge/computer-use; generate/summarize/extract/classify/rewrite/converse over NL; debugging refusals/cutoffs/streaming/tool-calls/tokens). SKIP only when another provider is being worked on (overrides all triggers): OpenAI/GPT/Gemini/Llama/Mistral/Cohere/Ollama named in the query; OR `grep -rE 'openai|langchain_openai|google.generativeai|genai|mistralai|cohere|ollama'` over the project hits (run this grep FIRST if no provider named — don't Read the file). | anthropics | — | 5d ago | ||
| 4 | add-ollama-toolAdd Ollama MCP server so the container agent can call local models and optionally manage the Ollama model library. | nanocoai | — | today | ||
| 5 | add-ollama-providerRoute a NanoClaw agent group to a local Ollama model instead of the Anthropic API. Ollama speaks the Anthropic API natively (v1/messages), so no provider code changes are needed — just env var overrides and a model setting. Use when the user wants to run their agent locally, cut API costs, or experiment with open-weight models. See docs/ollama.md for background. | nanocoai | — | today | ||
| 6 | gguf-quantizationGGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements. | Orchestra-Research | — | 1mo ago | ||
| 7 | llama-cppRuns LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU. | Orchestra-Research | — | 1mo ago | ||
| 8 | inference-serving-llama-cppRuns LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU. | davila7 | — | today |
Plugins 5 indexed · 5 profiled
5 of these are Claude plugins, which Claude Code documents (protocol-level). DER
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | claude-octopusRun multiple AI models against the same research, design, or coding task. Surface disagreements before you ship. | nyldn | 4.5k | 2d ago | ||
| 2 | octov11.26.0 - Custom provider presets in the tool helper, native skills and debate advisor checks. 31 personas, 56 commands, 64 skills. Run /octo:setup. | nyldn | 4.5k | 2d ago | ||
| 3 | claude-councilClaude Code plugin that asks several AI coding agents the same question and shows their answers side by side. Gemini, OpenAI, Grok, Perplexity, Kimi and any model OpenRouter routes to over API; codex, antigravity, grok, kimi and cursor CLIs on subscription auth; ollama locally. Codex specialists take coding tasks on their own branch (experimental). | hex | 951 | 4d ago | ||
| 4 | lumenPrecise local semantic code search via MCP. Indexes your codebase with Go AST parsing, embeds with Ollama or LM Studio, and exposes vector search to Claude through an MCP server — no cloud, no npm. | ory | 317 | 1mo ago | ||
| 5 | forkmindLocal-first LLM branching & debugging. Treat AI context windows like a Git repo — capture, branch, and regression-test LLM calls as a DAG. Free & local via Ollama, any OpenAI-compatible API, and MCP for agents. | ccplugins | — | 3mo ago |
Libraries 3 indexed · 3 profiled
No compatibility evidence is collected for this form.
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | a2a-adapterOpen Source A2A Protocol Adapter SDK for Different Agent Frameworks and Harnesses | hybroai | 108 | today | ||
| 2 | ai-mocks🤖 AI-Mocks is a Kotlin-based toolkit for mocking and testing LLM APIs with real streaming and SSE support. Simulate OpenAI, Anthropic, Gemini, Ollama, and A2A services locally for reliable integration testing. | mokksy | 53 | 1d ago | ||
| 3 | HandlerA2A protocol client for your terminal | alDuncanson | 47 | 6d ago |
Developer tools 1 indexed · 1 profiled
No compatibility evidence is collected for this form.
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | AgentOpsSDK and dashboard for monitoring agent sessions, tool calls, costs and failures, with integrations for CrewAI, OpenAI Agents SDK, AG2 and others. | AgentOps | 6.0k | 4mo ago |
Rules packs 1 indexed · 1 profiled
No compatibility evidence is collected for this form.
| # | Product | Publisher | Capabilities | GitHub stars | Last push | Save and compare |
|---|---|---|---|---|---|---|
| 1 | vybe-intelligence-vaultAn auto-updating open-source vault for AI agents, RAG systems, MCP servers, prompts, tools, templates, and next-generation web development. | rxmna8502 | 23 | today |
Which platforms these forms can reach
All platformsProtocol-level: the platform documents support for the standard. No specific pairing has been tested.
MCP servers 188 indexed here
36 platforms document MCP support CUR.
ContinueOpenCodeClaude CodeCodexGemini CLIZedOpenHandsClineWarpgooseQwen CodeBolt (bolt.new)KiroAmazon Bedrock AgentCoreAmpApps in ChatGPT / Apps SDK (now ChatGPT Plugins)Augment CodeChatGPTClaude (claude.ai apps)Claude DesktopCursorDevinDevin Desktop (formerly Windsurf)Factory (Droid)Gemini (app)Gemini Enterprise Agent Platform (formerly Vertex AI / Vertex AI Agent Builder)GitHub CopilotGitHub Copilot cloud agent (formerly Copilot coding agent)Google AntigravityJetBrains AI AssistantJunieLovableMicrosoft Copilot StudioReplit AgentTabninev0
Agent skills 8 indexed here
26 platforms document Agent Skills support CUR.
OpenClawOpenCodeClaude CodeCodexGemini CLIOpenHandsWarpgooseBolt (bolt.new)KiroAmpApps in ChatGPT / Apps SDK (now ChatGPT Plugins)ChatGPTClaude (claude.ai apps)Claude DesktopCursorDevinDevin Desktop (formerly Windsurf)Factory (Droid)Gemini (app)GitHub CopilotGitHub Copilot cloud agent (formerly Copilot coding agent)Google AntigravityJunieReplit AgentTabnine
Platforms publishers name
| Platform | Profiled products naming it |
|---|---|
| Codex | 13 |
| Claude Code | 10 |
| Cursor | 7 |
| Google Antigravity | 6 |
| ChatGPT | 5 |
| OpenCode | 4 |
| GitHub Copilot | 3 |
| OpenClaw | 2 |
| GitHub Copilot Extensions | 1 |
| Claude Desktop | 1 |
Integrations most often paired
Third-party services named by products with this capability, whole catalogue.
| Integration | Products | Share |
|---|---|---|
| GitHub | 5 | 2.0% |
| Android | 4 | 1.6% |
| Blender | 4 | 1.6% |
| Obsidian | 4 | 1.6% |
| Azure | 2 | 0.8% |
| PostgreSQL | 2 | 0.8% |
| Confluence | 1 | 0.4% |
| Gmail | 1 | 0.4% |
| Jira | 1 | 0.4% |
| Microsoft Excel | 1 | 0.4% |
| OpenAPI | 1 | 0.4% |
| Slack | 1 | 0.4% |
Keywords that trigger this capability
Matched as whole words, ignoring case, in a product's name, description and repository topics.
| Keyword | Products matched |
|---|---|
| ollama | 220 |
| llama.cpp | 2 |
| lm studio | 9 |
| local inference | 0 |
| run models locally | 0 |
| gguf | 17 |
| mlx | 8 |
Segment statistics
Whole catalogue (256 indexed products). Calculated 2026-11-21.
Crowded, slowing. Among the largest fifth of capabilities, but its launches grew more slowly than the catalogue’s. Size is ranked against all capabilities; momentum compares launches in the last 90 days with the 90 before, relative to the catalogue’s own ratio (62 or more = faster, 38 or less = slower; under 5 dated launches it is not measured). The HHI of 0.060 means stars are spread as if across about 17 equally popular projects — unconcentrated by the common 0.15 / 0.25 thresholds. Stars measure attention, not usage or quality.
Limits
- Classification is by the keywords above. A product that offers this capability without using one of them is not counted; one that uses a keyword in another sense is counted wrongly. Methodology
- Platform reach is protocol-level: it says a platform documents the standard, not that any product here was tested on it. Platforms named by publishers are their own claims.
- Stars belong to repositories. Products in a subfolder of a large repository show that whole repository’s stars and are excluded from the concentration figures.
- Launch counts use each source’s publication date; for auto-discovered repositories that is the repository’s creation date.
Page data last updated .