AgentSidekick

Model Serving

Inference servers for production model hosting. As of 2026-11-21, 54 indexed products offer it (31 with full profiles), in 7 forms: 27 MCP servers, 11 Agent skills, 11 Infrastructure, 2 Plugins. 17 were published in the last 90 days, against 8 in the 90 days before. docker-android has the most GitHub stars, 24% of the total across 43 products with their own repository.

CapabilitySegment label: Lagging the catalogue DERScored as an opportunity

Available as

All in the directory

Profiled products grouped by product form. Within each form: own-repository products by GitHub stars, then products in a subfolder of a larger repository (they show that repository's stars), up to 10 each.

MCP servers 27 indexed · 4 profiled

Protocol-level reach: 36 of the 45 active platforms AgentSidekick tracks document support for MCP. DER Which platforms

Model Serving: MCP servers by GitHub stars
#ProductPublisherCapabilities GitHub stars Last push Save and compare
1docker-androidAndroid in docker solution with noVNC supported, video recording, mcp server and AI-agentbudtmo16k2d ago
2ollayaJev-compatible API server for open decision models. Ollama for decision models: pull and serve 29 model families (Laya, pplx-decider, Clef, Winnow, coba...) locally or self-hosted behind /v1/systemone. The TypeSafe SDK works unchanged: set its base URL and pick an open model. CLI, desktop app, Docker, MCP server, public bench.ollaya-dev1.5k2d ago
3lmConnect Claude to any OpenAI-compatible LLM endpoint and offload routine work to a local model.houtini-ai12214d ago
4model-composeDeclarative AI pipelines in one YAML file. Tokens, audio chunks, and video frames flow between isolated components. Compose 100+ components for models, agents, speech, vision, and live broadcast. Run local models, cloud APIs, or both. Inspired by docker-compose.hanyeol118today

Agent skills 11 indexed · 11 profiled

Protocol-level reach: 26 of the 45 active platforms AgentSidekick tracks document support for the Agent Skills format. DER Which platforms

Model Serving: Agent skills by GitHub stars
#ProductPublisherCapabilities GitHub stars Last push Save and compare
1AI-Research-SKILLsComprehensive open-source library of AI research and engineering skills for any AI model. Package the skills and your claude code/codex/gemini agent will be an AI research agent with full horsepower. Maintained by Orchestra Research.Orchestra-Research15k1mo ago
2llamaguardMeta's 7-8B specialized moderation model for LLM input/output filtering. 6 safety categories - violence/hate, sexual content, weapons, substances, self-harm, criminal planning. 94-95% accuracy. Deploy with vLLM, HuggingFace, Sagemaker. Integrates with NeMo Guardrails.Orchestra-Research—1mo ago
3slime-rl-trainingProvides guidance for LLM post-training with RL using slime, a Megatron+SGLang framework. Use when training GLM models, implementing custom data generation workflows, or needing tight Megatron-LM integration for RL scaling.Orchestra-Research—1mo ago
4openrlhf-trainingHigh-performance RLHF framework with Ray+vLLM acceleration. Use for PPO, GRPO, RLOO, DPO training of large models (7B-70B+). Built on Ray, vLLM, ZeRO-3. 2× faster than DeepSpeedChat with distributed architecture and GPU resource sharing.Orchestra-Research—1mo ago
5hqq-quantizationHalf-Quadratic Quantization for LLMs without calibration data. Use when quantizing models to 4/3/2-bit precision without needing calibration datasets, for fast quantization workflows, or when deploying with vLLM or HuggingFace Transformers.Orchestra-Research—1mo ago
6evaluating-llms-harnessEvaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.Orchestra-Research—1mo ago
7sglangFast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.Orchestra-Research—1mo ago
8serving-llms-vllmServes LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.Orchestra-Research—1mo ago
9hugging-face-evaluationAdd and manage evaluation results in Hugging Face model cards. Supports extracting eval tables from README content, importing scores from Artificial Analysis API, and running custom model evaluations with vLLM/lighteval. Works with the model-index metadata format.sickn33—1d ago
10evaluation-lm-evaluation-harnessEvaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.davila7—today

All 11 profiled Agent skills with this capability

Infrastructure 11 indexed · 11 profiled

No compatibility evidence is collected for this form.

Model Serving: Infrastructure by GitHub stars
#ProductPublisherCapabilities GitHub stars Last push Save and compare
1BentoMLThe easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!bentoml8.9k4d ago
2cube-studiocube studio开源云原生一站式机器学习/深度学习/大模型AI平台,mlops算法链路全流程,算力租赁平台,notebook在线开发,拖拉拽任务流pipeline编排,多机多卡分布式训练,超参搜索,推理服务VGPU虚拟化,边缘计算,标注平台自动化标注,deepseek等大模型sft微调/奖励模型/强化学习训练,vllm/ollama/mindie大模型多机推理,私有知识库,AI模型市场,支持国产cpu/gpu/npu 昇腾生态,支持RDMA,支持pytorch/tf/mxnet/deepspeed/paddle/colossalai/horovod/ray/volcano等分布式tencentmusic5.1k1mo ago
3vllm-ascendCommunity maintained hardware plugin for vLLM on Huawei Ascendvllm-project3.1ktoday
4cube-studiocubestudio开源云原生一站式机器学习/深度学习/大模型AI平台/MaaS/mlops/人工智能平台/训推平台,算法全链路流程,多租户,算力租赁平台,token中转,拖拉拽任务流pipeline编排,多机多卡分布式训练,超参搜索,推理服务,VGPU虚拟化,云边端协同,边缘计算,自动化标注平台,deepseek等大模型sft微调/奖励模型/强化学习训练,vllm/ollama/mindie大模型多机推理,私有知识库llmops智能体,AI模型市场,支持国产异构算力调度,昇腾/寒武纪/海光/摩尔/沐曦等,支持ib/roce/RDMA,信创支持data-infra2.6k1mo ago
5envd🏕️ Reproducible development environment for humans and agentstensorchord2.2k1d ago
6aiciAICI: Prompts as (Wasm) Programsmicrosoft2.1k1y ago
7rtp-llmRTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.alibaba1.4ktoday
8prometheus-evalEvaluate your LLM's response with Prometheus and GPT4 💯prometheus-eval1.1k1y ago
9juliusSimple LLM service identification - translate IP:Port to Ollama, vLLM, LiteLLM, or 60+ other AI services in secondspraetorian-inc256today
10heronAgent and LLM API performance monitoring via network packet probe. Measures performance of OpenClaw, Claude, Codex, DeepAgents and more — deployed on the provider side, no SDK changes required.Netis1041mo ago

All 11 profiled Infrastructure with this capability

Plugins 2 indexed · 2 profiled

2 of these are Claude plugins, which Claude Code documents (protocol-level). DER

Model Serving: Plugins by GitHub stars
#ProductPublisherCapabilities GitHub stars Last push Save and compare
1amd-skillsAMD's verified Agent Skills in one plugin: route image/audio through local AI on Ryzen AI, serve LLMs on AMD Instinct GPUs with vLLM, optimize inference throughput with Hyperloom, and analyze GPU kernel and PyTorch trace performance.amd509today
2databricksDatabricks skills for the CLI, Apps, Lakebase, Model Serving, Lakeflow Jobs, Spark Declarative Pipelines, Declarative Automation Bundles (DABs), and classic-to-serverless migration.databricks407today

Agent projects 1 indexed · 1 profiled

No compatibility evidence is collected for this form.

Model Serving: Agent projects by GitHub stars
#ProductPublisherCapabilities GitHub stars Last push Save and compare
1OlaresOpen-Source Personal Cloud OS for Always-On Agentsbeclab5.4ktoday

Frameworks 1 indexed · 1 profiled

No compatibility evidence is collected for this form.

Model Serving: Frameworks by GitHub stars
#ProductPublisherCapabilities GitHub stars Last push Save and compare
1taOSSelf-hosted AI agent OS. Your memory, chat, agents, and files stay on hardware you own, offline by default, cloud by choice. Offline AI memory (taOSmd), self-hosted multi-framework group chat, a full web desktop + app store, and auto-clustering across the consumer hardware you already have (Orange/Raspberry Pi, Mac mini, gaming PC).jaylfc6091d ago

IDE extensions 1 indexed · 1 profiled

No compatibility evidence is collected for this form.

Model Serving: IDE extensions by GitHub stars
#ProductPublisherCapabilities GitHub stars Last push Save and compare
1Vibe DGC — coding agent for your own modelsWork, ChatDGC and Pips in VS Code and Cursor, on your own local or cloud models or a signed-in Claude Code or Codex.vibedgc110d ago

Which platforms these forms can reach

All platforms

Protocol-level: the platform documents support for the standard. No specific pairing has been tested.

Platforms publishers name

PlatformProfiled products naming it
Claude Code2
Codex2
Cursor1
OpenClaw1

Integrations most often paired

Third-party services named by products with this capability, whole catalogue.

IntegrationProducts Share
LinkedIn23.7%
Android11.9%
Databricks11.9%
Grafana11.9%

Keywords that trigger this capability

Matched as whole words, ignoring case, in a product's name, description and repository topics.

KeywordProducts matched
model serving7
vllm41
inference server1
text generation inference0
triton inference0
sglang5

Segment statistics

Whole catalogue (54 indexed products). Calculated 2026-11-21.

54DERIndexed products31 with full profiles
17DERLaunched, last 90 days8 in the 90 days before
29DERMomentum vs catalogue50 = same pace as the catalogue
24%DERLargest project's share of starsAcross 43 products with their own repository
0.146DERConcentration (HHI)About 7 equal-sized projects’ worth; unconcentrated
7SRCMedian starsPer product with its own repository
87%DEROpen sourceMarked open source; unknown counts as not

Lagging the catalogue. Mid-sized among capabilities, with launches growing more slowly than the catalogue’s. Size is ranked against all capabilities; momentum compares launches in the last 90 days with the 90 before, relative to the catalogue’s own ratio (62 or more = faster, 38 or less = slower; under 5 dated launches it is not measured). The HHI of 0.146 means stars are spread as if across about 7 equally popular projects — unconcentrated by the common 0.15 / 0.25 thresholds. Stars measure attention, not usage or quality.

Scored as an opportunity

Score breakdown

Coverage gap: Model Serving: score 52.1 HEU, competition High, confidence Low. Only 54 indexed products match "Model Serving", against 388 in Local & Open Models overall.

Limits

  • Classification is by the keywords above. A product that offers this capability without using one of them is not counted; one that uses a keyword in another sense is counted wrongly. Methodology
  • Platform reach is protocol-level: it says a platform documents the standard, not that any product here was tested on it. Platforms named by publishers are their own claims.
  • Stars belong to repositories. Products in a subfolder of a large repository show that whole repository’s stars and are excluded from the concentration figures.
  • Launch counts use each source’s publication date; for auto-discovered repositories that is the repository’s creation date.

Page data last updated .