AgentSidekick

Alternatives to promptfoo

The closest alternatives to promptfoo are Agenta, Ragas and Giskard: they are also developer tools and share Agent Evaluation, Evaluation, Retrieval-Augmented Generation with it. 9 direct alternatives were found by overlap of capabilities and integrations (top score 0.60), plus 6 adjacent solutions of a different product type. As of 2026-11-21. DER

Subject: promptfoo · Developer tool · OpenAI (Promptfoo) · licence MIT · 28k GitHub stars

Direct alternatives (9)

Same product type. Similarity is the share of capabilities and integrations the two have in common out of all either has (0–1).

#AlternativeSimilarity Shared with promptfooLicence Hosting Stars Last push Differs inCompare
1AgentaAgenta0.60
Agent Evaluation
MITUnknown4.9k2026-11-21fewer starsHead-to-head
2RagasVibrant Labs0.60
Retrieval-Augmented GenerationAgent Evaluation
Apache-2.0Unknown16k2026-11-14licence Apache-2.0, fewer starsHead-to-head
3GiskardGiskard0.60
Agent Evaluation
Apache-2.0Unknown6.0k2026-11-20licence Apache-2.0, fewer starsHead-to-head
4TruLensSnowflake0.60
Agent Evaluation
MITUnknown3.7k2026-11-20fewer starsHead-to-head
5DeepEvalconfident-ai0.60
Agent Evaluation
Apache-2.0Unknown20k2026-11-19licence Apache-2.0, fewer starsHead-to-head
6InspectUK AI Security Institute0.60
Agent Evaluation
MITUnknown3.3k2026-11-20fewer starsHead-to-head
7OpenAI Evalsopenai0.60
EvaluationAgent Evaluation
MITUnknown20k2026-04-14fewer starsHead-to-head
8BraintrustBraintrust0.60
EvaluationAgent Evaluation
UnknownUnknownUnknownUnknownlicence unknownHead-to-head
9PyRITmicrosoft0.43
EvaluationRed TeamingAgent Evaluation
MITUnknown5.0k2026-11-21fewer starsCompare

The competitor graph keeps at most the 10 closest products per subject (similarity ≥ 0.34). "Differs in" compares licence, hosting and stars against promptfoo.

Adjacent solutions (6)

Overlapping capabilities, but a different product form: the same job done as a skill, plugin, library or service instead.

ProductTypeSimilarity SharedStars
aludelInfrastructure0.50
Prompt OptimizationEvaluationRed TeamingAgent Evaluation
41
ai-evaluation-toolsInfrastructure0.44
EvaluationRed TeamingRetrieval-Augmented GenerationAgent Evaluation
16
skillcheckRules pack0.43
Prompt OptimizationEvaluationAgent Evaluation
31
judgevalInfrastructure0.43
Prompt OptimizationEvaluationAgent Evaluation
1.1k
gepaAgent skill0.40
Prompt OptimizationAgent Evaluation
7.5k
awesome-evalsInfrastructure0.40
EvaluationAgent Evaluation
1.0k

Head-to-head comparisons

How alternatives are determined

For every profiled product, the pipeline takes its specific capabilities (keyword- or researcher-assigned taxonomy nodes, not inherited parents) and the integrations named in its metadata, and computes the Jaccard overlap with every other profiled product: shared items divided by all items either has. Pairs scoring at least 0.34 are kept, the 10 best per product. Same product type is a direct alternative; a different type is adjacent.

Limits: capabilities come mostly from keyword rules on publisher descriptions, so a product with a short or unusual description can be missed or mis-paired. Similarity says the descriptions cover the same ground, not that the products are interchangeable, equally good or tested together. Stars are GitHub popularity for the linked repository. AgentSidekick has not run any of these products.

See the methodology, or compare promptfoo with the top three in the compare tool.

Page data last updated .