promptfoo
Test, evaluate, and red-team LLM applications
promptfoo is a developer-friendly tool for testing and evaluating prompts, models, and RAG systems. It supports automated evaluations, comparisons, and security red-teaming with a local web viewer.
Key features
- Automated prompt evals
- Model comparison matrix
- Red-teaming and security tests
- Local web viewer
Strengths
- Released under the MIT license
- Easy to set up — beginner-friendly
- Mature project with 24.1k GitHub stars
- Written in TypeScript
promptfoo replaces
Compare promptfoo
18 head-to-head comparisons.
- promptfoo vs OpenClaw
- promptfoo vs Hermes Agent
- promptfoo vs OpenCode
- promptfoo vs Hugging Face Transformers
- promptfoo vs Langflow
- promptfoo vs Dify
- promptfoo vs PaddleOCR
- promptfoo vs RAGFlow
- promptfoo vs OpenHands
- promptfoo vs screenshot-to-code
- promptfoo vs Langfuse
- promptfoo vs DeepEval
- promptfoo vs Ragas
- promptfoo vs Prompt flow
- promptfoo vs Arize Phoenix
- promptfoo vs OpenLLMetry
- promptfoo vs Agenta
- promptfoo vs Weave
Similar self-hosted ai apps
OpenClaw
Self-Hosted AIThe AI that actually does things
Hermes Agent
Self-Hosted AIThe AI agent that grows with you
OpenCode
Self-Hosted AIThe open source AI coding agent
Hugging Face Transformers
Self-Hosted AIState-of-the-art machine learning model library
Replaces OpenAI API
Langflow
Self-Hosted AIVisual framework for building AI agents and RAG pipelines
Replaces Vertex AI Agent Builder
Dify
Self-Hosted AIOpen-source platform for building production LLM apps
Replaces OpenAI Assistants, Vertex AI Agent Builder