OpenRLHF
High-performance RLHF framework for training LLMs
OpenRLHF is a scalable framework for reinforcement learning from human feedback, supporting PPO, DPO and other algorithms on large language models. It can be self-hosted on GPU clusters for end-to-end RLHF training.
Key features
- PPO and DPO support
- Distributed training
- Ray-based scaling
- vLLM-accelerated generation
Strengths
- Released under the Apache-2.0 license
- Mature project with 9.9k GitHub stars
- Written in Python
OpenRLHF replaces
Compare OpenRLHF
13 head-to-head comparisons.
Similar self-hosted ai apps
OpenClaw
Self-Hosted AIThe AI that actually does things
Hermes Agent
Self-Hosted AIThe AI agent that grows with you
OpenCode
Self-Hosted AIThe open source AI coding agent
Hugging Face Transformers
Self-Hosted AIState-of-the-art machine learning model library
Replaces OpenAI API
Langflow
Self-Hosted AIVisual framework for building AI agents and RAG pipelines
Replaces Vertex AI Agent Builder
Dify
Self-Hosted AIOpen-source platform for building production LLM apps
Replaces OpenAI Assistants, Vertex AI Agent Builder