SHSelfHostIndex
OP

OpenRLHF

High-performance RLHF framework for training LLMs

Self-Hosted AI ★ 10.1k stars Hard setup Apache-2.0

OpenRLHF is a scalable framework for reinforcement learning from human feedback, supporting PPO, DPO and other algorithms on large language models. It can be self-hosted on GPU clusters for end-to-end RLHF training.

Official siteSource code

Key features

  • PPO and DPO support
  • Distributed training
  • Ray-based scaling
  • vLLM-accelerated generation

Strengths

  • Released under the Apache-2.0 license
  • Mature project with 10.1k GitHub stars
  • Written in Python

Is OpenRLHF secure?

OSV.dev has no published security advisories for OpenRLHF's repository. No known vulnerabilities is not proof of security: it means none have been publicly catalogued.

Source: OSV.dev, checked 2026-10-14. How this is measured.

OpenRLHF replaces

Self-hosted OpenAI Fine-tuning alternatives

Health badge

Maintain OpenRLHF? Add its health score to your README. The badge links back to this page.

SelfHostIndex: 93 · excellent

Markdown
[![SelfHostIndex health score for OpenRLHF](https://selfhostindex.com/badge/openrlhf.svg)](https://selfhostindex.com/app/openrlhf/)
HTML
<a href="https://selfhostindex.com/app/openrlhf/"><img src="https://selfhostindex.com/badge/openrlhf.svg" alt="SelfHostIndex health score for OpenRLHF"></a>
reStructuredText
.. image:: https://selfhostindex.com/badge/openrlhf.svg
   :alt: SelfHostIndex health score for OpenRLHF
   :target: https://selfhostindex.com/app/openrlhf/

The score is recomputed on every nightly rebuild from public repository data — see how it is calculated.

Compare OpenRLHF

3 head-to-head comparisons.

Similar self-hosted ai apps