Reward Model Playground

Click the answer you think is better. Each click is one pairwise preference — exactly the data Surge collects. A Bradley–Terry reward model refits live in your browser after every click and ranks all six responses by learned reward. Watch it converge.

Prompt: “Explain why the sky is blue.”

0 preferences collected

Learned reward (Bradley–Terry, MLE)

Training signal

Log-loss

—

Train agreement

—

Loss = mean binary cross-entropy of the model predicting your picks. It drops as the model's rewards line up with your revealed preferences.

Under the hood: reward r_i per response, P(A ≻ B) = σ(r_A − r_B), fit by gradient descent on your clicks with L2 regularization — the same objective a real RLHF reward model trains on, just 6 items instead of millions. No server, no dependencies, one file. — built by Claude on Patrick's Mac