What is RLHF?

RLHF stands for Reinforcement Learning from Human Feedback. That might sound technical, so let’s unpack it one piece at a time.

Reinforcement learning is a type of machine learning where a system improves by receiving rewards for good behavior. Think of training a dog to sit. The dog tries things until it finally sits, and when it does, it gets a treat. Repeat this enough times and the dog associates sitting with the reward. Reinforcement learning on a machine learning model works the same way in principle.

Human feedback is in this case that “treat”. The “treat” isn’t calculated by an algorithm or pulled from a database. It comes from a person telling the model which of its outputs is better and why.

Put them together: RLHF is the process by which AI models learn to give better, more helpful, more accurate responses by receiving structured feedback from humans.

Here’s the key thing to understand: today’s AI assistants all went through an RLHF-like process. Pre-training gave these models knowledge. Human feedback is a large part of what made them helpful and safe to use: thousands of people told the model, over and over, which answers were better, and the model learned from that signal.

Companies pay well for RLHF because the quality of human feedback directly determines the quality of the resulting AI model. Garbage feedback in, garbage model out.

Rates depend on domain and task. On Sovrano, they typically run from €30/hr for preference ranking to €150+/hr for specialist SFT writing, paid monthly. AI labs and training data companies spend heavily on human feedback work, and that spend is growing.

Why so much? Because the alternative is worse. If you train a model’s reward signal using evaluators who can’t tell a good financial explanation from a bad one, the model learns to produce confidently wrong financial content. For companies deploying AI in regulated industries like finance, law, healthcare or education, skipping careful RLHF is not an option.

The flip side: good evaluators, people who catch subtle factual errors, spot weak reasoning and explain why one response is better, produce feedback that measurably improves the model.

This is the bread and butter of RLHF, the task type you’ll encounter most often.

How it works: You’re shown a prompt (a question or instruction that a user might give to an AI) and two or more AI-generated responses to that prompt. Your job is to rank the responses from best to worst.

Sometimes it’s a simple head-to-head: Response A versus Response B, pick the winner. Other times, you’ll rank three to five responses in order. The platform will usually give you specific criteria to evaluate against things like factual accuracy, helpfulness, clarity, completeness, and safety.

What it feels like: Honestly, it’s closest to grading essays. You’re reading carefully, checking claims, assessing whether the response actually addresses what was asked, and making a judgment call. The difference is that you’re not assigning a letter grade, you’re saying “this one is better than that one” and explaining why.

The hard part: The easy comparisons (one response is clearly wrong) aren’t where you earn your pay. The hard part is when both responses are decent and you need to identify which is slightly better and articulate the nuance. That requires attention to detail and domain knowledge.

How it works: Instead of comparing responses against each other, you evaluate a single AI response against a rubric, rating it on a numerical scale (often 1 to 5 or 1 to 7) across multiple criteria. Rubric scoring is covered in depth in our deep annotation guide.

A typical rubric might ask you to score:

• •Accuracy: Is the information factually correct?
• •Helpfulness: Does it actually answer the question that was asked?
• •Completeness: Does it cover all the relevant points, or does it leave important things out?
• •Clarity: Is it well-written and easy to follow?
• •Safety: Does it avoid harmful, misleading, or inappropriate content?

The key skill: Calibration. Your 4 out of 5 needs to mean the same thing as every other evaluator’s 4 out of 5. Platforms achieve this through training rounds where you rate example responses and compare your scores against a “gold standard.” If you’re consistently harsh or consistently lenient relative to the standard, you’ll need to adjust.

How it works: SFT stands for Supervised Fine-Tuning. In these tasks, you don’t evaluate existing responses, you write new ones. You’re given a prompt and asked to write the ideal response from scratch.

This is the highest-skill, highest-pay task type. Your response becomes a training example that the model will try to imitate. If you write a sloppy response, the model learns to be sloppy. If you write a precise, well-structured, accurate response, the model learns to do that.

When you’ll see these: SFT tasks are most common in domain-specific work. A project might recruit finance graduates and professionals to write ideal responses to financial questions, or engineers to write ideal technical explanations. Your professional or academic background is the reason you’re needed for these tasks.

How it works: Across all of the task types above, you’ll almost always be asked to write a short explanation of your decision. This rationale is important for two reasons: it lets QA reviewers verify that you made your choice for the right reasons, and it becomes data that helps the training pipeline understand why certain responses are better.