2025 has arrived and to celebrate we’ve updated our site to include details of free online courses in AI suitable for people with disabilities. Our focus remains on making participation in the digital content industry a choice that is available to everyone.
Your feedback on the new site is welcome.
Q: In the context of AI – what do RLHF and CoT mean?
A: In the context of artificial intelligence, particularly with large language models (LLMs), RLHF and CoT are two important and distinct concepts. One is a training method, and the other is a prompting technique.
RLHF: Reinforcement Learning from Human Feedback
RLHF (Reinforcement Learning from Human Feedback) is a machine learning technique used to fine-tune large language models. Its primary goal is to align an AI's behavior with human values, preferences, and instructions. It's a key reason why models like ChatGPT are so good at being helpful, harmless, and following instructions.
​
The process typically involves three main steps:
-
Supervised Fine-Tuning: An existing, pre-trained language model is initially fine-tuned on a smaller, high-quality dataset of human-written prompt-and-response examples. This teaches the model to follow instructions and generate a diverse range of helpful responses.
-
Training a Reward Model: Human annotators are presented with a series of responses generated by the fine-tuned model and are asked to rank them from best to worst. This human feedback is used to train a separate "reward model," which learns to predict a numerical score for how well a given response aligns with human preferences.
-
Reinforcement Learning: The original language model is then fine-tuned again, but this time using the reward model. The language model generates responses and gets a "reward" score from the reward model. Through a process called Reinforcement Learning, the model learns to generate responses that maximize its reward, thus becoming better at producing outputs that humans prefer.
In short, RLHF uses human preferences to create a reward system that guides the AI's learning process, making it more aligned with human expectations.
​
CoT: Chain of Thought
CoT (Chain of Thought) is a prompt engineering technique, not a training method. It's a way of writing prompts to elicit better reasoning and more accurate answers from an LLM, especially for complex, multi-step problems.
The core idea is to encourage the model to "show its work." Instead of just asking for a final answer, you ask the model to break down the problem into a series of intermediate, logical steps.
There are two main ways to implement CoT prompting:
-
Few-Shot CoT: You provide the model with a few examples of problems and their solutions, where each solution is broken down step-by-step. The model then uses this pattern to solve the new problem.
-
Zero-Shot CoT: This is a more recent and powerful variation where you don't provide any examples. You simply add a phrase like "Let's think step by step" to the end of your prompt. This simple instruction is often enough to trigger the model's reasoning capabilities, leading to a much more accurate answer than if you just asked for the final result.
CoT is particularly effective for tasks like mathematical word problems, commonsense reasoning, and symbolic logic, where an incorrect step in the reasoning process can lead to an entirely wrong final answer. By making the model's thought process explicit, CoT not only improves performance but also provides a level of transparency into how the model arrived at its conclusion.