Chain-of-thought: Letting a model reason in steps
When a Large Language Model (LLM) tries to solve a complex task directly, it makes mistakes more often. By forcing the model to write out its reasoning ("chain of thought") step-by-step before giving a final answer, reliability and accuracy increase significantly.
What is Chain-of-Thought (CoT)?
Under the hood, language models continuously calculate the most logical next word. If you ask a model to generate the answer to a complex problem all at once, it lacks the 'breathing room' to process logical steps. CoT solves this by literally having the model write out the necessary thinking steps in its own output. This acts as a kind of "scratchpad" for the AI.
When does this help?
Chain-of-thought is not necessary for every prompt. It specifically adds value in:
- Mathematical calculations and logic: Where multiple variables affect each other.
- Architecture and system design: For example, when designing a database schema or a complex RAG infrastructure with vector search.
- Complex data extraction: Analyzing large blocks of text with multiple conditions.
How do you ask for it?
The simplest way to activate this is via Zero-Shot CoT by adding a specific phrase to your prompt. Another method is Few-Shot CoT, where you provide an example of the desired reasoning style in your prompt.
"Which AI model is the most cost-effective for my application with 50,000 API calls per day, knowing that model A costs €0.01 per call with a 5% discount, and model B costs €0.009 per call plus €10 fixed costs?"
"Which AI model is the most cost-effective for my application with 50,000 API calls per day, knowing that model A costs €0.01 per call with a 5% discount, and model B costs €0.009 per call plus €10 fixed costs? Think step-by-step out loud. First, write out the full calculation for both models before giving your final advice."
Pitfalls and Considerations
Watch out for token consumption: Because the model generates more text, both the latency (waiting time) and the cost per request increase. Only use CoT when complexity requires it.
Additionally, it is important to remember that CoT does not rule out hallucinations; a model can confidently set up a flawed reasoning. For objective performance evaluations, it is best to compare the output with hard data, such as can be found on our benchmark results page.
Exercise: Your turn
Imagine you want to write a script that retrieves data from a vector database, processes it with a model like DeepSeek or Claude, and saves it to a local JSON file. You notice that the AI always tries to code directly, resulting in errors.
The assignment: Rewrite your prompt so that the AI is forced to first write out a pseudo-code step-by-step plan for this Retrieval-Augmented Generation (RAG) pipeline, and only then deliver the final Python code. Share your successful prompts and outputs in the community.