How an LLM works: the 10-minute basics course

By Ivo Donker — compiled with AI assistance (Claude & Gemini)

Lesson module context: This article falls under Module 1: Orientation & Fundamentals of the learning program on llmnet.nl. Here you learn to understand the basic mechanisms behind advanced language models.

When you ask an AI system such as ChatGPT, Claude, or Llama a question today, a fluent and seemingly intelligent answer appears on your screen within a matter of seconds. For many users, this feels as if there's a digital expert on the other end of the line who knows humanity's library by heart and understands what you mean. The reality under the hood, however, is a fascinating feat of large-scale statistics, advanced mathematics, and enormous computing power.

At its core, a Large Language Model (LLM) is not a knowledge system, not a search engine, and not a digital consciousness either. It's an advanced statistical pattern recognition system trained to perform one specific task: predicting the most probable next piece of text based on the text that precedes it. In this ten-minute guide, we unravel step by step how this mechanism works, from the moment you type a question to the final answer appearing on your screen.

1. What is a Large Language Model, fundamentally?

To understand how an LLM functions, we first need to let go of the idea that the model 'remembers' facts the way a database does. When a traditional database stores information, it keeps exact rows of text, numbers, and relationships in tables. If you ask for the capital of the Netherlands, the database looks it up in a fixed table and returns 'Amsterdam'.

A Large Language Model works fundamentally differently. It doesn't store exact text files. Instead, during its training phase, it analyzed billions of pages of text and derived statistical patterns from them. You can compare it to the way a human learns how a language 'sounds' or is structured. The model learns which words, sentence structures, and concepts regularly occur near each other. If you ask the question: "What is the capital of the Netherlands?", the model doesn't look in a filing cabinet, but calculates which sequence of characters is the most logical and probable continuation of that question.

This way of working directly explains both the enormous power and the built-in pitfalls of the technology. Because the model understands patterns, it can summarize texts, write code, rephrase, and write in different styles. But because it relies on probabilities and not on a hard fact-check, it can generate an incorrect answer with just as much confidence as whatever the pattern happens to suggest.

2. The first step: From raw text to tokens

A neural network and a computer processor cannot work directly with letters, words, or punctuation marks. Computers calculate exclusively with numbers. The very first step in processing your input (the so-called prompt) is therefore converting human language into numerical values. We call this process tokenization.

A token is a building block of language. It's not always exactly one word. For common Dutch words, one token can correspond to a whole word, such as kat or lopen. For rarer words, long compounds, or specific conjugations, so-called subword tokenizers chop the word up into several pieces. The Dutch word zonnepaneelinstallatie is, for example, split into separate fragments such as zonne, paneel and installatie. Spaces and punctuation marks are also counted as separate tokens.

If you want to understand how words are split into separate fragments by algorithms, read the in-depth article on tokenization explained on leren.llmnet.nl. There you'll discover exactly how subword splitting works and why this is mathematically so efficient.

After splitting, each unique token gets a fixed number from the model's 'dictionary' (the vocabulary). A model such as GPT-4 or Llama has a vocabulary of often 32,000 to more than 100,000 unique tokens. Your sentence "The cat sits on the couch" is converted by the tokenizer into a sequence of numbers, for example [1452, 8931, 312, 410, 1452, 4521]. From that moment on, the model calculates exclusively with this sequence of numbers.

3. Memory and weights: Where is the 'knowledge' stored?

After the text is converted into numbers, these values pass through the model's neural network. But where exactly is the knowledge, or the ability to give a sensible answer, located? That's stored in the model's so-called 'parameters' or 'weights'.

The weights are billions (and for the largest models, hundreds of billions) of mathematical numbers that determine the strength of the connections between the virtual neurons in the network. During the training phase, all these weights start out as random numbers. In the beginning, the model is essentially just guessing. As soon as the model makes a mistake during training predicting the next word in a training sentence, an algorithm (backpropagation) calculates how the weights need to be slightly adjusted so as not to make that mistake again next time.

By repeating this process thousands of billions of times across enormous data centers, the weights settle into a configuration that captures patterns in language, logic, facts, and even programming code in an abstract way. For a detailed dive into how these numbers are stored and adjusted during training, you can consult the explanation of parameters and weights in AI models on leren.llmnet.nl . This article clarifies exactly how the internal computation matrix is structured.

An important aspect of this is that the model's knowledge is 'frozen' after training. A standard LLM doesn't keep learning during the conversation you have with it. Once training stops, the weights no longer change. All the context you provide in the current conversation is used purely as temporary input for the mathematical calculation of the next token.

4. The Transformer architecture: Attention and context

The big breakthrough in modern AI took place in 2017 with the introduction of the Transformer architecture. Older language models read text strictly from left to right, word by word. As a result, they quickly forgot the beginning of a long sentence by the time they reached the end, or got confused by complicated sentence structures.

The Transformer introduced the concept of 'Self-Attention'. This enables the model, when processing a specific token, to simultaneously look at all other tokens in the context and calculate which tokens are most relevant to each other. Take the following two sentences as an example:

• "De bank staat in het park en zit erg comfortabel." ["The bench is in the park and is very comfortable to sit on."]
• "De bank heeft de rente op de spaarrekening verhoogd." ["The bank has raised the interest rate on the savings account."]

In the first sentence, the attention mechanism learns that the word 'bank' has a strong mathematical relationship with 'park' and 'comfortabel' [comfortable] (a piece of furniture). In the second sentence, the mechanism links 'bank' to 'rente' [interest] and 'spaarrekening' [savings account] (a financial institution). The model uses these interrelationships to determine the exact meaning of a token within the specific context.

To zoom in deeper on the mathematics of attention mechanisms, you can consult the article on what a transformer is on leren.llmnet.nl . This explains how this neural network architecture makes connections over long distances in texts.

5. Word by word: How the generation process works

When the prompt has flowed through all the layers of the network, the final layer calculates a so-called probability distribution over the entire vocabulary. The model doesn't immediately output one definitive answer; for every single possible token in its vocabulary, it produces a percentage indicating how likely it is that this token is the correct successor.

Suppose the input is: "De kat drinkt een bakje". The model then calculates, for example, the following probabilities:

• melk: 65% probability
• water: 25% probability
• koffie: 2% probability
• auto: 0.0001% probability

Now the model has to make a choice about which token it actually picks. We call this process 'sampling'. If the model always simply picks the token with the highest probability (greedy search), answers often become very predictable, repetitive, and rigid. To make answers more natural, varied, or creative, developers add a slight degree of randomness through settings such as Temperature and Top-p.

If you want to understand the controls with which you can adjust this randomness and creativity, read the guide article on sampling parameters such as temperature and top-p on leren.llmnet.nl. With this, you'll learn how to influence the probability distribution for tighter or more creative results.

Once a token is chosen — for example melk — something crucial happens: the chosen token is appended to the end of the existing text. The new input now becomes: "De kat drinkt een bakje melk". The entire mathematical process then runs again to predict the next token after that. We call this step-by-step, repeating process 'autoregressive generation'. An LLM therefore literally generates its answer token by token.

6. Why hardware is crucial and how we make models smaller

Because generating each individual token requires that all of the model's billions of weights be loaded from memory and passed through the compute core, LLMs place extremely high demands on computer hardware. It's not just the compute power (the number of teraflops) that determines speed, but above all the memory bandwidth: how fast the weights can be sent from video memory (VRAM) to the graphics processor (GPU).

A model with 70 billion parameters, where each weight is stored as a 16-bit number (2 bytes), requires at least 140 gigabytes of VRAM purely to keep the model in memory. This exceeds the capacity of virtually all consumer graphics cards. To solve this, researchers and developers use clever optimization techniques such as quantization.

If you're curious how to run these large computational models efficiently on consumer hardware, check out the overview of quantization explained on gids.llmnet.nl. There, it's explained how numerical precision is reduced from, for example, 16-bit to 4-bit to save memory space without losing too much quality.

7. The limits of an LLM: Hallucinations, context window, and latency

Now that we understand that an LLM is essentially an extremely complex probability machine, the fundamental limitations of the technology become immediately clear. It's important to know these weak points if you use AI professionally or for study:

1. Hallucinations: Because the model chooses tokens based on probability and grammatical pattern, and not on an internal truth check, it can make up facts that sound very plausible. The model doesn't 'know' what's true; it only knows what fits well textually.
2. Limited context window: A model cannot process an unlimited amount of text at once. The context window determines how many tokens the model can 'see' at the same time (including your question and the answer being built up). If a conversation gets too long, the oldest messages fall outside the window and the model 'forgets' what was discussed earlier.
3. High latency and compute costs: Because the answer is generated token by token, generating a long text costs a lot of time and power. Especially with complex networks, this requires significant capacity in data centers.
4. Lack of real-time logic or active reasoning: A standard LLM calculates the next token within a fixed number of mathematical steps. It cannot 'pause for a moment to think deeply' before it starts typing, unless specific techniques (such as chain-of-thought or reasoning models) are applied.

8. Practical summary: The roadmap from prompt to answer

To keep the whole process clear in your mind, you can summarize the journey from a prompt to a generated response in the following five consecutive steps:

Step	Phase	What happens under the hood
1	Tokenization	The input text is split into subword tokens by a tokenizer and converted into numbers.
2	Embedding & Attention	The numbers get meaning in a vector space; the Transformer network calculates the interrelationships between all tokens.
3	Probability calculation	The weights in the neural network calculate a probability percentage for every possible next token in the vocabulary.
4	Sampling	Based on parameters such as Temperature, the algorithm selects one specific token from the probability distribution.
5	Autoregressive loop	The chosen token is added to the input text and the entire process repeats for the next token.

By understanding these steps, you come to see that an LLM is a powerful tool for text processing, pattern recognition, and creative support, but that its output must always be critically checked for factual accuracy.