Yuval Avidani
Author
Let's fly high, but let's take off from the ground, not from the clouds. Everyone talks about LLMs, but almost nobody actually knows what's happening inside, so most people use this tool like amateurs: they throw in two words and hope for magic. I'm going to break this down to the very last crumb for you: what it is, why it works the way it does, and how it works under the hood, and along the way you'll stop and play with a real language model I built for you right here on this page. Once you understand what's happening inside, you'll stop fearing the tool and start controlling it.
First things first: what even is an "LLM"?
Those three letters stand for Large Language Model. Let's break down every word, because each one matters:
- Language: This thing deals with text, with words and sentences. Not images, not numbers, just written human language.
- Model: A "model" is a mathematical representation of a pattern in reality. Just like a weather model is a formula that predicts rain from past data, a language model is a mechanism that learned one very specific pattern from enormous amounts of text. We'll see exactly which pattern in a second.
- Large: "Large" describes how many things the model "remembers" (these are called parameters, we'll get back to that). For now just hold onto the intuition: it saw a ton of text, and it has a ton of "internal dials" that steer its guesses.
The one idea you need to understand: guessing the next word
Here's the big secret, and it's shockingly simple. Everything an LLM does at its core is one thing: given a sequence of words, it guesses the most likely word to come next. That's it. That's the whole story.
Think about autocomplete on your phone's keyboard. You type "I'm on my way", and the keyboard suggests "home." How does it know? It's seen thousands of messages where "I'm on my way" was usually followed by "home." A language model is the exact same idea, except instead of your thousands of messages, it saw a massive chunk of the internet: books, articles, Wikipedia, forums, code. The more text it saw, the sharper and richer its guess about the next word becomes.
And how do you build a whole sentence out of guessing one word? Simple: you repeat it. The model guesses a word, adds it to the sequence, then guesses again based on the new sequence, and so on, word after word, until a whole paragraph forms. Every impressive answer you've ever gotten was built exactly this way: guess, after guess, after guess.
Sounds abstract? Then let's stop talking and start playing. Below is a real language model, tiny but completely real, that I built right here in the browser. It counted, from real Hebrew text, which word comes after which, and every percentage you'll see is being calculated right now from that count:
What you just saw, exactly
Let's make sure every crumb is clear, because this is where the whole understanding lives:
- Why are there several possible words and not just one? Because in the text the model "trained" on, several different words appeared after a given word. For example, after "I" there were appearances of "love," "build," and "learn." The model doesn't pick one and erase the rest, it holds onto all of them, each with a probability.
- What's that % next to each word? That's the probability, meaning how likely it is that this specific word comes next. How does it calculate it? Very simply: if "love" appeared 4 times after "I" and "want" appeared 2 times, then "love" is twice as likely. The probability is basically just this relative count. The machine doesn't "know" what's true, it measures what's common.
- What did the "temperature" slider do to you? Did you notice that when you lowered it, the machine almost always picked the most common word, and when you raised it, the probabilities flattened out and it dared to pick surprising words? Temperature is literally "how much to risk." Low = careful and predictable. High = creative and surprising. This is the exact same "temperature" knob you'll find in the API of every real model, and now you know what it does from the inside, not just that "it affects things."
The only difference between this toy and GPT or Claude is scale: my model counted word-after-word across a few dozen sentences; a giant model "counts" far more sophisticated patterns, over a massive chunk of the internet. But the mechanism itself, guess the likely next thing based on what it's seen before, is exactly identical.
Why it "completes" rather than "knows," and this is where hallucinations are born
This is the most important point in this guide, so stop and really absorb it. The model doesn't reach into a database of facts to pull out an answer, it generates the most probable text that fits the question. This isn't wordplay; it changes everything.
Picture a brilliant improv actor who's read a million books. You throw him the start of a sentence, and he continues with such convincing confidence that it sounds like truth. Most of the time he'll be right, because the text he "grew up on" was mostly correct. But when he doesn't know something, he won't stop and say "I have no idea." He'll just keep improvising something that sounds right. In AI this is called a hallucination, and now you understand why it happens: it's not a random glitch, it's a direct and unavoidable result of "guess the most probable next word." The model is tuned for fluency and likelihood, not truth. That's why it can invent a source, a date, or a number with complete confidence, and that's exactly why you should never rely on it as a factual source without verifying.
Wait, does it work on "words" or something else? (Tokens)
Up until now I said "word" to build your intuition, but here's the precise version: a real model doesn't work on whole words, it works on tokens, pieces of words. A token is the smallest unit the model reads and writes: sometimes a whole word ("hello"), sometimes half a word, sometimes just a single suffix.
Why break words into pieces instead of working with whole words? Two reasons, both practical:
- So it doesn't get stuck on a word it's never seen. If the model only worked on whole words, a new word that never appeared in training (a brand name, a rare word, a new phrase) simply "doesn't exist" for it. But if it knows pieces, it can assemble almost any word from familiar chunks, exactly like how you can read a word you've never seen before by sounding out its syllables.
- To save resources. The number of words in a language is huge; the number of common pieces is much smaller. Working on pieces is simply more efficient.
This also explains a practical fact you'll run into: Hebrew text usually breaks into more tokens than English does, because most models were trained mainly on English, so their "pieces" are tuned for English, and Hebrew ends up chopped into smaller chunks. So the same sentence in Hebrew "costs" more and sometimes hits limits faster. When you hear about the "context window" (how much text a model can hold at once), the unit being counted is tokens, not words.
So how does it "understand" you at all?
If all it does is guess the next piece, how does it grasp context, intent, or a complex request? This is where a mechanism called the Transformer comes in, and at its heart is an idea called attention. No formulas, just the intuition and the why:
The model doesn't treat a word as a word, it turns every word into a list of numbers that represents its meaning relative to everything else. In this space of numbers, "king" and "queen" sit close to each other, and "Paris" relates to "France" the same way "Tokyo" relates to "Japan." The model learns the relationships between concepts, not just the words themselves. The attention mechanism lets it, while guessing the next piece, weigh which earlier words in the text are most relevant to this exact moment, and that's precisely what lets it track context across a whole paragraph, instead of just looking at the last word like the toy model above.
Why any of this actually matters to you
This understanding turns you into a completely different user. Three practical things that follow directly from the mechanism:
- Give it rich context. The model guesses based on the sequence you give it. "Write me an email" will get a generic answer; "write a short, polite email to a client who's late on payment, friendly but firm tone" will get a great one. The richer and clearer your sequence is, the better its guesses are.
- Always verify facts. Now you know why: it's tuned for likelihood, not truth. Every number, date, quote, or source, check them. It's excellent at phrasing, analysis, and ideas; it's dangerous when treated like an encyclopedia.
- Play with the temperature. You've already seen with your own eyes what it does. When you need precision, ask it to be concise and factual (essentially low temperature); when you need ideas, let it run wild.
An LLM is neither a magician nor a prophet. It's a brilliant guessing engine trained on a massive chunk of human language. The moment you treat it that way, a smart partner that needs direction and oversight, not an all-knowing oracle, you stop fearing it and start flying high with it.