Information Theory and Entropy in Word Games
You do not need mathematics to play well. But a little of it explains why some guesses feel productive and others feel wasted.
Last updated October 6, 2026
Information is reduction of uncertainty
In 1948 Claude Shannon proposed measuring information by how much it reduces uncertainty. The unit is the bit: the amount of information in the answer to a yes-or-no question whose two answers are equally likely.
If a puzzle’s answer could be any one of 16 equally likely words, you need four perfectly chosen yes/no questions to identify it, because each one halves the field: 16 → 8 → 4 → 2 → 1. In general, picking one item out of N equally likely items takes log2(N) bits.
Entropy: the average surprise
Real possibilities are rarely equally likely. Entropy is the average amount of information you expect to receive, weighted by how likely each outcome is. A question whose answer you can already predict carries little information; a question that could go many ways, each about as likely as the others, carries a lot.
That is the whole idea behind an “informative” guess: choose the one whose possible outcomes split the remaining candidates as evenly as possible.
What this looks like in practice
| Situation | Low-information move | Higher-information move |
|---|---|---|
| You know nothing about the word’s letters | Test rare letters such as Q or Z — the answer is almost certainly “not present” | Test letters that are present in roughly half of the candidates |
| You are down to _IGHT words | Guess them one at a time | Where the rules allow, use one guess that tests several of the possible first letters together |
| A clue has two plausible answers | Pick one at random | Solve a linked word first; one shared letter decides it |
Why English is easier than random letters
If all 26 letters were equally likely and independent, each letter would carry about 4.7 bits. English is far more predictable: after Q you expect U; after TH you expect a vowel or R. Shannon’s later experiments, in which people guessed the next letter of a text, suggested that with enough context English carries only around one bit per letter. That built-in predictability, called redundancy, is what lets you read a word with letters missing — and it is what every word puzzle quietly relies on.
Applying it to clue-based games
- A clue is a very large amount of information. It can cut tens of thousands of words to a handful. Statistics on letters only matter for what is left.
- Length and fixed letters multiply together. “Five letters, starts with T, ends with R” is three constraints applied at once.
- Linked answers share information. In ClueLoop, each solved word gives one letter to each neighbor. Solving the easiest clue first is the high-information move because it reduces uncertainty in two other places.
- In SPARES, the bonus clue is information about future steps. Working it out early tells you which letters will be dropped.
Keep it in proportion
Computing exact entropies requires knowing the game’s full answer list, which players normally do not. The practical takeaway is a habit rather than a formula: prefer the action that will teach you the most whichever way it turns out.