Answering Hard Questions: Fermi Estimation

As a quantitative researcher and data scientist, I spend a lot of time fretting over tiny details. In algorithmic trading, that fourth decimal place can make all the difference. But there’s a danger to focusing on minutiae, the risk of missing the forest for the trees. Data is great at providing specific, precise answers (and sometimes the answers are even true!), but bad at answering big-picture questions. And what do you do when the data doesn’t exist?

Those are the hard questions: Big picture questions where specific, high-quality data doesn’t exist.

Enter Fermi Estimation.

Enrico Fermi

Honestly, I don’t know a lot about Enrico Fermi. And frankly I don’t care to know more. It’s the process I’m interested in, not the man. He was a physicist, he worked on the Manhattan Project, and he helped build the first nuclear reactor. But he was also known for making incredibly accurate estimates with very little information.

Enrico Fermi, the namesake of Fermi Estimation

Sadly, Fermi himself died relatively young. But his process for generating incredible estimates lives on.

A Fermi Estimation Example

Here’s the trick: When facing a question about which we have little information, turn it into a function of questions about which we have more information. That’s it. Let’s try an example.

How many words are there in Moby Dick?

I have read the book, but I don’t even know how many pages it is. How can I turn this question — about which I have almost no information — into a function of questions about which I have decent information?

The time it takes to read a book equals the number of words in the book divided by your reading speed. I’m going to start with that function, rearranged to return words.

We’ve separated the question into two questions that are easier to answer.

So far so good, but I also don’t know how long it took me to read Moby Dick or how fast I read. However, I recently finished a book by TJ Klune, Somewhere Beyond the Sea (cannot recommend it highly enough; Klune is a wonderful storyteller). If I recall correctly, that has something like 400 pages and took me about twelve hours. A single-spaced page in Microsoft Word has about 500 words on it. Using all of that information, doing the math in my head, I get a reading speed of approximately 17,000 words per hour. Moby Dick is a harder book to read, so let’s lower that to somewhere between 10,000 and 15,000 words per hour. And I bet it took me longer to read, possibly even twice as long. Let’s say it took between 15 and 24 hours of solid reading time. Now we have a distribution of word counts with these estimated values:

Right now, we have four estimates of the number of words in Moby Dick.

Moby Dick is long but it’s not super long, so we can sanity check those estimates. If there are 500 words per page, they would imply that Moby Dick has:

We use what we know about the number of pages in Moby Dick to check if our four estimates make sense.

I’m just going to discard the estimates with what feels like the wrong number of pages based on how thick I remember the book being, then treat the two remaining estimates as equally likely. Doing the math in my head, we get:

Each of our two reasonable estimates is equally likely.

My final estimate for how many words are in Moby Dick is 232,500. We reached that estimate with only a general idea of how thick the book is, a basic estimate of how many words are on a page, a faint notion that Herman Melville is harder to read than TJ Klune, and a guess at how fast I read.

Ready? Let’s see if we can find the real answer. I picked this question because I have no idea what the answer is but I bet the answer is online somewhere.

I found these two answers.

If the right answer is 218,637, we were off by only 6%!

The Fermi Estimate of 232,500 is really close! We were off by about 10%. Not bad for how little information we started with! That’s the kind of remarkable accuracy that Fermi Estimates are known for.

Why Fermi Estimation Works

There are two good reasons why Fermi Estimation works so well:

1. It allows you to use better information than you could otherwise.

2. Errors are likely to cancel out.

Let’s see how that played out in our example.

Dissecting the Example

At first, the only useful information I had was:

1. A vague recollection of how thick the book was. I knew it was thicker than many books, but not as thick as The Way of Kings by Brandon Sanderson (also an excellent book).

2. A general idea that one page in Microsoft Word is about 500 pages. I’m not even sure where I got that idea, it’s just an estimate I heard once that seems reasonable.

Every other piece of information, we gathered during the Fermi Estimation process.

We started by defining a function that returns the word count, and at each step we made that function easier and easier to estimate.

We did three rounds of simplification.

We moved the question. We started with a question about a book I read several years ago and ended with questions about a book I read last month. By moving the question, we were able to use better information to answer it. That’s half the magic.

Each estimate we made was wrong, but hopefully the effects of those errors cancel out. When we estimate, we don’t know if our number is too low or too high. But if we are too high half the time and too low half the time, we should be really close in the end. Let’s take a closer look at our estimates.

1. We estimated that Somewhere Beyond the Sea is 400 pages. It is really 416 pages. Our estimate was 4% too low.

2. We estimated that there are about 500 words per page. It turns out that’s a big over-estimate: There are 300–350 words per page, on average, in a novel. Our estimate was 43-67% too high.

3. We estimated that I read Moby Dick at a speed of 10,000–15,000 words per hour. We have no way of being certain how accurate that is, but we can guess. Since Somewhere Beyond the Sea is 416 pages and there are ~325 words per page, that puts my reading speed for that book closer to 11,250 words per hour. And I am confident that Moby Dick was slower, so it looks like our reading-speed estimate was too high, and probably by a large margin.

4. We estimated that Moby Dick took me between 15 and 24 hours to read. Again, we have no way of knowing how close that estimate is. Knowing what we know now, it probably took me more than 24 hours to read. 216,000 at 8,000 words per hour is 27 hours.

5. We estimated that Moby Dick has more than 300 pages but less than 720 pages. My copy, from Bantam Classics, has 589 pages. Since the two estimates we used had 450 and 480 pages, this is an under-estimation.

We had three under-estimates and two over-estimates. Those two over-estimates were relatively large, which is why our final estimate was too high. But the errors do partially cancel out! Thus, our final estimate is more accurate than our individual estimates.

Fermi Estimation Process

Like most processes, this is easier when you have a series of steps to follow. Following standardized steps will also help you be more consistent and objective in your estimating. But over-adherence to any processes will eventually force you to make mistakes. So, take these steps as a starting point and be sure to adapt them to your specific situation.

1. Examine the question in detail — What do you actually want to know? What are the constraints? What units should the answer be in? What order of magnitude do you expect the answer to be? How far off can you be without being wrong?

2. Express the solution as a function of variables that are easier to estimate — In our example, we expressed “number of words” as a function of “reading speed” and “reading time”. The solution is objective and hard to guess, but the other variables are subjective and easier to estimate.

3. Estimate the variables you are confident about — Keep your acceptable margin of error in mind; your estimates can be off by around the same margin as your total tolerable margin of error, and as long as the errors partially cancel out, the final estimate will be close enough.

4. Repeat the process until you’ve estimated all the variables — We did three rounds of the process in the Moby Dick example, but you can keep making the estimates simpler as long as you need to. You might want to write it down.

5. Calculate your final estimate — Your final guess is a function of all the easier variables you’ve estimated, so just plug in the numbers.

Fermi Estimation can be applied to almost any question in almost any context. Try it out for yourself, and I’d love to hear what you come up with!