Statistics

A coin that flips heads 20 times in a row will produce heads on the 21st, statistics be damned

If a coin flip turns up heads 20 times in a row, what is the next flip going to be? Heads, of course. You could say that there's a 50-50 chance that it will be heads or tails, but with that much consecutiveness, the odds are extremely high that something is wrong with the system instead (like a fake coin). If the interviewer then adds, "Assume the coin is honest," you should insist there is something wrong with the system (Perhaps a robot is flipping the coin the same way each time). Eventually, your interviewer might get frustrated and exclaim, "Fine, assume there's a 50-50 chance it will appear heads," and then the conversation is over.

♦     ♦     ♦

A true believer in statistics would pick 1-2-3-4-5 for lottery numbers without second-guessing themselves

If a statistician played the lottery, they would most likely pick random numbers. They would go to the counter, ask for the "Quick Pick" and then see what happens. However, an even gutsier statistician, one who had a more internalized understanding of randomness and statistics, would pick consecutive numbers, like 1-2-3-4-5. A true believer would say that this is just as likely to win as the Quick Pick, and yet what human statistician wouldn't second-guess themselves for a second while picking consecutive numbers?

True, the likelihood of seeing sequences in the lottery is minuscule, which would seemingly compound the already difficult task of picking the winning the numbers. And yet, if you won with those, it would make a bigger news splash for the world of statistics. It would reinforce to the world that yes, random means that an equal probability of seemingly non-random and bizarre events to occur.

♦     ♦     ♦

DNA reports need nuance, including the average, the margins, the confidence, and the key: what causes the variation in survival

The need for more nuanced language in communicating statistics is becoming more and more urgent. While as it's only a venial rhetorical sin for the media to report averages without context, such simplification can prove catastrophic in a different setting. For example, as more and more people get personal genomics reports, it is bewildering to read that you have twice the risk of getting a rare cancer.

When DNA reports come back, instead of saying, "You have a 12.6% chance," they should say, "We can say with a 95% certainty that you have between a 5% and a 22% chance, with the variation depending on lifestyle choices."

Most statistics communication comes across as cold, reductionist, and fatalistic. And yet, the vocabulary of statistics is broad enough to describe hopeful, free-will data. The trick is that whenever we publish an average, we should accompany it with the standard deviation. And we publish the standard deviation, we should include information on what tends to cause that deviation.

♦     ♦     ♦

Finding an investment strategy that backtests well proves just as much as publishing a scientific study the one time it works

What if I told you that I had a stock-picking strategy that beats the S&P 500 80% of the time? You might ask for some evidence, and so to bolster my claim, I might mention that I backtested the strategy over the past 25 years and in four out of five years, it out-performed the benchmark. Since only 10% of mutual funds beat the S&P 500 in any given year, this might be compelling evidence.

But I'd be withholding some very crucial information: this is the third strategy that I've concocted. My first two showed promise, but ultimately failed in the backtests.

For example, my first strategy was to pick stocks according to the release dates of game-changing products. So, in the case of Apple, you would have made bets in the days after the iPhone and the iPod came out. I did the same for other companies over the past ten years, but when I backtested this, the results were a wash. (Also, this assumes you can identify a game-changing product before the rest of the investors can, which is nowhere near a safe assumption).

My second strategy was to invest in Fast Company's annual list of innovative companies. The idea is that innovation is the key to growth, but here again, the backtest over the past ten years came out a wash.

My third strategy was to invest in companies that appeared in the Great Place to Work Institute's Top 100 list (based on employee surveys). Not only does it have an interesting narrative logic, in the backtest, it beat the S&P 500 in around 20 of the previous 25 years.

The lesson is that given enough iterations, you will eventually find a strategy that backtests incredibly well against benchmarks. Likewise, the top-ranked mutual funds in any given year will have amazing historicals that show a fairy tale chart of inevitability. And yet, the only thing inevitable is that next year, there will be a different list of funds at the top, and the following year, a different list, and so on.

♦     ♦     ♦

If I calculated my odds of success, would it factor the odds that I could change those odds?

What happens in situations where an awareness of the odds affects the odds of an event? Do the odds settle on some equilibrium? Does this problem suggest the importance of faith, so that you can make your own odds?

For example, I once tried to calculate my odds of success on a project. But then I also tried to calculate the odds that I could change my odds of success. But then I thought, shouldn't that be included in the original odds of success? I could then apply these meta-odds over and over again, but eventually, it would settle down into some equilibrium odds. But then, couldn't I change those as well?

♦     ♦     ♦

If the odds of becoming president are 90%, does that include the over-confidence that that candidate would have knowing those odds?

Intrade and Betfair are part of a growing group of websites that lets you bet money on the probability of current events, like whether certain candidates get elected President. Apparently, their prediction market is the most accurate predictor, which leads to paradoxes while interpreting the data.

For example, during the 2008 presidential campaign, in the weeks leading up to Election Day, the odds were around 90% that Barack Obama would win. But did those odds include the effect that those odds would have on Obama's confidence? Even if Obama didn't read Intrade, there must have been some positive sentiment floating in the air, from an admixture of polling and political analysis, that could have adversely triggered his hubris. But would those risks be factored into those odds? And if that lowered the odds, would that then also affect his confidence?

♦     ♦     ♦

Since a human random number generator can't seem purposefully noisy, adding a few streaks would make it convincing

If you were trying to pretend to be a random number generator, what would you do? You couldn't start with a 1 or a 100: It's too coincidental for those extremes to appear. If you started a random sequence with, "1, 100, 50 ...," most likely people would think something was amiss.

At the same time, you don't want to throw out bland numbers in the middle, like 27 or 77. That might make people think you were creating a mélange for mélange's sakes. You'd need to mix in some small numbers and some large numbers, to show that your random number generator wasn't afraid of skirting with those untouchable extremes. So a 3 or a 92 would be appropriate now and then.

Variety is good, to some extent, but eventually, you'd have to throw in some expected unexpectedness. A surprise 1 or 100 would be good (but not both), and definitely some anomalous streaks, like a 33 followed by a 34.

A good random number generator has to alternate between piquing someone's superstitions and disappointing with pure, snowy noise.

♦     ♦     ♦

Someone who has a 50% chance of winning the election, after winning it, will have actually had a 100% chance in retrospect

The language of probability confounds our rhetoric. When we say that President Obama has a 90% chance of winning the election, the statistician's explanation is, "If the election were to be held today, in 9 out of those 10 times holding that election, it would lead to his victory." But in what universe do 10 hypothetical elections exist? The election that will lead to his victory is a singular event, and so what are we to make of these alternative elections?

Sure, a multi-verse theory of physics could describe an infinite number of hypothetical scenarios that exist somewhere. However, are we then saying that 90% of those alternative universes have Obama winning, and in 10% he is not?

Such thought experiments either sharpen our skills at "probabilistic thinking" or they point to actual un-addressed paradoxes in our understanding of the nature of randomness.

♦     ♦     ♦

Statistical Insignificance

The words "statistically significant" don't belong together, because there's nothing objective about significance. Something has statistical significance when it deviates from a normal distribution curve in a way that is rare. For most people "statistical" just translates into "scientific," leading the whole expression to mean, "scientists think it's important." If a sample of 30 men and 30 women show that men have 120 points and women have 125 points, the magazine article could say that the difference is statistically significant. Technically this means that there is likely a population difference between men and women, but to the average reader, it says Men Are from Mars, Women Are from Venus, leading to a new social order for a new generation.

♦     ♦     ♦

When playing slots, isn't it best to balance bet size with quantity, so that you can beat the Law of Averages without risking it all?

You have two strategies when playing slot machines. Starting with 100 credits, you could either bet everything on one pull, or you could bet 1 credit at a time. If you bet piecemeal, the Law of Large Numbers will reduce your winnings to the expected value of losing everything. But if you bet everything, the odds are high that your first and last pull will yield nothing, leaving you with the shortest game possible.

Isn't the best strategy somewhere in the middle? Place a reasonable bet, something small enough for a few attempts at the jackpot, but not so small, that your gains aren't balanced by the subsequent many losses.