The Mathematics of Diversification

I

What is variance?

Two friends both earn €1,000 per month on average. One gets exactly €1,000 every month. The other gets €2,000 in January, nothing in February, €2,000 in March, nothing in April. Same average. Completely different lives.

That spread — how far outcomes deviate from the mean — is variance. High variance means wild, unpredictable results. Low variance means smooth and predictable. Neither is inherently good or bad. What matters is whether you can survive the bad swings long enough for the good ones to arrive.

In investing, variance is the cost of chasing higher returns. The question isn’t whether you have variance — every investment does. The question is whether you can absorb it.

II

The Kelly Criterion — how much to bet when you have an edge

Suppose you know your probability of winning is higher than the market assumes. How much should you bet? Too little and you leave returns on the table. Too much and a bad streak wipes you out before the edge pays off.

Kelly gives the mathematically optimal answer:

If you win €2 for every €1 you risk, b = 2. If you have a 60% chance of winning (p = 0.6), Kelly says bet 40% of your capital — not 100%, not 10%.

The relationship below shows the optimal bet size across different win probabilities, for three levels of payout. Notice the flat zero region on the left: with even odds (b=1) Kelly says don’t bet at all unless your win rate exceeds 50%. Even with better payouts, there is always a threshold below which the correct bet is nothing.

“Diversification is protection against ignorance. If you know what you’re doing, it makes little sense.” — Warren Buffett

Kelly tells you to concentrate when your edge is real. Spreading your bets across lower-edge opportunities dilutes your geometric growth rate — you’re averaging your best idea with worse ones. The practical caveat: Kelly assumes your probability estimate is correct. If you’re wrong about your edge, you’re not just leaving money on the table — you’re overbetting into a losing position.

III

Geometric growth — why averages deceive

Here is the most important math in investing. You start with €1,000. Year one you gain 50%. Year two you lose 40%. The arithmetic average return is +5% per year. That sounds fine.

But what actually happened to your money?

After year one: €1,500. After year two: €900. You lost money. The arithmetic average of +5% is technically correct and completely useless. The geometric compound rate — what actually happened to €1 — is approximately −5% per year.

This gap between arithmetic and geometric returns is entirely caused by variance. A smooth 5% every year leaves you at €1,629 after a decade. Wild swings averaging 5% leave you at €590. Same average. €1,039 difference. Variance is expensive — even when the average return looks fine.

Kelly Criterion maximises the geometric growth rate — not the arithmetic one. That is what makes it the right tool: it accounts for the damage volatility does to compounding.

The deeper reason this matters has a name: ergodicity. A system is ergodic when the average outcome across many parallel players equals the outcome of one player over many rounds. Investing is not ergodic — you don’t live 10,000 parallel lives. You live one, sequentially, and a single wipeout year permanently reduces your base. This is why the time-path of returns (geometric) matters more than the statistical average (arithmetic).

Here is where the plot twists. Variance is the enemy of compounding — but it can be harvested. If you hold two volatile, uncorrelated assets and periodically rebalance back to a fixed allocation, the geometric growth rate of the portfolio can exceed either asset alone. This is sometimes called Shannon’s Demon, after Claude Shannon’s thought experiment. The rebalancing mechanically forces you to sell high and buy low. The wilder the swings and the lower the correlation, the larger the bonus. Diversification, done right, doesn’t just protect — it generates returns that neither component could achieve on its own.

IV

What diversification can and cannot do

Not all risk is created equal. Finance distinguishes two kinds. Unsystematic risk is specific to a single company or position — an earnings miss, a product recall, a CEO scandal. Systematic risk is market-wide — recessions, interest rate shifts, geopolitical shocks. Diversification can eliminate the first kind almost entirely. It can do nothing about the second.

This is the ceiling most investors miss. Adding positions reduces portfolio variance — but only up to a point. The curve flattens fast. Going from one stock to five eliminates a large share of company-specific risk. Going from twenty to fifty barely moves the needle. Beyond a certain threshold, you are just buying the market — and paying extra fees for the privilege.

The shape of this curve has practical consequences. If you hold fewer than five positions, adding one more does a lot. If you hold thirty, adding five more does almost nothing. The marginal benefit of diversification is front-loaded and decays rapidly.

What matters just as much is what you diversify across. Holding ten technology stocks is not meaningful diversification — they share the same sector risk, respond to the same interest rate environment, and crash together. True diversification means spreading across uncorrelated dimensions: asset classes (equities, bonds, commodities, real estate), geographies, strategy types (momentum vs. value vs. carry), and time (dollar-cost averaging). Five genuinely uncorrelated positions can reduce variance more than fifty correlated ones.

Diversification is not about the number of positions. It is about the number of independent risks.

There is also a point where diversification turns harmful. Peter Lynch called it diworsification — adding positions beyond your ability to understand and monitor them. Every new holding adds monitoring cost, tax complexity, and rebalancing friction. If you cannot articulate the thesis for each position, you are not diversifying — you are diluting conviction while adding overhead. The optimal portfolio sits between the extremes: enough positions to eliminate most unsystematic risk, few enough to maintain edge on each.

Finally, the deepest case for diversification is not about variance at all — it is about tail risk. Real markets have fat tails. Events that a normal distribution says should occur once every ten thousand years happen every decade. A concentrated portfolio can survive normal variance. It often cannot survive a six-sigma event in a single position. Diversification’s most important job is not smoothing average returns — it is ensuring that no single catastrophic outcome can permanently remove you from the game.

V

So — when does diversification make sense?

Diversification is not a virtue or a vice. It is a rational response to a specific set of conditions. The decision depends on two variables: how confident you are in your edge, and how much you can absorb a total loss.

The dangerous scenario most investors fall into: they believe they have high edge (quadrant 1) when they actually have low edge (quadrant 3). Overconfidence causes concentration into positions that aren’t actually as strong as assumed. Diversification in that case isn’t diluting edge — it’s protecting you from your own miscalibration.

One more variable matters: correlation. Diversification only reduces variance when your positions don’t move in lockstep. Owning ten tech stocks is not meaningfully diversified — they all respond to the same forces. Owning five uncorrelated positions can cut portfolio variance more than owning fifty correlated ones. The quality of diversification, not the quantity, is what determines whether spreading your bets actually buys you anything.