Sorting the Sources of Success

MY CAREER WAS LAUNCHED by a trash can.
Like many seniors in college, I wasn’t sure what I wanted
to do for a living, but I knew that I needed a job. Drexel Burnham
Lambert, an investment bank that was hot at the time, came on campus to recruit students for a new training program. My interview
went well enough that I was called to the firm’s headquarters in
New York City. I put on my best suit and tie, polished my shoes, and
headed to the Big Apple.
Early the next morning, we candidates gathered in a vast conference room and listened intently as the leader of the program told
us what to expect for the day. “You will have full interviews with
six members of our staff,” she informed us, “and then each of you
will have  ten minutes with the senior executive in charge of our
division.” When it was clear that she had everyone’s attention, she
added, “If you want the job, you’ll have to shine in that interview.”
My half dozen interviews went as well as could be expected. When
they were over, a member of the staff led me down a long corridor to
an office paneled in dark wood, with deep wall-to-wall carpeting and
a picture window overlooking a panorama of downtown Manhattan.
Introduction.indd
A sharp-eyed administrative assistant ushered me in, and the senior
executive greeted me warmly. Then I saw it.
Peeking out from underneath a huge desk was a trash can bearing
the logo of the Washington Redskins, a professional football team.
As a sports fan who had just spent four years in Washington, D.C.,
and had attended a game or two, I complimented the executive on
his taste in trash cans. He beamed, and that led to a ten-minute interview that stretched to fifteen minutes, during which I listened and
nodded intently as he talked about sports, his time in Washington,
and the virtues of athletics. His response to my opening was purely
emotional. Our discussion was not intellectual. It was about a
shared passion.
I got the job. My experience in the training program at Drexel
Burnham was critical in setting the trajectory of my career. But
after a few months in the program, one of the leaders couldn’t resist
pulling me aside. “Just to let you know,” he whispered, “on balance,
the six interviewers voted against hiring you.” I was stunned. How
could I have gotten the job? He went on: “But the head guy overrode
their assessment and insisted we bring you in. I don’t know what
you said to him, but it sure worked.” My career was launched by
a trash can. That was pure luck, and I wouldn’t be writing this if
I hadn’t benefited from it.
The Boundaries of Skill and Luck
Much of what we experience in life results from a combination of
skill and luck. A basketball player’s shot before the final buzzer
bounces out of the basket and his team loses the national championship. A pharmaceutical company develops a drug for hypertension that ends up as a blockbuster seller for erectile dysfunction.
An investor earns a windfall when he buys the stock of a company
shortly before it gets acquired at a premium. Different levels of skill
and of good and bad luck are the realities that shape our lives. And
yet we aren’t very good at distinguishing the two.
Part of the reason is that few of us are well versed in statistics.
But psychology exerts the most profound influence on our failure to
identify what is due to skill and what is just luck. The mechanisms
that our minds use to make sense of the world are not well suited
to accounting for the relative roles that skill and luck play in the
events we see taking shape around us. Let me start with some examples that are clearly controlled by either luck or skill.
The drawing for Powerball, a multistate lottery, went off uneventfully on the evening of Wednesday, March 30, 2005. The first five
balls came through the clear tubes: 28, 39, 22, 32, 33. The final ball,
which came from a separate machine, clicked into place: 42. The
whole process took less than a minute.
Sue Dooley, the staff member who was overseeing the drawing
that night, rolled the machines back into the vault and drove from
the television studio to the Powerball headquarters five miles away.
Based on the statistics, she expected that perhaps one ticket would
take home that day’s jackpot of $84 million and that three or four
people would have picked five of the six numbers correctly, winning
second place.
She turned on her computer and waited for the states to report
their results. The trickle of winners she had expected was actually a
torrent. In total, there were 110 second-place winners. The statisticians employed by Powerball had warned that six or seven times the
predicted figure was well within the realm of chance, but an outcome
nearly thirty times the expectation appeared statistically impossible. Another oddity was that nearly all of the winning tickets had
the same sixth number, 40. Truth be told, the officials at Powerball
would have preferred it if the winners had picked all six numbers
correctly, because the jackpot is split evenly among them. No matter how many people win, it costs Powerball the same amount. But
each winner of the second prize receives a set amount, which meant
that in this case, Powerball had to pay out $19 million more than it
had anticipated.
Dooley called her boss and together they puzzled over possible
explanations, including numbers shown on television, pattern play
lottery columns, and even fraud. None of them checked out. The next
morning, they got their first inkling of what had happened. When
a staff member at a prize office in Tennessee asked a winner where
he had gotten his numbers, he answered, “From a fortune cookie.”
Later a winner in Idaho said the same thing, and shortly thereafter
winners in Minnesota and Wisconsin echoed the reply. Jennifer 8.
Lee, a reporter from the New York Times, jumped on the story and
traced the fortune cookies with the winning numbers back to the
factory of Wonton Food in Long Island City, New York.  Derrick
Wong, a vice president at the company, explained that they had
put numbers in a bowl and randomly picked out six of them. Since
generating the number sequences takes time, the company printed
the same numbers on different fortunes so as to save labor on the
4 million cookies the factory produced each day.
1
 Each of those very
lucky winners took home between $100,000 and $500,000, according
to how much they had bet.
Marion Tinsley won a lot, too, but it wasn’t because he was lucky.
Tinsley was known as the greatest player of checkers (also known as
draughts) in the world. In 1948 he was crowned as the United States
champion; shortly before his death in 1994, he tied Don Lafferty
and a computer program named Chinook for first place. In the intervening forty-five years, Tinsley lost only seven individual games
for a near-perfect record. In two of those games he was defeated by
Chinook. Despite the fact that he didn’t play for long periods of time
(he was a professor of mathematics at Florida State and Florida
A&M Universities), he reigned as world champion in three separate
decades.
2
Tinsley’s success resulted from years of deliberate practice. In his
youth, Tinsley spent eight hours a day, five days a week, studying
checkers, and he continued to study the game, though less intensely,
throughout his life. He cultivated a prodigious memory that allowed
him to recall the flow of games he had played decades earlier. Tinsley
was fiercely competitive and claimed that he could beat all comers,
man or machine, as long as his health didn’t fail
Both those who won the Powerball lottery and Marion Tinsley
enjoyed great success. But it’s easy to see that the causes of the two
types of success differed markedly. The lottery outcome that day
was a matter of pure good luck for the 110 winners and pure bad luck
for Powerball. But Tinsley’s success was almost entirely the result
of skill. With all the luck in the world, you would have almost no
chance of winning if Tinsley were across the table from you. For
practical purposes, we can regard Tinsley’s success as all skill.
Unfortunately, most things in life and business are not that clear.
Most of the successes and failures we see are a combination of skill
and luck that can prove maddeningly difficult to tease apart.
The purpose of this book is to show you how you can understand
the relative contributions of skill and luck and how to use that
understanding in interpreting past results as well as making better
decisions in the future. Ultimately, untangling skill and luck helps
with the challenging task of prediction, and better predictions lead
to greater success.
Skill, Luck, and Prediction
Shortly after winning the Nobel Prize in Economics in 2002, Daniel
Kahneman, a retired professor of psychology at Princeton, was asked
which of his 130-plus academic papers was his all-time favorite.
4
 He
chose “On the Psychology of Prediction,” a paper he cowrote with
the late Amos Tversky that was published in Psychological Review in
1973. The paper argues that intuitive judgments are often unreliable
because people base predictions on how well an event seems to fit a
story. They fail to consider either how reliable the story is or what
happened before in similar situations. More formally,  Kahneman
and Tversky argue that three types of information are relevant to
statistical prediction. The first is prior information, or the base rate.
For example, if 85 percent of the taxicabs in a city are green, then
85 percent is the base rate. Absent any other information, you can
assume that whenever you see a taxicab there’s an 85 percent chance
that it will be green. The second type of information is the specific
evidence about an individual case. The third type of information is
the expected accuracy of the prediction, or how precise you expect it
to be given the information you have.
5
I had a conversation with a doctor that illustrates these three
types of information. He mentioned that he had a treatment for
improving a specific ailment that succeeded about 50 percent of the
time (the base rate). But he added that he could induce almost any
patient to undergo the treatment if he simply told them, “The last
patient who was treated this way is doing great!” (specific evidence
about an individual case). For the patients who were evaluating the
treatment, the story of success swamped the statistics.
The key to statistical prediction is to figure out how much
weight you should assign to the base rate and specific case. If the
expected accuracy of the prediction is low, you should place most
of the weight on the base rate. If the expected accuracy is high,
you can rely more on the specific case. In this example, the doctor
gave the patient no reason to believe that the procedure had better than a 50/50 chance of working for him. So the patient should
place almost no weight on the specific evidence that it worked for
one patient, and should rely instead on the base rate in making his
decision.
Here’s how the weighting of the base rate and the specific case
relate to skill and luck. When skill plays the prime role in determining what happens, you can rely on specific evidence. If you’re
playing checkers against Marion Tinsley, you can easily predict
the winner on the basis of your knowledge of Tinsley’s deadly skill.
In activities where luck is more important, the base rate should
guide your prediction. If you see someone win a million dollars, that
doesn’t change the odds of winning the lottery. Just because someone wins at roulette, it doesn’t help you to guess where the ball will
end up on the next spin.
Unfortunately, we don’t usually think this way. When we make
predictions, we often fail to recognize the existence of luck, and as a
consequence we dwell too much on the specific evidence, especially
recent evidence. This also makes it tougher to judge performance.
Once something has happened, our natural inclination is to come up
with a cause to explain the effect. The problem is that we commonly
twist, distort, or ignore the role that luck plays in our successes and
failures. Thinking explicitly about how luck influences our lives can
help offset that cognitive bias.
Quantifying Luck’s Role in the Success Equation
The starting place for this book is to go beyond grasping the general idea that luck is important. Then we can begin to figure out the
extent to which luck contributes to our achievements, successes,
and failures. The ultimate goal is to determine how to deal with luck
in making decisions.
This book has three parts:
• Chapters 1 through 3 set up the foundation. I start with some
working definitions of skill and luck, examining the types
of interactions where luck is relevant and noting where our
methods to sort skill and luck may not work. I then turn to
why we have such a difficult time comprehending the influence
that luck exerts. The basic challenge is that we love stories
and have a yearning to understand the relationship between
cause and effect. As a result, statistical reasoning is hard, and
we start to view the past as something that was inevitable.
The section finishes by looking at the continuum from all-luck
to all-skill. I examine a basic model to help guide intuition.
These ideas include the paradox of skill and what determines
the rate of reversion to the mean.
• Chapters 4 through 7 develop the analytical tools necessary
to understand luck and skill. I open with methods for placi
activities on the luck-skill continuum. Where an activity
falls on that continuum provides a great deal of insight into
how to deal with it. I then look at how skill changes over time.
Simply put, skill tends to follow an arc: it improves for some
time, peaks, and then glides lower. Next, I turn attention
to the distributions—or the range of values—of luck. In
activities where the results are independent of one another,
simple models effectively explain what we see. But when a
past  result affects a future result, predicting winners becomes
very difficult. The most skillful don’t always win. I close this
part by showing the difference between a useless statistic
and a useful one. Useful statistics are persistent (the past
correlates highly with the present) and predictive (doing well
or poorly correlates strongly with the desired goal). As we will
see, many statistics fail this basic test.
• Chapters 8 through 11 offer concrete suggestions about
how to take the findings from the first two parts of this book
and put them to work. I begin by outlining ways to improve
skill. Where little luck is involved, deliberate practice is
essential to developing skill. Where luck is rampant, we
must think of skill in terms of a process, because the results
don’t provide clear feedback. Checklists can also be of
great value because they improve execution and can guide
behavior under stressful circumstances. I then look at how
to cope with luck. When you are the favorite, for example,
you want to simplify the game so that you can overwhelm
your opponent. If you are the underdog, you want to inject
luck by making the game more complex. Because luck is in
part what remains unexplained, controlled tests allow for a
more accurate reading on causality. If you want to know if
an advertisement worked, for example, you need to consider
the purchasing behavior of those who saw the ad versus those
who didn’t. This part also includes an in-depth discussion of
reversion to the mean, an idea that most people believe they

understand, even though their behavior shows that they don’t.
The book finishes with ten concrete tips on how to overcome
the psychological, analytical, and procedural barriers in
untangling skill and luck.
This analysis of skill and luck will focus on business, sports, and
investing because these are the areas I know best. Naturally, these
realms are quite different. Sports are the easiest activities to analyze because the rules are relatively stable over time and there is
lots of data. Other social processes, including business, have fewer
rules and boundaries than sports and therefore tend to be more
complex. Still, many of the same analytical methods are valid.
6
Markets in general are the most difficult to analyze because prices
are established through the interaction of a large number of individuals. Here again, the nature of the problem may be somewhat
different from sports, but many of the tools for sorting out the
relative influence of skill and luck still apply.
Part of the fun and challenge of analyzing skill and luck is that
it’s a multidisciplinary endeavor. Statisticians, philosophers, psychologists, sociologists, corporate strategists, professors of finance,
economists, and sabermetricians (those who apply statistical methods to the study of sports) all have something to contribute to the
discussion.
7
 Unfortunately, the people within these disciplines don’t
always reach outside their fields. You will see ideas from each of
these disciplines, and I’m hopeful that bringing them together will
lead to a sounder and more balanced approach to analyzing decisions
and interpreting the results.
Untangling skill and luck is an inherently tricky exercise, and there
are plenty of limitations, including the quality of the data, the sizes
of samples, and the fluidity of the activities under study. The argument here is not that you can precisely measure the contributions
of skill and luck to any success or failure. But if you take concrete
steps toward attempting to measure those relative contributions,
you will make better decisions than people who think improperly
about those issues or who don’t think about them at all. That will
give you an enormous advantage over them. Some statisticians,
especially in the world of sports, come across as know-it-alls who are
out of touch with the human side of things. This characterization is
unfair. Statisticians who are serious about their craft are acutely
aware of the limitations of analysis. Knowing what you can know
and knowing what you can’t know are both essential ingredients
of deciding well. Not everything that matters can be measured, and
not everything that can be measured matters.
While there are wide swaths of human activity where the ideas
in this book are hard to apply, the ideas have concrete application in some important areas and should serve as a template for
thinking about decisions beyond the scope of this book. Luck may
explain that you met your future wife after your buddy lured you
out on a Thursday night, but this book will have little to directly say
about that or other issues of love, health, and happiness. We need to
define the activity we’re talking about and what measures we need
to use to evaluate that activity effectively.
In his book The Theory of Gambling and Statistical Logic, Richard
Epstein, a game theorist trained in physics, notes that there is no
way to assure that you’ll succeed if you participate in an activity
that combines skill and luck. But he does say, “It is gratifying
to rationalize that we would rather lose intelligently than win
ignorantly.”
8
 Luck may or may not smile on us, but if we stick to a
good process for making decisions, then we can learn to accept the
outcomes of our decisions with equanimity