The Five Year Eureka Moment
Daniel Kahneman met Amos Tversky in 1969 when Tversky came to Hebrew University to give a talk.
As Kahneman recalls in his 2011 intellectual biography, Thinking, Fast and Slow, the two researchers hit it off and decided to pursue a joint project: figuring out if some people had more of an intuitive grasp of statistics than others.
They discovered that the answer, universally, was a resounding “no.”
“Our expert colleagues…greatly exaggerated the likelihood that the original result of an experiment would be successfully replicated,” Kahneman recalls of their results. “They also gave poor advice to a fictitious graduate student about the number of observations she needed to collect.”
“Even statisticians are not good intuitive statisticians,” he concluded.
This small observation led to a bigger idea: perhaps humans are hardwired with cognitive shortcuts to help them make sense of an uncertain world, and perhaps these shortcuts, in certain situations, consistently lead to irrational conclusions.
This hypothesis was profound. At the time, social science believed that humans were fundamentally rational, and only emotion, like fear or anger, could lead us to irrational behavior. Kahneman and Tversky were proposing that humans, on the contrary, were wired for illogic.
To support this view, they gathered over twenty different examples of cognitive shortcuts consistently leading to irrational conclusions. They combined the results into a paper titled “Judgment Under Uncertainty: Heuristics and Biases.”
They published the paper in the journal Science where it has since become one of the most important studies in all of social science. (According to Google Scholar, it’s been cited over 13,500 times since its publication.) The paper formed the foundation for the field of behavioral economics, which won Kahneman the Nobel Prize in 2002 (Tversky had died seven years earlier).
Here’s what caught my attention about this story. This paper — Kahneman and Tversky’s first publication on their theory — came out in 1974, a half decade after they first began pursuing the underlying ideas. In other words, it took them a full five years to refine a rough hunch, through systematic exploration and discussion, into an idea too good to be ignored.
They were, in short, diligent.
The reason I’m telling you about Kahneman and Tversky, however, is that I’m convinced that there must be more to the story…
The Insufficiency of Diligence
Here’s another story of research diligence.
In 2007, as a third year graduate student at MIT, I was studying the application of distributed algorithm theory to the setting of wireless networks. Around this time, my collaborators and I came up with a model for these algorithms that we thought captured something important about wireless communication.
We ended up publishing the following series of peer-reviewed research papers that explored the mathematical limits of this model:
- Gossiping in a Multi-Channel Radio Network: An Oblivious Approach to Coping with Malicious Interference
by Shlomi Dolev, Seth Gilbert, Rachid Guerraoui, and Calvin Newport
Proceedings of the International Symposium on Distributed Computing (DISC). September, 2007.
- Secure Communication Over Radio Channels
by Shlomi Dolev, Seth Gilbert, Rachid Guerraoui and Calvin Newport
Proceedings of the ACM Symposium on the Principles of Distributed Computing (PODC). August, 2008.
- Interference-Resilient Information Exchange
by Seth Gilbert, Rachid Guerraoui, Darek Kowalski, and Calvin Newport
Proceedings of the IEEE Conference on Computer Communications (INFOCOM). April, 2009
- The Wireless Synchronization Problem
by Shlomi Dolev, Seth Gilbert, Rachid Guerraoui, Fabian Kuhn and Calvin Newport
Proceedings of the ACM Symposium on the Principles of Distributed Computing (PODC). August, 2009.
- Leveraging Channel Diversity to Gain Efficiency and Robustness for Wireless Broadcast
by Shlomi Dolev, Seth Gilbert, Majid Khabbazian and Calvin Newport
Proceedings of the International Symposium on Distributed Computing (DISC). September, 2011.
As I write this, we’re currently preparing a new paper on this topic for publication.
Notice: 2007 to 2012 is five years. This is exactly the time it took Kahneman and Tversky to develop their career-defining mega idea, and yet I’m not holding my breath for a call from Stockholm.
(To be fair, this research direction is solid. These papers were all published in high quality, competitive venues, and combined, they have been cited over 100 times. But they’re not the type of results that make a researcher famous.)
Both my team and Kahneman’s team were equally diligent, but one obtained more remarkable results than the other. My question, then, is simple but important: why the difference?
My working answer to my simple question is that there’s a key subtlety in leveraging diligence to achieve remarkable results:
The Directed Diligence Theory
It’s not enough to just focus relentlessly on a small number of things (though this is almost always necessary). You must also direct this diligence by simultaneously and systematically exposing yourself to the reality of what’s valuable in the relevant field.
Kahneman and Tversky’s diligence, for example, was directed by their understanding, as psychology professors, that the model they were pursuing was a radical departure from an orthodoxy that had started to show strain. The field was looking for new models and they knew they were on to one possibility.
In my last post, I offered Steve Martin as another example of diligence breeding remarkability. When you read his memoir, you find a similar direction to his focus. Martin studied comedy like an academic anthropologist, picking apart what was doing well and what was becoming dated. His deep understanding of the evolution of comedy in the 1970s directed his diligence toward real results.
Returning to my own example, it was only a few years ago that I began to internalize this lesson. Just because an idea was interesting to me, I now accepted, was not enough by itself to justify diligent pursuit.
So I made a change to my research method…
Notice in my list above of publications on this topic, there is a two year gap between 2009 and 2011. What happened in these years? I left my theory group to become a postdoc in a systems group that focused on making real world wireless networks better.
This was not an easy transition for a theoretician. I had spent the previous five years working primarily on whiteboards, proving theorems. My first day in the systems group, by contrast, I found that someone had left a toolbox on my desk. A toolbox!
This was a different world.
But here’s the thing: I learned a lot about how real wireless networks work and what the people who build them actually worry about. Since that experience, and my continued extensive interaction with systems researchers, I’ve noticed my diligent work on wireless network theory has begun to drift toward increasingly interesting shores. In a grant application I submitted this past fall, for example, I was able to detail a trio of serious problems from real wireless networks that my style of theory now has the potential of seriously solving.
It might take another five years before I’ll know if this new experiment in directed diligence pans out. But it already feels right.
Remarkable accomplishment requires a remarkable amount of focus; this much is clear. But focus without grounded direction is unlikely to hit the sweet spot.
The key observation, however, is that this directed diligence approach is not about figuring out in advance what you were meant to do or identifying a can’t miss idea. It’s instead about coupling your diligence with continued exposure to what real value looks like. You won’t start out knowing exactly where your story is heading, but you can have confidence that you’ll end up with the right sort of ending.
(Photo by moriza)
This post is part of my series on the diligence hypothesis, which proposes that focusing relentlessly on a small number of things for a large amount of time is a key strategy for crafting a remarkable life. Previous posts on this topic include:
- Closing Your Interests Opens More Interesting Opportunities: The Power of Diligence in Creating a Remarkable Life
See also my related series on the deliberate practice hypothesis.
19 thoughts on “Learn the Landscape Before Putting on Blinders: How to Direct Diligence Toward Remarkable Results”
The parallels between this post and the entrepreneurial idea of first finding a customer’s problem, then solving it (as opposed to first building something, then trying to find a customer) are amazing.
Yes to Satvik’s idea. Having diligence mixed with finding a real world application for it.
When I first started reading this post, I thought you were going to relate K & T’s work to explain why most people don’t succeed- they have illogical ideas about what it takes (Follow your passion, Personality tests…). And then the post was going to be about trusting the evidence (focus on deliberate practice, hard focus, do a small number of projects) vs trusting your gut (doing an insane amount of things, the need to be stressed out).
I think your blog is kind of like a behavioral economics view of success in college and the working world. The only difference is that you dont just look at the problems, you give us amazing solutions.
I’ve found this to be the case in undergraduate work as well. If you’re given a somewhat open-ended assignment, like a paper or presentation, in a class, you want to hit on an important, contentious contemporary topic. But the catch is that finding this topic requires research and intuition.
I definitely think that what you have thought of has merit.
Already I am sort of experiencing the uncertainty in finding the sort of project that will really help me reveal some sort of insight. I am doing research right now, but I know that eventually I will probably be looking for other studies as well…we’ll just have to wait and see…
Actually, something else came to mind when I read this. I was talking the other day with someone about how even though you may have refined skills, that might not even be good enough to get you to where you want to be. Such has sort of convinced me that even though you might not get to where you want to be immediately, you will at least get somewhere where people will find your skills valuable and will be willing to use (and pay) for them.
Your post also reminded me of the post you wrote about “abandoning your big dream” because new challenges arise every day, so obviously there will be demand for people who can take these challenges. What these challenges are, we don’t know for certain yet, but they provide opportunities for people with the abilities and skills to solve them, which can in turn provide them with something to talk about!
It’s very similar to the Lean Startup Approach (which is what I believe Satvik was alluding to) — start with an idea, come up with a minimum viable product, test and receive feedback from users/customers, and finally refine. The challenge, I think, is to know when to delve horizontally into other related fields.
The part of the theory I most appreciate is this:
“… simultaneously and systematically exposing yourself to the reality of what’s valuable in the relevant field.”
It’s so important for us to form habits of removing the blinders at regular, predetermined intervals to check for opportunities in our field. Otherwise, it’s easy to be distracted by one sensational development that snags our attention, and miss subtle (and perhaps more valuable) opportunities.
Another great post.
I think what is really hard is finding out what things really matter and have a real world application. Something that you or I might think is important might not seem that way for other people.
I think when you were publishing your papers, you were thinking that they might have a great impact on the scientific world, in contrast to what you are thinking now?
Nice post. I’m particularly interested in your motivation for shifting focus (groups).
Could you please elucidate on the key differences between the theory group and the systems group?
How did your background in theory manifest itself when you were working on problems in the systems group?
Was the 2011 publication a combination of your new systems knowledge with your previous research? (the authors throughout 2007-11 are mostly the same).
Did these researchers really discover that people are irrational, at least in certain situations? My suspicion is that human decision making is based primarily on experience, much of it collected by our unconscious, rather than reason or unreason. They asked the wrong question.
Bill Clinton is an example. My experience is that he’s a pathological liar, lying to cover up his misdeeds or to get something from others. My rational side recognizes that sometimes Clinton tells the truth. If I were on an elevator with him and asked the time, he’d probably give me the right time. My irrational side, however, wants to punish him and perhaps avoid be bamboozled by him, so it inclines toward never believing a word he says.
That’s why I’d suggest that, as with Bill Clinton, so with life, we make decisions based on experience rather than reason or unreason. We don’t disbelieve everything a Clinton says, we merely hold it in doubt, waiting for better evidence. And in the vast majority of life situations, that’s probably the better choice that reasoning from often inadequate data or just doing something.
For an example, imagine a fisherman in the woods who sees a bear not far away. It’s not reason that leads him to instinctively hunker down behind a bush. He’s doing that before a thought has time to pass through his mind. No is there any real chance that he’ll do something irrational like run toward the bear screaming loudly. He’s driven by neither reason nor unreason. He’s doing instinctively something that his experience, including that of shared tales, has told him is best.
Could there be any LUCK involved in these processes of discovery?
Is it possible that you are falling into the very fallacy of creating theories using limited information, and backward narration that Kahneman, and others have identified?
One of my major disagreements with how you write, a little too much narrative fallacy. Undeniably, when a large number of people all apply themselves diligently a very small (disproportionately small) Number of them will find breakthroughs, and provide sparkling stories for your blog. Then you can use those specific possibly luck influenced successes to create a general theory of success.
You’ve forgotten about the graveyard of people who have relentlessly pursued skills only to find out that they were less useful and in demand than they thought they would be, and wound up with no significant career advantage.
But I think your right, leaving yourself open to opportunity is certainly tied to success in any field.Realizing that YOU DONT REALLY KNOW what results your theories might have. is an important part of the equation.
But I am a big fan, and I have used your study tactics to improve my grades! I hope this does not come off as hater-ish thank you!!!!
I also think that your negative advice articles advising students and employees to not “make the leap” into the void of entrepreneurship are spot on. I want to see more “dont do X because you dont know” style posts! rather than “i have a theory!” posts 🙂
The question that keeps bothering me about this is: How do you know which opportunity to “leap” to? I’m trying to follow the principle of hard focus in directing my research, but do I know which outside, or slightly unrelated, opportunities would be beneficial and which would just be halting progress on the area of my focus?
“Thinking, Fast and Slow”, have anyone read this book, any comment on it?
I do not think for a moment that contribution in the theoretical world means less than the “practical world”. It would be a lot easier to summarise your entry as “how to focus hard to get famous and recognised”. Am I right to interpret your entry in this way?
I think what’s going on here is actually more subtle. It’s not about finding a problem then going after a solution. It’s more integrated. You’re working diligently on something while continuing to expose yourself to what is proving valuable in your field. This provides feedback to your diligent pursuit which in turn focuses what you expose yourself to.
In other words, it’s similar to the entrepreneurial model of an agile start-up. You don’t have all the answers when you start, but you keep constantly adjusting and constantly pushing forward on what is proving to be important.
It was starting to cross that boundary more, at least, in terms of the terminology we used.
Their Nobel Prize-winning research, conducted over decades, is probably resistant to such easy dismissal. Kahneman’s book is good place to dive into the nuances of what they were up to. I’m not doing it justice in this short post.
The point is that you don’t. It’s the combination of diligence and the continual reshaping of the effort that comes from exposure to the field, that leads to valuable places. If you could map it out in advance, everyone would be making major breakthroughs.
Kahneman himself says repeatedly in his book, Thinking, Fast and Slow, that luck plays a tremendous role in making very successful people just that. Our conscious minds weave a story of how we have pulled ourselves up by our bootstraps but it’s only a story.
If you don’t win a Nobel Prize, Cal, then maybe there is no identifiable formula for accomplishing everything you want in life.
Hello Professor Newport. I am COMPLETELY on board with your postulates about passion. Like happiness, it ensues. But I’m a lot more bearish on your views of the state of Flow. The reason I bring up Flow is because this post reminds me of Csikszentmihalyi. It’s probably the case that you’ve read Mike’s book “Creativity.” Is that correct? I will admit that it’s been a while since I’ve reread it. But, if memory serves me, your Directed Diligence Theory sounds a lot like Csikszentmihalyi’s book. How say you?