You and Your Research
In March 1986, the famed mathematician and computer scientist Richard Hamming returned to his former employer, Bell Labs, to give a talk at the Bell Communications Research Colloquia Series. His talk was titled “You and Your Research,” and it’s goal was straightforward: to deliver lessons for serious researchers about how to do “Nobel-Prize type of work” (a topic familiar to Hamming given the large number of Nobels won by his colleagues during his Bell Labs tenure).
This talk is famous among applied mathematicians and computer scientists because of its relentlessly honest and detailed dissection of how stars in these fields become stars — a designation that certainly applies to Hamming, who not only won the Turing Prize for his work on coding theory, but ended up with an IEEE prize named after him: the Richard W Hamming Medal.
A problem with his talk, however, is it’s length and density. It’s easy to lose yourself in its transcript, nodding your head again and again in agreement, then coming out the other side unable to keep track of all the ideas Hamming outlined.
My goal in this blog post to help bring some order to this state of affairs. Below I’ve summarized what I find to be the major points from Hamming’s address. To identify the sections of the speech that correspond to each point I use the wording from this transcription.
I can’t claim that the following is comprehensive (among other things, I do not annotate the questions after the talk), but I’m confident that I capture most of what’s important in this seminal seminar.
Idea #1: Luck is not as important as people think.
[location: see the section that starts with the sentence “let me start not logically, but psychologically…”]
Hamming notes that luck is a common explanation for doing great research. He doubts this explanation by noting that great researchers — like Einstein — do multiple good things in their career.
As an alternate explanation, he cites the following Newton quote: “If others would think as hard as I did, then they would get similar results.”
Idea #2: Knowledge and productivity are like compound interest
[location: “Now for the matter of drive.”]
Hamming notes that “most great scientists have tremendous drive.” He emphasizes the value of working hard by noting that the benefits of extra effort compound over time like interest. If you’re reading a little bit more and thinking a little bit longer than your colleagues, over time the gap between you and them grows.
He then supplies, however, a crucial caveat: you have to apply this effort intelligently. “That’s the trouble,” he elaborates, “drive, misapplied, doesn’t get you anywhere.” He illustrates this axiom by discussing former colleagues who worked harder than he did but have nothing to show for it.
Idea #3: Become comfortable with ambiguity
[location: “there’s another trait on the side which I want to talk about…”]
Hamming notes that great scientists form an “emotional commitment” to the theories they pursue and are able to carry on despite doubts and identified flaws.
He adds, however, that they are still careful about tracking and later addressing these shortcomings that arise along the way.
Idea #4: Creativity requires focus
[location: “Now again, emotional commitment is not enough…”]
Hamming notes his belief in the theory that “creativity comes out of your subconscious.” To make a breakthrough, he believes, you must focus without distraction on the problem long enough for your subconscious to start working it over.
As he summarizes: “If you’re deeply immersed and committed to a topic, day after day after day, your subconscious has nothing to do but work on your problem.”
Idea #5: Important work comes from important problems
[location: “Now Alan Chynoweth mention that I used to eat…“]
After talking about his efforts to seek out big thinkers in the Bell Labs cafeteria, Hamming emphasizes: “If you do not work on an important problem, it’s unlikely you’ll do important work.” He notes that the “average scientist” does not work on important problems but instead plays it safe. He wanted to avoid such mediocrity in his interactions.
He clarifies that for a problem to be important, not only most its impact be clear, but you must also possess a “reasonable attack.” He estimates that most great scientists will keep 10 to 20 great problems in mind at any one time, and if they learn a technique that might apply to one of them, they’ll “drop all the other things and get after it.”
Hamming recalls that he blocked off his Friday afternoons as “Great Thoughts Time,” and spent them only discussing and thinking about “great thoughts.”
Idea #6: Keep your door open
[location: “Another trait, it took me a while…”]
Hamming notes that at Bell Labs the scientists who kept their doors shut got more done in the short term. By contrast, the scientists who kept their doors open were distracted more often but ultimately this increased exposure to other people, problems, and ideas led them to more important work in the long term.
Idea #7: Transform isolated problems into general problems
[location: “I want to talk on another topic…”]
Hamming notes that early on he used to work on isolated problems. This depressed him: “I could see life being a long sequence of one problem after another after another.”
This changed his approach to focus on bigger, more general ideas that others could build on. “By changing a problem slightly” he clarifies, “you can often do great work rather than merely good work.”
He goes on to add that you should never work on an isolated problem unless it is “characteristic of a class.”
Ideas #8: Sell you results
[location: “I have now come down to a topic which is…”]
Hamming notes that the idea of marketing your results is “distasteful” and “awkward” to discuss. He also notes, however, that it’s necessary because “the fact is everyone is busy with their own work,” and you cannot expect them to discover your work on their own.
He suggests three things you must do to bring you ideas to other peoples’ attention: (a) “learn to write clearly,” (b) “learn to give reasonably formal talks,” and (c) “learn to give informal talks” (and know which type of talk is appropriate and when.)
He emphasizes that when giving a talk, in most venues, “most of the time the audience wants a broad general talk and wants much more survey and background than the speaker is willing to give. As a result, many talks are ineffective.”
In this regard, a very good book i have read in the past is “The Mathematician’s Mind:
The Psychology of Invention in the Mathematical Field”
Lecture by Richard Hamming on same topic you mentioned in your blog.
https://www.youtube.com/watch?v=a1zDuOPkMSw
Excellent. It’s exciting to see a video version of Hamming talking on the subject…
Very Inspiring , thanks
How did this switch from talking about Hamming to Shannon? That confused me.
+1. How did it go from Hamming to Shannon?
I often mix up those two names (both worked on similar theories). I made the fix in the post.
Keep your door open?
Doesn’t that relate back to earlier discussions about the need to be accessible by email if we put it in a more modern context?
This seems to imply that such accessibility is more important than working in focused blocks of time.
I believe there needs to be a balance between the two. But further, one needs to control one’s incoming communications so as to filter out the noise while preserving valuable input and collaboration from colleagues.
Hamming is certainly firm on this idea that your door should be open. However, I think the level of distraction he’s talking about here is orders of magnitudes smaller than what we encounter in an age of networked computers. In Bell Labs, in the 60s and 70s, to have your door opened might mean that a really smart colleague would drop in, on average, once or twice a day, and in doing so probably trigger your thinking about something interesting. I think deep work was easily supported under such a condition.
Universities and research institutes typically have a lot of smart researchers confined within the same building. The researchers have external visitors on a regular basis and it’s good for researchers to know about the important open problems and general research directions of related research fields. This is the typical benefit of hallway discussions that you are directly included in or can join if you keep your door open. Non-typical benefits are opportunities for scientific collaborations with strangers that complement your skill set.
The first reason these kind of hallway discussions don’t take place over email is because you don’t know who you should send the email to in the first place. The second thing is that it’s an awkward medium for the kind of discussions that typically happen around a whiteboard. The third reason is that it would take forever to establish common ground with someone outside your field through an email exchange. It can definitely happen in theory but it doesn’t seem to happen in practice.
Regarding in-house conversations: the barrier to knock on someone else’s door is a matter of minutes, but it exists. If I think for another 3 minutes I usually solve the problem on my own without disturbing anyone else. The fact that I can solve the problem in 3 minutes means the problem is trivial and therefore uninteresting to others.
Unless I plan to waste peoples time or make a fool out of myself I need to think about how to present the problem before leaving my office. Having thought about the problem and about how to present it maximizes my chances of having a useful conversation about the problem with someone else. They might already know about the problem I’m working on under a different name in their field, or have ideas or tools available for attacking the problem.
The benefit for the helper varies. The cost is comparatively small and if they get a paper out of it their time was well spent. The immediate benefit for them is more knowledge which helps them in solving other problems. More long term benefit is reputation as a skilled helpful person which will provide opportunities to solve other problems. For this later part to pan out you need to be the expert among the experts though.
Good point. My challenge is how to relate these ideas on deep work vs. availablilty/distraction to a corporate environment as opposed to a research environment. And it seems to be true that the most valuable collaboration does not occur via email in either environment. However, an examination of my email traffic shows that there is certainly an exchange of necessary information to the operation of the business that is better served by that medium of communcation than by a personal visit. That said, there is no reason such communications cannot wait a few hours for me to surface from a spell of deep work. This all goes well if I have colleagues who respect my time and vice versa.
How does Idea #6, Keep your door open, agree with some of the thinking around “deep work”?
Thank you for posting this and, more importantly, for parsing through Richard Hamming’s talk.
What Newton said reminded of this quote from Johann Sebastian Bach:
“I worked hard. Anyone who works as hard as I did can achieve the same results.”
This should mean a lot, coming from history’s greatest composer.
Hi Cal,
I was wondering how you can be sure that you are investing your ‘drive’ intelligently (idea #2)? Are there any other methods despite looking for important (#5), general (#7) problems?
All the best,
Jakob
“He estimates that most great scientists will keep 10 to 20 great problems in mind at any one time” this surprises me… it is much more than I thought. I seem to max out at 3-4 problems at a time. In fact I event restrict myself to 3-4 problems max and have even been trying to get that number down to 1 (I guess it depends on how high we keep those problems in our consciousness). Would be curious to know what others think of this?
There is one other point that got missed out. Cannot really call it as an idea, but is important. That is ‘Age’. I am talking with experience. When a person is young, they have more time and energy to invest on their goal. A young person has the leeway to make mistakes and learn faster from their fall. An older person with a lot of other competing responsibilities and duties cannot he able to invest that time even if he/she wants to. There is not enough space to make mistakes and learn. Then there is the risk of rocking the family boat – kids, finances, aging parents. Not that it is not achievable, but will need double the effort.
I read your blog for inspiration and to keep the optimism going within myself. I have started very late in my career after completing parental duties of raising young kids. I make mistakes that of a new bee, feel miserable that people my age are up ahead in the career ladder, worry of my reputation. But then remind myself that if I get sidetracked with these thoughts, I will lose sight of my goal.
The reason I am writing my experience is to motivate the younger generation to make the best of the time they have when they are young since this blog is primarily read by young people and by parents of young people like me.
Thanks for the post on Richard Hamming. It has motivated me to pick up myself again and walk. I know, in 10 years from now, my fall will be history.
I got to this page after reading your GTD critique blog post. That was a good one. Then I got here. At first I was like, “wait this sounds familiar” and then I realized that I basically used much of the same ideas to pitch my company (www.melioravit.com) to academics. I must have read or listened to this talk at some point in the past, because much of what I have is quite similar. Need to credit Hamming on the site now.
Beyond that, once ideas 2-5 are executed well enough, idea 8 is by far the most important. Not writing well and giving poor talks kills many (maybe even most) academics. People are busy and academics are crazy busy. Getting an academic to stop what they are doing to read your stuff or listen to your talk should be a top goal for every academic.
Who wants their research to be forgotten or repeated? No one.
Who wants their research built upon? Everyone.
‘…it’s goal was straightforward: to deliver lessons for serious researchers about how to do “Nobel-Prize type of work”’
FYI, “it’s” should be “its.”
“It’s” is short for “it is.”
Though it still makes me wince, I see this error so often, I realize it’s (sic) becoming increasingly accepted, but I don’t think the field of science is that tolerant.
Hi Cal,
I wanted to chime in, if somewhat belatedly, to emphasise the remark about compounding: it’s not just about the extra work. It’s about breaks too. Often when taking a break from tackling a hard problem, I am tempted to check social media du jour/email/etc, but that makes it (if only slightly) harder to resume what I was doing. In contrast, if I refrain from doing this, even if “on a break”, my mind keeps twisting and turning on the problem, on a kind of “background”. At the end of the day, this extra effort also compounds, and it becomes noticeable as days turn to weeks, etc.
Anyway, just my two cents.
Hi Cal,
What is the best way to contact you? I haven’t been able to find your email.
Thanks 🙂
Really cool article Cal. thanks for posting. I miss so much the ability to obsess on problems created through research (on my PhD many years ago). Modern society has gone way too far in the wrong direction — constant distraction, too much information & not enough applied knowledge.
Just found this site … thank you very much! I have read the whole transcript; hard to put it down! I knew Dick Hamming at the Naval Postgraduate School, back in the ’80’s. His comments about “open door” really hit me. He used to come bustling into my office with, “Had your ever considered …”, right out of the blue. After practicing on me (blackboard and all) he would incorporate the thoughts into his next class lecture (“Hamming on Hamming”). Wonderful, egotistical, arrogant, gentle man! I loved him.
Great article!
Mispelling though on Idea 6 “sell you results” -> “sell your results”