The Wisdom of (Math Nerd) Crowds
A couple weeks ago, I complained that my academic paper reading speed was slower than I would like given its importance to my productivity. I asked for your advice and you responded with over 60 comments and numerous private e-mails.
My goal in this post is to synthesize the best ideas from this feedback, as well as the results of my own self-reflection, into a clear answer. In particular, I’ve identified three big ideas relevant to trying to read technical papers — and in particular those containing mathematical proofs — as efficiently as possible.
Idea #1: There are no magic bullets…
This conversation helped cement an idea that I’ve long suspected to be true (but sometimes resist):
To develop a detailed understanding of a published mathematical proof is an ambiguous process that requires multiple attacks and can take an unpredictable amount of time (not unlike proving something in the first place).
As a result, you must be selective about what proofs you decide to dig into, as the time commitment is potentially serious. In the study of algorithms (my field), for example, in most cases when reading a relevant paper it’s sufficient to dive down just deep enough to answer the following questions:
- What is the main result and how does it compare to what was known before (or what a naive approach would provide)?
- What is the high-level insight/trick deployed in the bound argument that enabled this improvement?
With experience, I’ve found that I can consistently produce this level of understanding within an hour (sometimes less if the paper is well-written or building on my own results).
This knowledge is not enough by itself to deploy or extend the technique presented in the paper, but it is enough to recognize future opportunities where this technique might be relevant to a problem you care about (at which point, you’ll have to dive deeper). In other words, maybe just one out of ten papers you read will end up proving directly useful to your own work, so it makes sense to learn just enough from the papers you read to identify whether or not they’re in that crucial 10%.
Idea #2: There are ways to be more efficient…
If you must understand the details of a proof, then in addition to the high-level suggestion from above of preparing yourself psychologically for a difficult battle, the following low-level strategies might also help:
- Instead of trying to read through the proof linearly, build a hierarchy of dependencies among the lemmata and theorems. Summarize each lemma and theorem in your own words and summarize each dependency relation; e.g., how does this theorem use the following three lemmata? Once you have this map, it becomes clearer where to begin a deeper dive and provides context for what you’re reading.
(Last time I deployed this full proof-mapping process — which can be quite arduous — I ended up uncovering a flaw in a reasonably well-cited paper.) - In general, you should never start reverse-engineering a mathematical derivation until you understand what it is trying to show, why you expect it to be true, and how it will be used. If possible to assign some of this reverse engineering to a grad student, do so: it’s helpful to both parties.
- Create your own system of notation and rewrite the relevant statements and re-derive the main results (or, rough approximations of the main results) using this notation. You’ll likely have to revise this notation system many times before you’re done, but this process will make it much easier for you to conceptualize the deeper insights of the argument.
(I had to do this last week for a proof that I needed to understand better. It took me something north of six hours to complete! But I do certainly understand better now what is going on underneath the covers of this particular line of thinking.) - Form reading groups with like-minded academics. Something about collaboration has a tendency to bust open mental road blocks and incite more creative thinking.
Idea #3: But perhaps the best strategy of all…
Get the authors on the phone or pull them aside at a conference and have them walk you through the argument. Nothing is more efficient than having the original author fill in the details of his or her thoughts.
(This latter strategy, of course, becomes more available as your status in your field grows. It might not be advisable, for example, for a first year PhD student to apply it with too much regularity!)
I like the notation suggestion.
Rewriting a proof in different (often more personal) notation is like translating foreign language passages, or transcribing music from one key to another. Just the process of it cements your understanding.
I like that parallel, Jeff!
On idea #3. A lower status academic might be able to use this idea a lot more often were he to create a series of good videos with formulas from important papers that he does understand well (preferably outside his main area of interest). Then, when approaching other academics, he could offer to repay their time investment by creating a similar video for them and making it available on vimeo. Preferably on a paper other than the one our lower status academic is most interested in. Or, on that paper but with a delayed “publishing schedule” of six months. The argument being that the better able other academics are able to understand his ideas, the more he will be cited, the greater his influence, the more attractive academic groupies will pander after him at conferences, etc.
On idea #3. A lower status academic might be able to use this idea a lot more often were he to create a series of good videos with formulas from important papers that he does understand well (preferably outside his main area of interest). Then, when approaching other academics, he could offer to repay their time investment by creating a similar video for them and making it available on vimeo. Preferably on a paper other than the one our lower status academic is most interested in. Or, on that paper but with a delayed “publishing schedule” of six months. The argument being that the better able other academics are able to understand his ideas, the more he will be cited, the greater his influence, the more attractive academic groupies will follow adoringly behind him at conferences, etc.
Sorry, but this is a ridiculous level of commitment. Academics are literally institutions whose job it is to come up with and disseminate research. Even as a high school student, I could write to the most famous academics in the world with specific questions or troubles, and they would correct me right away or tell me the key missing insight.
As long as you show you’re intelligent and trying, anyone would take a minute to set you straight. It only takes them a minute!
(This latter strategy, of course, becomes more available as your status in your field grows. It might not be advisable, for example, for a first year PhD student to apply it with too much regularity!)
Why is that? Why dissuade anyone from doing this? With a proper explanation / introduction (“I’m a university student interested in [topic] and came across your paper; I was hoping you could explain this point to me…”), I don’t see any real downside to this — unless you harass or repeatedly bug the same professor or department without having first thought about the proof / paper considerably.
FOR GREAT INCREASE OF MATHEMATICKAL EFFIECIENCY IN GREAT REPUBLICK OF MATHAKHSTAN!!!
But with a comment like “you should never start reverse-engineering…” I can’t take you seriously. I Pity the fools who listen to you.
(This latter strategy, of course, becomes more available as your status in your field grows. It might not be advisable, for example, for a first year PhD student to apply it with too much regularity!)
Why? Why dissuade anyone from doing this? With a proper explanation / introduction (“I’m a university student interested in [topic] and came across your paper; I was hoping you could explain this point to me…”), I don’t see any real downside to this — unless you harass or repeatedly bug the same professor or department without having first thought about the proof / paper considerably.
Conferences serve a lot of purposes. Some people have scheduled meetings. Others are are trying to squeeze in some research in the spur of the moment. Others are desperately trying to get a hold of the speaker from the last session, but unfortunately he or she is stuck with someone who refuses to let the speaker go to the bathroom before the next session starts. Conferences are also social gatherings where researchers meet their friends, and the more senior researchers go back 20 or 30 years together.
As flattering as it is to meet people who like and have read your work, there’s a limit on how much time people are prepared to spend explaining it. What that limit is depends on the person, but assume that a bigger name/more senior means less free time on their hands.
Looking at stereotypes I think it’s basically two distinct sets of students. The first set haven’t done their homework or thought much about the problem at all, and they have no qualms in endlessly stalking the top names with any trivial question they can think of (ask a fellow student or supervisor those questions instead). The second set have spent weeks preparing, and they are too shy to approach the person at all.
I would like to add to this a general strategy I learned from my colleague about reading papers in general: read the abstract then skip ahead to the theorems.
This is in contrast to my usual strategy of reading linearly from the beginning: abstract, introduction, previous work, results, etc. Reading linearly, I learn a lot about the field, but it’s inefficient because I get tired by reading so deeply…
Sorry, I eventually forgot to add a proper comment yesterday. 🙂
After reading that other post that led to this one, there’s something I’d like to comment on:
You said that reading the text surrounding a theorem and its proof often takes you longer than reading the formulas and unambiguous statements of the theorem, lemma or definition.
Well, I can relate. I’m not very good in Mathematics, but I can tell you things began to improve when I stopped reading anything else but the theorem and the proof on page. Only *after* I had at least a general understanding of the subject at hand I went reading the explanatory text. It was easier and didn’t feel like a heavy rock in my chest.
Don’t know if that’s your case, but emotions and schematization play an important role in my understanding of sciences (as well as any other subject) – they need to *flow* so I have no time to waste over a passage that confuses me. I study non-linearly, and starting with theorems, formulas and proofs (and then all the rest) helps me keep the flow going.
And I read aloud if I can. Listening to myself talking creates a duality, so to speak – a part of me talks, another part thinks. But the effort is reduced to a bare minimum. I even found connections between topics when I did this while prepping for a Math exam three years ago.
Well, that’s my two cents, I guess.
– Luana S.
Thanks for the great post, Cal. Will definitely use your suggestions to become better at understanding papers. Also, this is a good example for employing deliberate practice to become better at a task for which there are no straightforward quantitative measures.
What I wanted to ask you is if you have any suggestions on how to administrate and collect all the information you’re getting. I often find myself spending an hour or so trying to understand a paper, hoping to be able to use it in the future, but I do not really have a system for dealing with the information, so I usually just end up accumulating huge piles of unreadable notes. How do you deal with this?
I respect all the answers here, but it’s things like this that make me glad I pursued other fields of study than forcing myself to continue with deep math study. The intense thinking involved makes my head hurt just thinking about it!
Not specifically related to this post, but I see that those handy little tags you used to have on your blog aren’t available anymore. Not sure if you could add them back in again but it would be wonderful to be able to track down all the deep work, hard focus, or deliberate practice posts relatively quickly.
Cheers,
Ian