An Obsessive Digression
For the past two weeks, I’ve been trying to prove a bothersome theorem. It’s not particularly flashy, but I need it for a paper. More importantly, it felt like it should be easy and I took it personally that it’s not.
Predictably, I began to obsess about this proof — by which I mean I took to returning to the proof again and again during breaks in my working day. It became a staple during my commutes to and from work, and began to hijack blocks of time from my otherwise carefully constructed schedules.
Earlier this week, the weather was nice, so while waiting out the traffic at home in the morning I sat outside in my backyard with my grid notebook (something about grid rule aids mathematical thinking) and, as I had been doing, noodled on the theorem.
Except this time: something shook loose.
I scribbled notes for an hour, drove to campus, and set about trying to formalize my new idea.
It didn’t work.
But now I had the scent. Long story short, six hours later I had a proof that seems to work for a more or less reasonable version of the problem (time will tell).
I started that day with a pretty elaborate time block schedule. It was ignored; as was my e-mail inbox; as were several pretty important administrative obligations. But the important thing is that I think I finally tamed that damnable theorem.
Obsession as Productivity Tool
In my work as a theoretician, (bounded) obsession of this type plays an important role in productivity.
A main theorem in this 2014 paper, for example, was finally proved on the metro, whereas the main theorem in this 2013 paper was cracked on a speaking trip to Canada (I started working on it when I arrived at the airport in D.C. and had the key points nailed down by the time my limo arrived at the hotel in Waterloo) . In an interesting coincidence, the breakthroughs in this 2014 paper and this 2011 paper both happened while stuck at home during (different) snow storms.
In all cases, if I hadn’t allowed the relevant problem to evolve into an obsession, I might not have solved it. They required lots of hours of deep thinking under lots of conditions: both products of obsession.
With this in mind, my contention in this post is that this trick of the theoretician is relevant in many more fields.
Important things are hard to do. Obsession supports hard accomplishment.
The challenge, of course, is keeping the obsessions under control (a somewhat oxymoronic task), and learning when to unobsess when progress stalls too much (at best, 1 in 3 of my obsessions yield theorems).
In the final accounting, however, obsession remains a tool that’s not talked about much but should be, as it often plays a key role in elite level knowledge work.
Great post.
I grew up in Waterloo, I bet you went to UW…
Thanks.
Actually, Laurier…
I’ve experienced such obsessions as this, and it is always when I’m trying to solve a problem. Once the obsession starts, I can tell that I never really let it go even when I’m doing other things. My obsessed subconscious continues to work on it. I think that this is an important part of solving very difficult problems.
The weird thing is that I am a little ADHD, but once an obsession hits, I enter a period of hyperfocus.
That’s actually a symptom of ADHD. https://www.additudemag.com/adhd/article/612.html
I find the “obsessed subconscious” to be important to the work I do as well. In fact, if I know I’m working on a particularly difficult problem, it’s useful to examine the parameters, consider strategies, and then put it away for a few days. Only then can my obsessed conscious brain make inroads on the problem, it seems.
I agree with Mark – I suspect that there was some enforced ‘down time’ just before the ah ha moment (on the plane? rescheduling things on the snow storm day?). You needed the focus AND the time for being distracted for it all to come together. I sometimes think being in nature helps (hence the coming together in your back garden)…work hard at the task then take a walk with the dog often helps me over hurdles when I’m writing up.
I agree. There have been numerous times when I have stayed up late working to fix a bug in my code. I am usually unsuccessful during these late-night sessions, but as soon as I wake up, I can amazingly remember one tiny syntax or logic error in hundreds of lines of code and fix it. “Sleeping on it” is real!
Thanks for sharing.
I have been working on a proof of a theorem for my paper in the past two months, only able to prove a simplified version of it. It also looks like to be easy, however, it is not personally. Any specific tips on how to get unstuck and progress from your experience?
Obsess for a while then move on for a while then return for a while and so on. After I’ve cycled back to a proof a few times I might give it up until the next time I meet someone who might have a new insight or I encounter a result that might help.
I think obsession also evidences itself in the quality of the output. You “needed” a proof, but it was looking ugly, if it could be solved at all. When one allows things to just be “tasks” on a list, quality inevitably suffers. Obsession drives one to care about the result, about the artistic merit of the solution.
In my field I find that if the solution I’ve patched together because I “have to get this done” is ugly enough, I procrastinate. I can’t make myself write the code. I start to lose confidence in the whole project because one component is looking kludgie. This happened just this week, and then the other morning I popped awake with a full solution in my head, and it was elegant, and it was right.
I don’t think the subconscious mind would dwell on a problem long enough to reach an elegant solution if there wasn’t some degree of sub/conscious obsession involved. Just wanting to complete the task is not enough. Beautiful solutions take more work, and that extra work only gets done by (slightly?) obsessive people.
The title of this post is oxymoronic. Can super-deep, obsessive work that is done infrequently–when needed– be a habit? Or are you always in some state of deep, obsessive work?
It’s frequent enough that we can safely call it a habit.
Cal, good post, and I think that this is an element that you had addressed more fully in “so good…”
getting good at things requires the dedication and consistency and without some sort of obsession, that is hard to pull off for most people. and it is hard to manufacture that for an arbitrary subject.
I think that is the helpful part of passion, though I appreciate so much your pushback on untested, or passion without substantive work.
“Passion” is too fuzzy of a term. I think, to be more dry, that an “evidence-based and firmly held belief that the pursuit is something you can with time conquer and it would be quite worthwhile to encounter” is sufficient to begin a long-term, Steve Martin-style obsession.
Ha ha. Nice. The one thing you are really obsessed with is figuring out and documenting how to do top notch work and you come to the conclusion that the answer is … Obsession. I like the post.
meta-obsession with meta-reflection 🙂
Just read an interesting book that speaks to this type of problem solving. Evidently research shows the best way to develop solutions is what I’m going to label as “controlled obsession” (I know, another oxymoronic phrase).
Gather all the information you need, or could possible need, to solve the problem. Study it, look at it from different angles, and then after you’re satisfied Mr. Pareto, go do something else. Let it go. Drop the subject from your conscious.
In many studies this approach consistently provided the greatest percentage of correct answers vs. the more traditional “hard work” mantra (think about it until you bleed out of your ears). The theory is that our subconscious is always working and sometimes just needs a little space to figure things out.
“Upside of Your Dark Side” is the name of the book.
I’ve been doing some reading on the science behind such things. It’s an interesting topic and one that is not as addressed as often as you might expect. I’m increasingly convinced that subconscious thinking plays a big role in my work.
I find my most “A-Ha” moments while exercising, particularly long runs or walks. For some reason, the physiological stress seems to open up the psychological/mental paths. And it seems science bears this out.
Obsession, passion, whatever term we may call it, only boils down to one objective, task completion. This is all innate for a human being but the moment you are hyper-focus just like Bill Gates, that maybe called obsession in a good sense. We tend to pour our everything for a job completion yet we must understand that we dont have to bleed every battle we encounter. Working smart is a tool in which we can satisfactorily get task done, I mean tasks, a lot of them. A lot of resources and people think way greater than our mind could conceive that catalyst for a work completion.