Explore a better way to work – one that promises more calm, clarity, and creativity.

The Danger of Black Box Studying

Note: Having handed in my dissertation last week, I guess, for the first time in over 20 years, I’m no longer a student. Worry not, however, Study Hacks isn’t going anywhere in the near future.

Economic Troubles

I recently received the following e-mail from a Berkeley student:

I left [an economics] exam positive I would get an A…The mean was a 77…I ended up getting a 55 — absolutely awful…I feel beyond frustrated by this and am wondering why, perhaps in your analysis, did I think I did so well when I absolutely nuked it?

I receive several e-mails of this type each week. They all follow the same basic format. The student is surprised by doing poorly on a test and is hoping that I can offer some ingenious strategies that will prevent the disaster from happening again.

I’m happy to answer these e-mails, but I’ve been fearing recently that a dangerous sentiment lurks beneath — a sentiment I need to combat.

The Black Box

I discuss a lot of technical studying tactics here on Study Hacks. I fear that this may have communicated a false image of studying as a black box: On one side of the box, study strategies are inserted, and, on the other side, test grades are output. Therefore, if the test grades defy your expectation — as it did for our Berkeley student from above — the answer is to change the inputs. When students write me with stories of academic woes, they are often hoping that I can offer up a different set of inputs to the black box that will guarantee to generate the grades they desire.

Busting Open the Black Box

Studying, however, is not so abstract. My response to the Berkeley student, for example, was to ask the following question:

Why did the students who got the top grades on this test score so much higher than you?

My goal with this question was to bust open the black box. I wanted her to think about the details of this specific class with its specific style of tests, and then answer the question of what exact type of preparation would have been best for this challenge. (Longtime fans of Study Hacks will recognize this as the post exam post-mortem method.)

Only once you have an answer to this context-specific question should you turn to the specific study strategies I talk about here. Think of them as an arsenal of tools to help you conquer a well-defined challenge, not inputs to an abstract process.
Our Berkeley student, for example, noted that the professor took concepts that were mentioned only briefly in class, and then expanded them into long, complicated questions on the exam. She stumbled on these questions which led to the poor grade.

With this specific analysis in mind, she can now imagine how to best prepare for the next exam in the course. She might, for example, build a one-page study guide for each concept mentioned in class — even the small ones. In addition to doing an out loud lecture explaining these ideas, she might also search for a few sample problems from her textbook, and even the web, for each concept, so that she can practice applying them in different contexts. This would better prepare her to tackle this style of question on the next exam — both in terms of knowledge and confidence.

The point here is that her studying process is not a black box. The specific strategies enter the scene late, only after she spent the time to immerse herself in the details of the unique challenge before her.

My Challenge to You

Next time you have trouble on an exam, I want you to consider doing the following. First, perform a post-mortem of the type I described above. Get a sense of exactly what type of perparation would produce A’s in this specific class. Second, after this analysis is done, contact me for advice on strategies that best fit the preparation requirements you identified.

Once you bust open the black box, you’ll be surprised by how quickly you can turn around your performance.

21 thoughts on “The Danger of Black Box Studying”

  1. Congrats on your dissertation!

    Your black box idea makes sense. It essentially goes to say that not all study strategies are optimal for every situation. It reminds me of the one post you had about the guy that doesn’t take notes and just listens to lectures over and over on a recorder. That kind of strategy stuck in a Chemistry black box would be disastrous.

  2. Since you’ve completed your dissertation, I guess you’re a doctor now…
    You’ll do fine with the ladies lol.

    Congrats. Your blog really put a lot of things into perspective to me though I haven’t been reading long. You changed the way I look at studying and some articles have caused me to question how productive I am. You’re going to go far for sure. Good luck on your post-doctoral studies and keep writing.

  3. That’s awesome. What was your dissertation about? Probably something really complicated and difficult!!

    Thanks for this article. This has happened to me a few times..

  4. Thank you all. We’re moving this weekend (from Cambridge to Boston), so I’m about to pack away my computers and modems for a few days. I’ll be back in action next week…

  5. Congrats on the dissertation, Cal!! Glad to hear that Study Hacks will still be up and running.

    What will you be doing in Boston, working or doing post-Doc?

  6. Congrats Cal.

    Black Box’s can be dangerous. The post-mortem method is good and I think its a good strategy with a lot more things than just tests, like papers. If you think you put in 100% and you get out 55 the best thing you could do is step back and evaluate. A lot of people forget this though and look for a quick fix. Nice post.

  7. There can be a psychological momentum in unrealistic self-assessments as well. It is sometimes referred to as the “Dunning-Kruger Effect”:

    People tend to hold overly favorable views of their abilities in many social and intellectual domains. The authors suggest that this overestimation occurs, in part, because people who are unskilled in these domains suffer a dual burden: Not only do these people reach erroneous conclusions and make unfortunate choices, but their incompetence robs them of the metacognitive ability to realize it. Across 4 studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although their test scores put them in the 12th percentile, they estimated themselves to be in the 62nd. Several analyses linked this miscalibration to deficits in metacognitive skill, or the capacity to distinguish accuracy from error. Paradoxically,improving the skills of participants, and thus increasing their metacognitive competence, helped them recognize the limitations of their abilities.

    Like they say sometimes – In order to be able to study successfully, one needs to be humble.

  8. Pingback: zatexolin
  9. Hi! (Found this blog via the Wil Wheaton blog mention…)

    I’m a part-time college professor (other part = stay-at-home dad), and students doing worse than they expected actually happens a fair amount. A self-assessing post-mortem is good — but what would be even better is to meet with the professor, and (in a non-confrontational!) way, ask him/her how you fell short. Most professors are super-impressed that a student is actually taking the class seriously, and wants to do better (as opposed to “simply passing”).

    Broadly speaking, students tend to under-perform in one of three ways:

    1) They’re just plain wrong. (i.e. They **think** they know their stuff. But they’ve learned it wrong. This also includes reversing or confusing similar concepts. Doh!)

    2) They don’t elaborate or clarify sufficiently. (i.e. They say stuff which, yeah, is correct. But they don’t go far enough. [e.g. “The American Revolution took place in 1776, and pitted the American colonists against the British. The End.”])

    3) They don’t answer all parts of the question. (E.g. “Explain Social Disorganization Theory, and briefly explain a crime-reduction intervention consistent with that theory.” Students define it, but then forget to create an intervention.)

    This probably applies to the humanities and social sciences. May not apply to math-based things, like Chemistry or Stats. 😉


  10. …ask him/her how you fell short. Most professors are super-impressed that a student is actually taking the class seriously

    Thank you for the addition. Your three observations are particularly relevant to my readers. I should have included this general “ask the professor” strategy in the original article as I often suggest it to students. The other addendum I provide — which I’m sure you agree with — is to be 100% clear that you’re not complaining or hoping to improve your grade.

  11. Congratulations Cal! Keep up the good work.
    Thanks very much for all the advice and inspiration in the past year. You have helped me immensely!

  12. Congratulations Cal! Keep up the good work.
    Thanks very much for all the advice and inspiration in the past year. You have helped me immensely!

  13. I have had this happened to me many times…leaving a test thinking I did way better than I actually did. So far this year your study tips have helped me greatly, and I actually end up getting the grade I thought I did!!


Leave a Comment