Study Hacks Blog Decoding Patterns of Success

Yoshua Bengio’s Deep Thoughts on Deep Thinking

May 22nd, 2018 · 26 comments

Thinking about Thinking

One of the things that surprised me when researching Deep Work was how rare it was to find examples of smart people talking coherently about the process of effective thinking.

This is why I was pleased when a reader recently pointed me toward a video interview with computer scientist Yoshua Bengio — one of the big names in machine learning.

Around an hour and ten minutes into the interview, Bengio turns his attention to the topic of productive thought:

“There is also something to be said about concentration…to really make big progress in science you also need times when I can be very focused and where the ideas about a problem, and different points of view, and all the elements sort of fill my mind. I’m completely filled with this…that’s when I can be really productive.”

Filling your mind with a single problem, Bengio emphasizes, is not a fast process, noting:

“It might take a really long time before you reach that state…”

Hard problems are hard to understand. The best researchers in my field of theoretical computer science, for example, tend to be those who have the drive, brain power and flexibility to read other peoples’ papers constantly to spark a new recombination or extension that advances the field — an exhausting process.

But as Bengio emphasizes, this effort is worth it:

“[After you fill your mind with a problem is] when you can really start seeing through things and getting things to stand together and solving…now you can extend science, now when things are solid in your mind you can move forward.”

Our culture likes to emphasize eureka moments, but we often miss the hard, patient work that goes into preparing the mind to generate breakthroughs.

(Hat tip: Daniel)

26 thoughts on “Yoshua Bengio’s Deep Thoughts on Deep Thinking

  1. Ayrat says:

    You might also find interesting the book by
    Josh Waitzkin
    the Art of Learning
    where he emphasizes the importance of internalizing of the basics
    to the degree that it becomes your native language.

    1. EA says:

      Yep, that was a good book! (Great chess player)

  2. Brian J says:

    Cal — love your blog and work, but I have to admit this post really shocked me because there has been a long tradition in philosophy of trying to get at this very question, from Aristotle’s pioneering work on logic to Descartes’ Discourse on Method. And, since you’re mathematically inclined, it might delight you to know that Plato himself looked to the methods utilized by geometers for mathematical discovery as a model for articulating what we do in thinking in general. Plato was interested in how mathematical analysis lead to moments of deep comprehension (called “nous” in Greek); in turn, that deep comprehension was then re-organized into a synthesis:
    https://hsm.stackexchange.com/questions/5629/analysis-vs-synthesis-in-greek-mathematics?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa
    Sorry to lecture you here but philosophy is my bread and butter.

  3. Adam says:

    This book has a very useful discussion of focussed vs diffused thinking in learning science that may be worth discussing.
    https://barbaraoakley.com/books/a-mind-for-numbers/

    The eureka moment can come when you are relaxed (diffused) after concentrated effort on a problem (the incubation period in the literature on learning). In short, I don’t that there is one way to think deeply to move a problem ahead.

  4. sri says:

    Aha!: The Moments of Insight that Shape Our World by William B Irvine is a good book on the topic of insights, how they come about, the role of subconscious mind, and the prerequisite of multiple attempts made by the conscious mind etc.

  5. Viliami says:

    Hi Cal
    In a podcast interview for Straight A in Optimise, you mentioned that you systematically did trial and error to find the best study methods that suited your situation, and it seems that you’ve done this as well when creating a system that allowed you to master certain skills in a limited time period. I’d like to ask, how do you create such systems from scratch?

  6. Daniel Palacios says:

    I couldn’t agree more with the last paragraph. Our culture see the eureka moment as something only related to “geniuses”; of course that they don’t tend to take in mind all the hard and deep work behind the moment.

    Indeed, this post is the right place to share one of my all time favourite quotes:

    “Chance favors the prepared mind.”
    -Louis Pasteur

    1. Cato says:

      Similarly:
      “Genius is one percent inspiration, ninety nine percent perspiration.” —Thomas Alva Edison

  7. Vikrant Singh says:

    Hi Cal,

    There was a post “Are You Using Social Media or Social Media Using You” & then another on “Brain Hacking and Sean Parker on building FB”, I found something that adds to that. Hope you and other out here find it useful.
    This video has formed FB employees including Sean Parker telling the deep dark things about FB itself.

    https://www.youtube.com/watch?v=39RS3XbT2pU

  8. It’s a little bit complicated but really interesting.
    Learning, and reaching a better understanding something that will lead us to real changes.
    A.I. it’s simply a new frontier will be used to really have a better life for all the people?

  9. Fiona Mumoki says:

    Awesome pieces of advice!

  10. Esther Lake, Author says:

    How wonderful to hear what intelligent people are thinking. Refreshing and motivating, a great antidote to all the nonsense out there.

  11. William Pennat says:

    The best researchers in my field of theoretical computer science, for example, tend to be those who have the drive, brain power and flexibility to read other peoples’ papers constantly to spark a new recombination or extension that advances the field — an exhausting process. — From the article

    That’s precisely why I left academia. I got sick of reading poorly written, poorly reasoned tracts just so I’d have a set of references for my own. At least 90% of everything in academic journals is dull, pedestrian crap. I don’t see how it could inspire anyone to greater creative thinking. (Sorry….)

    1. george says:

      Absolutely right!

  12. j says:

    to really make big progress in science you also need times when I can be very focused ….see anything wrong here heavy thinker?

  13. Michael Will says:

    In this 1959 essay, Isaac Asimov turned his powerful brain to these concepts:
    https://www.technologyreview.com/s/531911/isaac-asimov-asks-how-do-people-get-new-ideas/

  14. David Willhite says:

    The missing link in this discussion is, humans have a soul. A soul is “who” a person is, and is separate from the brain. These scientists seem to be approaching the problem of computer learning (or truthfully AI) from an evolutionary baseline. Wrong assumptions end with wrong conclusions.
    When a person dies, their consciousness (soul) leaves the body along with their life force (spirit) and goes on to the afterlife without the brain (computer).

  15. Larry Cox says:

    The subconscious: Brilliant people tend to ignore it because they feel as though they have it under control. But it will shape the directions they take in their creative thinking.

    There is a lot of missing data about this. It would change the world a lot if more people were aware of how insidious the subconscious can be, and knew how to handle it.

    A mastery of the basics is obviously necessary in doing things: The singer and his voice; the photographer and her camera; the coder and his IDE. But I think this is also true at higher more conceptual levels. It’s hard because so few ever achieve it, and if they do, they rarely write it down in a way that others could understand it.

  16. Gregory says:

    Amazing video, good advice! I likes Professor Yoshua Bengio works.

  17. Raymond says:

    You’ve changed my life when I saw your video on active recall in Texas 3 years ago. This one thing that you said changed how I approached everything that required learning from chess to learning math. You said “If you’re not actively explaining something without looking at your notes, YOU’RE NOT REALLY LEARNING.” I relied way too much on rereading my notes or someone else’s notes and after reading them, often times I would feel that I “understand” By putting your quote into practice I have realized that I can use “explaining out loud” as a litmus test to see if I really understand what I say I understand and do this in front of an mirror and I realized that I actually don’t understand what I say I understand. So basically you made me realize how clueless I really am and that’s the best thing that has happened to me. I’m now actively explaining things but I still need to work on simplifying my explanation for every subject.

  18. Great blog! I have never gave it thought in the way you did! I am definitely excited to read more on this topic and see what other people comment about it!! Thank you!

Leave a Reply

Your email address will not be published. Required fields are marked *