Ray Kurzweil on the Future of Intelligence, AGI, Deep Learning, and the Hierarchical Neocortex

Ray Kurzweil, a renowned futurist and inventor, discusses the history and future of Artificial General Intelligence (AGI), tracing its roots back to the 1956 Dartmouth conference and the early bifurcation into symbolic and connectionist (neural net) schools. He recounts his early interactions with Marvin Minsky and Frank Rosenblatt, highlighting Rosenblatt's prescient but untried idea of multi-layer neural networks, which forms the basis of modern deep learning. Kurzweil emphasizes that the current explosion in deep learning capabilities is due to the combination of multi-layer nets and the exponential growth of computing power, enabling the processing of massive datasets, exemplified by systems like AlphaGo and autonomous vehicles. Kurzweil delves into the limitations of current deep learning, particularly its reliance on vast amounts of labeled data ("life begins at a billion examples") and its "black box" nature, which hinders explainability. He contrasts this with human learning, which can generalize from few examples, and introduces his own hierarchical model of the neocortex. This model, developed over decades and now supported by neuroscience, posits that the neocortex is a hierarchy of repeating modules, each learning simple sequential patterns, which then combine to form complex understanding. He argues this hierarchical structure is crucial for understanding language and the natural world, and for making AI more transparent and efficient in learning. The discussion extends to the evolutionary history of the neocortex, from its emergence in early mammals, enabling creativity and innovation, to its rapid expansion after the Cretaceous extinction event, leading to primates and eventually humans with advanced frontal cortices for language, art, and tool-making. Kurzweil describes his work at Google, applying his hierarchical model to natural language understanding, citing the success of systems like Smart Reply and recent advancements in paragraph comprehension tests, which now surpass average human performance. He predicts that once AI reaches human levels in a domain, it rapidly surpasses them, though language may take longer than simpler games. Kurzweil also touches upon the societal implications of accelerating technological progress, including the future of jobs and the concept of "longevity escape velocity." He argues that while automation eliminates old jobs, it creates new, more fulfilling ones, a historical pattern. He believes that with continued diligence and rapid advancements in biotech and AI, humans are on the cusp of extending life expectancy faster than time passes, potentially within a decade. He advocates for a hierarchical approach to AI, integrating the strengths of connectionist systems with the structural organization needed for explainability and efficient learning from limited data, moving beyond the limitations of purely symbolic or flat deep learning models.

all the excitement we see now about deep learning comes from a combination of two things both many layer neural Nets and the law of accelerating returns which I'll get to a little bit later which is basically the exponential growth of computing so that we can run these massive nets and handle massive amounts of data

there's still a problem in the field which is there's a motto that life begins at a billion examples

humans can learn from a small number of examples your significant other your professor your boss your investor can tell you something once or twice and you might actually learn from that

the neocortex is organized as a hierarchy of modules in each module can learn a simple pattern

neocortex is neocortex

the world is hierarchical that's why evolution developed a hierarchical brain structure to understand the natural hierarchy in the world

another criticism of deep neural Nets they don't explain themselves very well it's a big black box that gives you pretty remarkable answers

I believe we're only about a decade away from longevity escape velocity we're adding more time than is going by not just the infant life expectancy but to your remaining life expectancy

if you have a hierarchy it's much better at explaining it because you could look at the content of the of the modules in the hierarchy and they'll explain what they're doing

what's missing from deep learning is this hierarchical aspect of understanding