Biological vs. Artificial Neural Networks: Evolution, Understanding, and the Future of AI

The discussion centers on the fundamental differences between biological and artificial neural networks, particularly highlighting how biological systems leverage evolutionary "quirks" and "glitches" into functional features, a process largely absent in current AI. John Hopfield emphasizes that evolution has optimized neurons to exploit various possibilities, leading to complex behaviors like synchronization, which can be computationally significant. He uses the analogy of the Millennium Bridge's oscillation and synchronized pedestrian walking to illustrate how loosely coupled systems can lock together, forming a computational feature.

A crucial distinction is drawn between evolutionary adaptation (learning over generations) and individual learning (within a lifetime). While both are present in biology, Hopfield finds the latter more amenable to study. He also points out the significant structural difference: biological brains are inherently three-dimensional, enabling complex wiring and problem-solving, whereas computer chips are predominantly two-dimensional. This structural difference impacts what is computationally easy or hard. Furthermore, biological systems exhibit collective properties, like brain rhythms, which are currently not utilized in artificial neural networks, despite the vast number of components in modern computers.

The conversation touches upon the iterative nature of AI development, where the field periodically draws new inspiration from neurobiology when existing models reach their limits. Hopfield suggests that future breakthroughs in understanding the mind might still come from a physics-based approach, emphasizing the importance of experimentation, mathematical modeling, and a deep, intuitive "understanding" beyond mere memorization or lookup tables. He implies that AI needs to incorporate more biological principles, such as feedback mechanisms and collective properties, to achieve true understanding or human-level intelligence.

The broader implications revolve around the long road ahead for AI to truly emulate or surpass biological intelligence. Hopfield speculates that it will take "generations" of evolution in AI, continually integrating more insights from neurobiology, to pass increasingly broad aspects of the Turing test. He acknowledges the surprising utility of current "manifestly non-biological" learning systems but posits that deeper understanding requires embracing the "messiness" and emergent properties that evolution has honed in biological brains, potentially by making artificial components "more messy" or incorporating collective phenomena.

"one of the things very much intrigues me is the fact that neurons have all kinds of components properties to them and evolutionary biology you have some little quirk and how a molecule works or how a Silbert and it can make me and use of evolution will sharpen it up and make it into a useful feature rather than a glitch"

"the glitches become features in them in the biological neural network"

"if you take things which oscillate their rhythms which are sort of close to each other under some circumstances these things will have a phase transition and suddenly is the rhythm wolf everybody will fall into step"

"most artificial neural networks don't even have action potentials let alone have the pathology for synchronizing them"

"adaptation is everything when you get down to it but the difference there are there differences between adaptation where your learning goes on on the over generation that over evolutionary time as your learning goes on at the timescale of one individual who must learn from the environment during that individuals lifetime and biology has both kinds of learning in it"

"if you look at computer chips computer chips are basically two dimensional structures they two point one dimensions but they really have difficulty doing three-dimensional wiring biology biology is the neocortex is actually also sheet-like and insists on top of the white matter which is about ten times the volume of the gray matter and contains all what you might call the wires"

"I find things most interesting that I begin to see how to get into the edges edges of them and tease them apart a little bit and see how they work and since I can't see the evolutionary process going on I am in awe of it but I find it just a black hole as far as trying to understand what to do"

"I think it's going to go on generation after generation the way it has where what you might call the AI computer science community says let's take the following this is our model of Neurobiology at the moment let's pretend it's good enough and do everything we can with it and it does interesting things and after the while sort of grinded at the sand to say Oh something else is needed from neurobiology"

"if you if it were useful to know when are you going more than 45 miles an hour you just capture that and you wouldn't worry about where it came from"

"all the transistors are somewhat similar and most physical systems with that many parts all of which are selfs or have collective properties yes soundly is an error Earthquakes what have you have collective properties whether there are no collective properties used in artificial neural networks in AI"

"one of the most biggest surprises to me was how well learning systems was there manifestly non-biological how important they can be actually and you how important it how useful they can be in a high you"