Peter Norvig on AI Trust, Explainability, and the Seduction of Low-Dimensional Metaphors

The discussion centers on the critical challenges of explainability, trust, and validation in artificial intelligence, particularly concerning neural networks. Peter Norvig argues that a singular focus on "explainability" is insufficient; instead, the emphasis should shift towards building comprehensive trust through rigorous validation and verification processes. He highlights that while current AI systems demonstrate remarkable success in many domains, they also exhibit surprising and often catastrophic failures due to their inherent lack of introspection. A core problem, Norvig suggests, stems from our "seduction by low-dimensional metaphors" when attempting to conceptualize and understand the complex, high-dimensional operational spaces of AI.

Norvig draws crucial distinctions, differentiating between a mere explanation and a more robust "conversation" or "testing" approach. He emphasizes that a single output explanation from an AI might not accurately reflect reality or reveal underlying biases, citing the example of a loan denial that could be based on religion rather than stated collateral. To truly understand an AI's decision-making, he advocates for adversarial testing and the analysis of patterns across numerous cases, rather than relying solely on individual explanations. He also challenges the common perception of AI performance existing in a "2D flat space" where we are "nearly there," proposing instead that AI operates in a "million-dimensional space" where slight deviations can lead to unpredictable and unknown outcomes.

To address these issues, Norvig recommends moving beyond simplistic explanations to implement interactive, conversational interfaces with AI systems, enabling users to probe "what if" scenarios and engage in a dialogue. He underscores the importance of robust testing methodologies, including adversarial examples, to validate a system's resilience and expose vulnerabilities that highlight the limitations of our current understanding. Furthermore, he posits that a fundamental shift in our conceptual framework—recognizing the true high-dimensional nature of AI models—is essential for developing effective solutions to improve their robustness and reliability.

The conversation extends to the broader societal challenge of cultivating trust, drawing parallels between the inherent trust humans place in each other (e.g., navigating a crowded city without fear) and the pervasive skepticism directed towards AI. Norvig notes that human societies have evolved complex mechanisms for building trust, which AI systems will similarly need to integrate. He also suggests that communication technology, by enabling vast data collection and global connectivity, often plays a more significant and transformative role in current technological shifts than AI technology itself, highlighting the interconnectedness and synergistic effects of various technological advancements.

"anytime you use noodle networks anytime you learn from data form representation from day in an automated way it's not very explainable as to or it's not introspective to us humans in terms of how this neural network sees the world where why does it succeed so brilliantly on so many in so many cases and fail so miserably in surprising ways and small"

"I prefer to talk about trust and validation and verification rather than just about explain ability and then I think explanations are one tool that you use towards those goals"

"the explanation alone is not enough right so you know we were used to dealing with people's and with organizations and corporations and so on and they can give you an explanation and you have no guarantee that that explanation relates to reality"

"I think a conversation is a better way to think about it than just an explanation as a single output and I think we need testing of various kinds"

"the way I think about it is I think part of the problem is we're seduced by our low dimensional metaphors"

"it's not a 2d flat space that we've got mostly covered it's a million mentioned space and cat is this string that goes out in this crazy bath and if you step a little bit off the path in any direction you're in nowheres land and you don't know what's gonna happen"

"we seem to approach the skepticism always always you know it's like they have to prove through a lot of hard work that they're even worthy of even inkling of our trust"

"the most amazing thing about humans is that you can walk into a coffee shop or a busy street in a city and there's lots of people around you that you've never met before and you don't kill each other"

"a lot of what we've seen is more due to communications technology than AI ta AI technology"