Machine Learning, Education, and the Philosophy of Computer Science: A Dialogue with Charles Isbell and Michael Littman

This episode features a lively discussion between Charles Isbell and Michael Littman, two prominent computer science professors, on the fundamental nature of machine learning and the philosophy of education. A central point of contention, and eventual agreement, revolves around whether machine learning is merely "computational statistics" or a distinct discipline. While acknowledging the statistical underpinnings, both guests emphasize that ML transcends pure statistics by incorporating elements of computer science, software engineering, and a focus on data interaction that goes beyond mathematical formalism. They highlight how different perspectives, such as those seen in early ICML (computer science focus) versus NeurIPS (statistics-impressing computer science), shaped the field's development.

The conversation delves into the practicalities of teaching machine learning, with Isbell describing an assignment where students are encouraged to "steal" algorithm code but are rigorously tested on their ability to analyze and understand data's impact on algorithm performance. This approach underscores the critical role of data in ML, arguing that understanding data's nuances is often more important than the intricate details of algorithm implementation. They discuss the concept of "Software 2.0," where programming shifts from explicit coding to designing hyperparameter spaces and curating data, further blurring traditional disciplinary lines.

A significant portion of the discussion is dedicated to the role of hardship and struggle in education. Both guests agree that struggle is essential for deep learning and increased joy in mastery, but they distinguish between "hopeless suffering" and "hopeful struggling." The educator's role, they argue, is to curate experiences that challenge students without breaking their will, providing what students need rather than simply what they want. This pedagogical philosophy is contrasted with the differing ethos of universities like Brown (more nurturing) and Georgia Tech (historically more demanding, exemplified by the "drown proofing" requirement).

Finally, the episode touches on the personal and professional collaboration between Isbell and Littman, highlighting how their complementary perspectives and long-standing friendship have enriched their work, including co-teaching a widely accessed machine learning course. Their dynamic illustrates how intellectual disagreements, when approached constructively, can lead to deeper understanding and a more nuanced appreciation of complex fields like machine learning and the art of teaching.

whether or not machine learning is computational statistics it's not but it is well it's not and in particular and more importantly it is not just computational statistics

statistics is how you're going to keep from lying to yourself which i thought was really deep it is a way to keep yourself honest in a particular way

chemistry is just physics but i don't think it's as useful to think about chemistry as being just physics it's useful to think about it as chemistry the level of abstraction really matters here

icml was machine learning done by uh computer scientists and uh nurbs was machine learning done by computer scientists trying to impress statisticians

my opinions may have changed but not the fact that i'm right

the thing about machine learning is it's not just sorting numbers where in some sense the data doesn't matter what matters is well does algorithm work on these abstract things and one less than the other in machine learning the data matters it does it matters more than almost anything

you have to give them what they need without bending to their will and students are like that you have to figure out what they need you're a curator your whole job is to curate and to present because on their own they're not going to necessarily know where to search

struggling and suffering aren't the same thing right being poetic oh no no i actually appreciate the poetry and i one of the reasons i appreciate it is that they are often the same thing and often quite different right so you can struggle without suffering you can certainly suffer and suffer suffer pretty easily you don't necessarily have to struggle to suffer so i think that you want people to struggle but that hope matters

there's nothing wrong with waiting until the last minute the secret is knowing when the last minute is