The YouTube Algorithm: Engineering Recommendations, Societal Impact, and the Future of AI Curation with Christos Goodrow

This episode features a conversation with Christos Goodrow, VP of Engineering at Google and head of search and discovery at YouTube, often referred to as the 'YouTube algorithm.' The discussion highlights YouTube's immense scale, serving 1.9 billion users who watch over 1 billion hours of video daily, making it the second most popular search engine. Lex and Christos explore YouTube's critical role not just as an entertainment platform but as a wellspring of knowledge, emphasizing the profound engineering challenge of building a recommendation system that enriches users' lives and helps them discover new ideas, which Lex posits will be one of the most impactful areas of AI in the 21st century.

The conversation delves into the nuanced mechanisms of the YouTube algorithm, distinguishing between search functionality (syntactic and semantic matching) and recommendation systems. Christos explains collaborative filtering, where videos watched together by many users are 'clustered' in an embedding space, allowing the system to recommend related but diverse content, even across seemingly unrelated topics like science and jazz. A significant portion addresses the complex ethical and societal responsibilities of YouTube, particularly in managing political discourse, misinformation, and online 'meanness' or trolling. Christos clarifies that YouTube aims to demote borderline content and promote authoritative sources, rather than settling ideological debates, and employs a hybrid approach of human evaluators and machine learning to enforce policies and scale content moderation.

Practical insights include the evolution of 'quality' metrics beyond mere views to watch time and, crucially, post-viewing user satisfaction surveys, which provide richer feedback for the machine learning system. The algorithm views each user as a 'DNA strand' or 'vector' of their watch history, using this to personalize recommendations and introduce diversity. The discussion also touches on the challenge of surfacing high-quality content with low view counts and the platform's ongoing efforts to mitigate unfair biases in its AI systems by instructing human evaluators towards scientific consensus and expertise, and by diversifying reviewer backgrounds.

Broader implications underscore YouTube's immense power and responsibility in curating internet content and shaping global conversations. The long-term vision articulated by CEO Susan Wojcicki is to strike a delicate balance between openness and responsibility, ensuring that YouTube's actions today are viewed as having 'figured this out' by future generations. The episode highlights the continuous, difficult work involved in navigating the philosophical gray areas of truth, ideology, and human interaction on a platform with world-changing impact, emphasizing that both human judgment and advanced algorithms are indispensable for this monumental task.

recommendation systems will be one of the most impactful areas of AI in the 21st century and YouTube is one of the biggest recommendation systems in the world

if from birth when I was born and there's many people born today with the internet I watched YouTube videos non-stop do you think there are trajectories through YouTube video space that can maximize my average happiness or maybe education or my growth as a human being

what you want to do when you're trying to increase diversity is find something that is not too similar to the things that you've watched but also something that you might be likely to watch

we're not trying to settle that choose a side or anything like that what we're trying to do is make sure that the the people who are expressing those point of view and and offering those positions are authoritative and credible

we want to do our jobs today in a manner so that people 20 and 30 years from now will look back and say you know YouTube they they really figured this out they really found a way to strike the right balance between the openness and the value that the openness has and also making sure that we are meeting our responsibilities to users in society

my own experience has demonstrated that you need both of those things [algorithms and human input]

we asked people to have a bias towards scientific consensus that's something that we we instruct them to do we ask them to have a bias towards demonstration of expertise or credibility or authoritative ness

the way the recommendation system of YouTube sees a user is exactly as the history of all the videos they've watched on YouTube

quality has a lot to do with the authoritative nough sand the credibility and the expertise of the people who make the video

we move from views to thinking about the amount of time people spend watching it