Mark Zuckerberg on Photorealistic Avatars, Mixed Reality, and the Future of the Metaverse

This podcast episode features Mark Zuckerberg and Lex Fridman discussing and experiencing Meta's cutting-edge 'Codec Avatars' within the metaverse, highlighting the profound sense of presence and realism achieved. The conversation delves into the technical intricacies of these photorealistic avatars, which are generated from detailed scans and encoded for bandwidth efficiency, capturing subtle facial expressions and flaws that enhance immersion beyond traditional cartoonish avatars. They emphasize how this technology fosters a deeply meaningful connection, making remote interactions feel as intimate as being in the same physical room, thereby breaking geographical barriers.

Zuckerberg outlines the future vision for making these avatars more accessible, aiming for a quick, two-to-three-minute smartphone scanning process to generate high-quality models. He also introduces Quest 3 as Meta's first mainstream mixed reality (MR) product, distinguishing it from virtual reality (VR) by its ability to superimpose digital objects and people onto the physical world. This MR capability, powered by a new Qualcomm chipset and improved pass-through cameras, offers enhanced comfort and safety for users, allowing them to see their physical surroundings while engaging in immersive digital experiences.

Beyond social interaction, the discussion explores broader applications of this converging technology, including remote work meetings, gaming, and the potential for digital holograms to replace many physical objects in daily life. Zuckerberg posits that the 'real world' will increasingly become a blend of physical and digital elements, with MR glasses eventually allowing seamless integration. He also touches on the fascinating prospect of embodying AI assistants as realistic avatars in future meetings, further blurring the lines between human and artificial presence.

The episode concludes with a philosophical exploration of user preference between photorealistic and expressive avatars, and the psychological acceptance of blending different visual styles within the metaverse. While photorealism serves as a crucial bridge for transitioning users from the physical to the hybrid digital-physical world, the long-term balance remains an open question. The overall implication is a fundamental shift in human communication, interaction, and our perception of reality, driven by increasingly sophisticated and accessible immersive technologies.

this technology is incredible and I think it's the future of how human beings connect to each other in a deeply meaningful way on the internet

it feels like the future this is truly truly incredible

the goal and we have a project that's working on this already is just to do a very quick scan with your cell phone where you just take your phone kind of wave it in front of your face for a couple of minutes... and then produce something that's of the quality of what we have right now

the thing that you can do in the metaverse that is different from what you can do on a phone is like doing stuff where you're physically there together and and participating in things together

I do think we're we're kind of beyond the The Uncanny Valley here where it and it does feel like you it doesn't feel uh it doesn't feel weird or anything like that

Quest 3 coming out which is going to be the first mainstream mixed reality product right where you're really taking digital um you know expressions of either a person or or objects and overlaying them on the physical world

I actually think the real world is the combination of the physical world and the digital worlds coming together

I think you'll also have meetings in the future where you're basically you know maybe you're sitting there physically and then you have you know a couple of other people who are there's Holograms and then you have like Bob the AI who's an engineer on your team who's helping with things and he can now be embodied as a um you know as as a a realistic Avatar as well and just join the meeting