George Hotz on Winning, Reinforcement Learning, and Discovering Life's Universal Reward Function

George Hotz articulates his unique perspective on the meaning of life, defining it as "to win." He clarifies that this isn't about material success but rather an agent's fundamental drive within a complex world. Drawing parallels to reinforcement learning, Hotz posits that an intelligent agent, initially uncertain of its purpose, seeks an ideal course of action. This quest for winning is framed as a deep, mathematical problem of self-discovery and optimization, moving beyond simplistic interpretations of personal gain.

The technical foundation for this philosophy is rooted in theories by Jürgen Schmidhuber, specifically the idea of building a maximally compressive model of the world. Hotz explains that the ideal strategy involves exploring the environment in a way that maximizes the derivative of compression of past experiences. This concept serves as a "personal goal function," guiding his actions and intellectual pursuits. The ultimate aim is to understand the underlying rules of existence, or "the game," and subsequently, how to achieve victory within it.

Hotz describes his current state as operating under imperfect information and significant uncertainty regarding the true "reward function" of life. His ongoing endeavor is a dual process: simultaneously reducing this uncertainty about purpose while maximizing the perceived reward. This involves a continuous learning loop, where understanding the game's rules and the universal reward function are intertwined objectives, rather than distinct, sequential steps. It's an active, dynamic search for an inherent purpose that may either be discovered or self-assigned.

The broader implications of Hotz's perspective touch upon existential philosophy and the nature of agency. His quest for a "universal prior" or "universal reward function" transcends mere technical optimization, hinting at a deeper, almost spiritual, search for fundamental truths. It suggests that purpose is not a static given but an emergent property of an intelligent agent's interaction with its environment, constantly being refined and understood through a process of exploration, compression, and maximization.

you've said that the meaning of life is to win

I am an agent I am put into this world and I don't really know what my purpose is

the ideal thing mathematically you can go back to like Schmidt hover theories about this is to uh build a compressive model of the world

to explore the world such that your exploration function maximizes the derivative of compression of the past

I think that in the future I might be given a real purpose or I may decide this purpose myself and then at that point now I know what the game is and I know how to win

I'm still just trying to figure out what the game is

you have uh imperfect information you have a lot of uncertainty about the reward function and you're discovering it exactly what the purpose is

you're both reducing the uncertainty and maximizing at the same time

What is the universal reward function?