Jim Keller on Moore's Law, Microprocessor Design, and First Principles Engineering

This episode features legendary microprocessor engineer Jim Keller, delving into the fundamental differences and similarities between the human brain and computers. Keller explains the abstraction layers of computer engineering, from atoms and transistors to logic gates, functional units, and instruction set architectures like x86 and ARM. He highlights that modern computers achieve high performance not through simple, sequential execution, but by fetching hundreds of instructions, computing complex dependency graphs, and executing operations deeply out-of-order, a process he terms "found parallelism" in contrast to the "given parallelism" of GPUs. He also touches on the market's demand for complex, high-performance machines over simple, slow ones.

Keller elaborates on the evolution of branch prediction, from simple last-outcome recording to sophisticated, neural network-like pattern recognition systems that achieve over 99% accuracy, enabling larger effective instruction windows. He describes the art and science of computer design as navigating complex decision trees, akin to Robert Frost's "road less taken." The discussion also covers the concept of deterministic computation, noting that while programs are designed to yield consistent results, their underlying execution paths are highly non-deterministic. He also explores the emerging trend of noisy computations in AI, questioning the necessity of absolute precision when input data is inherently noisy.

A significant portion of the conversation focuses on Keller's philosophy of engineering and organizational design. He views organizational structure as a computer architecture problem, where individual engineers are "functional units" with diverse skill sets. He emphasizes the critical distinction between following "recipes" and achieving "deep understanding," advocating for the latter to drive true innovation. Keller argues for frequent, radical refactoring and rewriting of computer architectures every 3-5 years, rather than incremental improvements, despite the short-term business fears associated with such disruptive changes. He likens this to Steve Jobs' concept of moving to a new, initially lower, but ultimately higher-potential optimization curve.

Finally, Keller addresses the persistent debate around Moore's Law. He defines it broadly as a 2x performance increase every 2-3 years, noting that predictions of its demise have been constant for decades. He asserts that Moore's Law is not dead, attributing its longevity to a continuous cascade of thousands of innovations, each with its own diminishing returns, collectively yielding exponential progress. He points to the vast remaining potential for miniaturization, noting that modern transistors (1000x1000x1000 atoms) are still orders of magnitude larger than the quantum limit (10x10x10 atoms), suggesting significant room for future advancements.

"computers are you know there's really two things there's memory and there's computation right"

"the special charm of computer engineering is there's a relatively good understanding of abstraction layers"

"the market for a simple complete clean slow computers is zero"

"humans think of serial narrative so read read a book right there's a you know there's the sends after sentence after sentence and there's paragraphs now you could diagram that imagine you diagrammed it properly and you said which sentences could be read in anti order any order without changing the meaning right"

"there's a little supercomputer inside the computer that's trying to project that calculates which way branches go"

"if you run this program a hundred times it never runs the same way twice ever and the answer it arises the same in but it gets the same answer every time it's just just them is just amazing"

"most people don't think simple enough"

"every five years you should do one from scratch"

"Moore's law was gonna die in 10 to 15 years and I thought that was true at first but then after 10 years it was gonna die in 10 to 15 years and then at one point it was gonna die in 5 years and then it went back up to ten years and at some point I decided not to worry about that particular product mastication for the rest of my life"

"a modern transistor is something like a thousand by a thousand by thousand atoms right and you get quantum effects down around two to two to ten atoms so you can imagine the transistor as small as 10 by 10 by 10 so that's a million times smaller"