John Carmack on Programming Philosophy, Game Development Evolution, VR, and the Path to AGI

John Carmack, a legendary programmer and co-founder of id Software, delves into his lifelong passion for programming, from his first "10 print John Carmack I" on a TRS-80 to his current work in AI. He recounts the early days of game development, highlighting how groundbreaking experiences like the precursor to Doom evoked visceral reactions in players, demonstrating the power of 3D navigation and immersive graphics. Carmack emphasizes that early game development was heavily constrained by hardware limitations, necessitating ingenious "hacks" and deep optimization, such as his Apple II scrolling technique, to achieve desired effects. This era fostered a culture where programmers had to operate at the "limits of the system" to make ideas a reality, a skill he notes is still crucial in demanding fields like virtual reality.

The conversation explores the philosophy and practicalities of programming languages. Carmack discusses the historical context of `goto` statements, acknowledging their necessity in early BASIC but their general avoidance in modern structured programming, though he admits rare, specific uses. He shares his preferences, using Python for AI/ML despite its performance pitfalls for loops, and C++ for "serious programming," favoring a "C-flavored" approach over complex template metaprogramming. His experience with functional languages like Lisp and Haskell influenced his C++ style, particularly in valuing immutable state and considering a program's entire lifespan, including maintainability and handover between engineers.

Carmack offers nuanced perspectives on language design and adoption. He contrasts the simplicity and widespread utility of C (despite its memory safety issues) and Go with the malleability but potential long-term complexity of Lisp. He acknowledges JavaScript's phenomenal engineering and ubiquity, despite its perceived sub-optimality for large-scale applications without strong type systems. A key distinction he makes is the value of garbage collection, arguing it's a net positive for most software by preventing common memory vulnerabilities, even if some elite programmers view it as fostering "laziness."

A recurring theme is the trade-off between efficiency and overall value. Carmack stresses the importance of understanding theoretical "speed of light" performance but cautions against over-optimization when it doesn't significantly enhance user experience or system value. He uses the analogy of biological evolution, which is inherently inefficient but produces complex, adaptive systems, to describe the organic, often messy, development of large software ecosystems like JavaScript. He also draws parallels to other fields, like nuclear energy, where sacrificing efficiency for cost-effectiveness or simplicity could lead to radical innovation, underscoring that the "best" approach is often context-dependent and focused on delivering the most impact.

games just didn't do that you know the games were the God's eye view you were a little invested in your little guy you can be like you know happy or sad when things happen but you just did not get that kind of startle reaction your face something in the back of your brain some reptile brain thing is just going oh something just happened and that was one of those early points where it's like yeah this is going to make a difference this is going to be powerful and it's going to matter

go-tos aren't poisonous sometimes they're the right thing to do uh usually it's because there's a a language feature missing like nested breakes or something where it's it can sometimes be better to do a go-to cleanup or go-to error rather than having multiple Flags multiple if statements littered throughout things but but it is rare

it felt like this magical thing to me this idea that the computer would just do exactly what you told it to I mean and there's a little bit of the genie monkeys paw sort of issues there where you'd better be really really careful with what you're telling it to do but it wasn't going to back talk you it wasn't going to have a different point of view it was going to carry out what you told it to do and if you had the right commands you could make it do these pretty magical things

so much of kind of the first couple decades of the programming work that I did was largely figuring out how to do something that everybody knows how they want it to to happen it just has to happen two to 10 times faster than sort of the straightforward way of doing things would make it happen

if you make a loop you know a triple nested Loop and start doing operations in Python you can be thousands to potentially a million times slower than a proper GPU tensor operation and these are staggering numbers

it's not the writing of the program initially it's the whole life span of the program and that's when it's not necessarily just how fast you wrote it or how how fast it operates but it's how can it Bend and adapt as situations change

garbage collection is unequivocably a good thing for most programs to be written in... there is so much objective data on the the vulnerabilities that have happened in CN C++ programs sometimes written by the best programmers in the world it's like nobody is good enough to avoid ever shooting themselves in the foot with that

so much of my systems design and systems analysis goes around you should understand what the speed of light is like what would be the best you could possibly do here and it sounds horrible but in a lot of cases you can be a thousand times off your speed of light uh velocity for something and it's still be okay

the difference between operating in 1 second and 750 milliseconds uh not that huge you'll see it in webpage statistics but most of the usability stuff not that great but if you get down to 50 milliseconds then all of a sudden this just feels amazing

in the earliest days of programming when you had finite you could count the bites that you had to work on this you had all the the kind of hackers playing code golf to be one less instruction than the other person's multiply routine to kind of get through and and it was so perfectly crafted it was a crystal piece of artwork when you had a program because there just were not that many you couldn't afford to be lazy in different ways