The State's Entrepreneurial Role in Driving Technological Innovation, Beyond Market Failure Theory

This podcast episode critically examines the traditional neoclassical economic concept of "market failure" as the primary justification for state intervention in the innovation economy. It argues that while welfare economics frames state action as correcting market inefficiencies, historical evidence and a deeper theoretical understanding reveal that politically legitimate missions, particularly national security and national development, have been the true drivers of state-promoted technological innovation at scale. The discussion begins by contrasting the seminal work of institutionalist Richard Nelson and mathematical economist Kenneth Arrow, both of whom identified sources of market failure (positive externalities, inappropriability, uncertainty, increasing returns) that theoretically justify state funding for basic research and R&D. However, Nelson himself later rejected market failure as a "disastrous way to think about the role of the state," advocating for an evolutionary economics perspective.

The episode delves into the nuances of market failure, exploring Arrow's conditions for general market equilibrium, which crucially require complete markets that do not exist in reality, especially for information. It highlights the peculiar economics of information, where efficiency argues for free distribution but eliminates production incentives, and customers cannot assess value until purchased. The discussion also introduces "Pastor's Quadrant" to challenge the clear line between basic and applied research and "innovator's dilemma" to explain why established firms struggle with exploratory R&D. Empirical work is presented, quantifying the significant gap between the social return to R&D (almost three times the private return) and the value captured by private companies, identifying both positive knowledge spillovers and negative product market spillovers.

Moving from theory and empirics to political history, the episode provides compelling examples of state-driven innovation. The British state's demand for guns during the First Industrial Revolution pulled the artisanal economy into radical industrial innovations. Similarly, the U.S. Army's armories perfected interchangeable parts, laying the foundation for mass production across various industries. For the digital revolution, the U.S. Department of Defense was a decisive actor, not only funding R&D but also acting as a creative and collaborative customer, and crucially, "crowding in" private R&D rather than crowding it out. This historical perspective reveals that national security, rather than abstract market failure, consistently legitimized and drove state investment.

Further historical examples include Alexander Hamilton's Report on Manufactures, Henry Clay's American System, federal land grants for railroads and universities (Morrill Act), and protective tariffs, all aimed at national development and allowing infant industries to mature. The episode concludes by emphasizing that the Korean War, not Vannevar Bush's "Science, The Endless Frontier" report, was the true catalyst for sustained post-WWII state investment in science and technology. It advocates for thinking in terms of "national symptoms of innovation" rather than solely market failure, offering a broader framework to understand the state's complex and often entrepreneurial role in fostering innovation and integrating state interventions with private sector dynamics.

"market failure as defined in neoclassical welfare economics is a disastrous way to think about the role of the state"

"Pastor's quadrant has entered the language of the economics of innovation"

"our economic system does not possess markets for commodity options"

"ironically correcting market failure depends on imperfect competition but monopoly rents not only fund innovation they reduce the incentive to innovate"

"the marginal social return to research and development that they identify is almost three times the marginal private return"

"leaving warfare and armaments out of the history of U.S. industry is like the proverbial production of Hamlet without the prince"

"state research and development crowded in private research and development as a complement and with the incentive for further private research"

"if there had been no Korean War it's very hard if not impossible to prove a counterfactual"