Most Deep Learning Research is a Waste: Prioritizing Practicality Over Academic Trends

Jeremy Howard, an expert in deep learning, asserts that a significant portion of current deep learning research is unproductive, primarily due to misaligned incentives within academia. He argues that the pressure on scientists to publish leads them to focus on topics familiar to their peers, resulting in minor, incremental advances in highly studied areas that often lack practical utility. This academic culture, he contends, prioritizes publication volume and recognition over solving real-world problems, creating a substantial gap between theoretical research and practical application.

Howard highlights a critical distinction between academically trendy research and truly impactful areas such as transfer learning and active learning. Transfer learning, which enables high-quality AI work with fewer resources and less data, is described as a "world-changing thing" yet remains significantly under-researched in academia. Similarly, active learning, focused on optimizing human involvement in data labeling processes, is frequently reinvented by practitioners facing real-world challenges but is largely neglected in academic publications. He emphasizes that while industry and practitioners naturally gravitate towards these practical solutions, the academic world "has no reason to care about practical results."

The discussion implicitly recommends a shift towards problem-driven research, exemplified by Howard's own experience developing GLM-fit for Natural Language Processing (NLP). He created this transfer learning algorithm out of necessity for a course, aiming to teach practical skills, and it unexpectedly "smashed the state-of-the-art" on a crucial dataset. This anecdote illustrates that focusing on genuine needs can yield significant breakthroughs, even if initially pursued outside traditional academic publication pathways. The implication is that researchers, particularly junior ones, should be encouraged or enabled to pursue high-impact, practical problems rather than opting for "safe option" incremental improvements.

This critique extends beyond deep learning to science in general, suggesting a systemic issue where publication metrics overshadow societal impact. Such a system can lead to a misallocation of intellectual resources, hindering progress in critical areas that could democratize advanced AI capabilities. The conversation underscores the importance of bridging the gap between academic research and industry needs, fostering environments where practical utility is valued as highly as theoretical novelty, and potentially rethinking how research success is measured and incentivized to promote more impactful scientific contributions.

"most of the research in the deep mining world is a total waste of time"

"scientists need to be published which means they need to work on things that their peers are extremely familiar with and can recognize in advance in that area"

"there's nothing to encourage them to work on things that are practically useful"

"you get just a whole lot of research which is minor advances and stuff that's been very highly studied and has no significant practical impact"

"if we can do better at transfer learning then it's this like world-changing thing we're suddenly like lots more people can do world-class work with less resources and less data"

"everybody kind of reinvents active learning when they actually have to work in practice"

"the academic world just has no reason to care about practical results"

"I only want to teach people practical stuff and I think the only practical stuff is transfer learning and I couldn't find any examples of transfer learning and NLP so I just did it"

"smashed the state-of-the-art on one of the most important data sets in a field that I knew nothing about"

"I don't care whether I get citations or papers whatever I was right there's nothing in my life that makes that important"