The Foundational Duality of Statistics: Inference, Decision Theory, and the Bayesian-Frequentist Divide

This podcast clip features Michael I. Jordan defining statistics as a discipline residing between mathematics, science, and technology, focused on principles for making reliable inferences and decisions with quantifiable error probabilities. He traces its history back 250 years, initially known as "inverse probability" to infer underlying mechanisms from observed outcomes, contrasting with probability's origin in predicting outcomes from known states. The term "statistics" was coined by Laplace during Napoleon's era for state data analysis, but the field was formalized in the 1930s, deeply intertwined with game theory and decision theory, emphasizing loss functions, probability models, and risk in decision-making.

it's somewhere between math and science and technology it's somewhere in that convex hull

statistics was called inverse probability that was the name of the field

it's all about here's a here's not just data and you analyze it here's a loss function here's what you care about here's the question you're trying to ask here is a probability model and here's the risk you will face if you make certain decisions

there is Bayesian ways of thinking and frequentist and they are different they they all they sometimes become sort of the same in practice but they're Fazal way different

it is very much like wave in particle duality and that is something you have to kind of get used to in the field

the frequentist says I'm gonna look at the X the data and I've got average over the distribution so I take the expectation loss under X theta is held fixed

the Bayesian perspective says well no I'm gonna look at the other argument at the loss function the theta part... I could have my own personal probability for what it is

empirical Bayes sort of starts with the Bayesian framework it's it's kind of arguably philosophically more you know reasonable and kosher write down a bunch of the math that kind of flows from that and then realize there's a bunch of things you don't know because it's the real world

False Discovery Rate which is you know you're making not just one hypothesis test or making one decision you're making a whole bag of them

the Bayesian goes the other direction from the data back to the state of nature and that's actually what false discovery rate is