No, Statistical Models Are Not Just Representations of Objective Reality

A persistent misconception I encounter - especially among frequentists who conflate performance guarantees with epistemic justification - is the claim that statistical models are “convenient yet perhaps imperfect representations of objective reality."¹ This framing obscures a deeper conceptual split in how different schools of inference understand what modeling is for. Here’s the simple point I want to press: the notion that models are “just representations of the world” bakes in assumptions that Bayesians do not share - and for good reason.

Some more reflective frequentists acknowledge that models are idealizations and incorporate tools like severity testing or model checking. But what I’m criticizing here is a proceduralist frequentism: one that treats the validity of a method as guaranteed by its nominal operating characteristics, often under the guise of “objectivity.” Frequentists appeal to long-run error control, like coverage or type I error, to justify inference. The idea is that if you behave according to certain rules, you won’t be “wrong” very often. This is appealing because it looks like a kind of objectivity - guarantees detached from the particular data you happened to observe. But the central confusion here is conflating good long-run performance with good epistemic justification - two notions that differ both in aim and scope. A bet that wins in the long run may still be irrational for the particular evidence at hand.

This belief that statistical models aim to mirror the world leads to common misreadings of Bayesian inference. When a Bayesian assigns a prior distribution to a parameter, they’re not claiming the world is randomizing that value on each repetition. They’re expressing uncertainty about a fixed but unknown quantity. Confusing this for an attempt to “model beliefs instead of reality” reflects a misunderstanding of the epistemic role of modeling in inference.

A Bayesian model represents a state of knowledge - not necessarily a generative process. The prior, likelihood, and posterior are tools for normatively updating beliefs in light of data. We don’t assume the world samples θ from a prior. Rather, we treat θ as fixed but unknown, and use the prior to express our uncertainty about its value. Recognizing this dissolves a related objection: the idea that Bayesian distributions must be treated like sampling distributions. They’re not designed for that. Critiquing them as if they should be is a category error.²

Take a familiar case: the Beta-Bernoulli model for a coin with unknown bias. The coin has some fixed probability of heads. The prior doesn’t imply that this bias is random - it encodes our uncertainty about it. As we observe data, the posterior concentrates, and the model behaves like a frequentist estimator. But the entire point of the Bayesian formulation is to allow principled reasoning under epistemic uncertainty, especially before asymptotics kick in.

In response, some frequentists appeal to long-run performance - expected loss, calibration, etc. - as though Bayesian inference is indifferent to performance. But Bayesians care about calibration too; they just calibrate beliefs to data, rather than tying inference to hypothetical ensembles of procedures.

That is: Bayesians don’t discard performance - they just don’t define rational belief in terms of it. Even if frequentist procedures aren’t intended to represent belief states, they still carry epistemic implications. They aim to justify scientific claims via controlled error rates. But justification, from a Bayesian perspective, demands more than performance - it demands that model assumptions connect meaningfully to the world.

Of course, Bayesianism has its challenges - prior specification being the most familiar. But at least the framework makes transparent what frequentism often obscures: that all inference rests on epistemically-laden assumptions.

Misrepresenting Bayesian modeling as belief in ontologically random parameters - or critiquing it as though it were - is a sign one hasn’t done the conceptual work. It’s not enough to cite performance results when the conversation is about epistemology. As long as these strawman portrayals persist - where Bayesian models are treated as naive metaphysics or confused randomness - they’ll obscure the real conceptual divide.

That confusion keeps alive the myth that Bayesianism is metaphysically extravagant. In fact, it’s epistemically modest. Statistical modeling is about inference under uncertainty. And some of us are content to do that carefully, one coherent update at a time.

¹ LinkedIn comment

² More examples of this same confusion at work: LinkedIn post, LinkedIn post, LinkedIn post