Statistical Inference Should Use All Available Information

In some sense this post is a follow up from my previous comments on so-called “frequentist pursuit”. Recall that in several previous posts I’ve made it clear that the Objective Bayesian (OB) often makes use of frequentist information in their inferences. In doing so, the OB is charged by the snarky frequentist, “If you’re doing frequentist inference anyway, you should just be a frequentist!”

Of course, the OB isn’t doing frequentist inference even if the OB is interested in certain frequentist properties such as coverage. That is, the OB isn’t performing NHST rejection tests nor are they doing Fisherian maximum likelihood inference. The OB, instead, takes the position that their inference should be maximally informed by all available information when performing their inference.

Both subjectivist Bayesians and frequentists make the mistake of seeing the inferences of each other as being largely irrelevant to their view. Frequentists see Bayesian inferences as being hopelessly and arbitrarily biased (if interpreted frequentist-ly) and subjectivist Bayesians see frequentist inference as incoherent, exploitable, and as answering the wrong questions in the first place even if interpreted Bayesian-ly. Seen in this way, they have no choice but to leave each other’s inferences on the table as irreconcilable.

In my mind, the Objective Bayesian, however, does not see these as being irreconcilable. The OB is able to do this for a few reasons:

1. Unlike the subjectivist Bayesian who elevates Bayes’ eponymous rule to a near-inviolable rule of inference, the OB recognizes that Bayes’ Rule is itself an expression of a more general minimum-divergence inference rule and that there are conditions under which the diachronic application of Bayes’ Rule is incorrect and the more general minimum-divergence inference is correct.

2. The more general minimum-divergence inference method, while generally in accord with Bayes’ eponymous rule, better allows for incorporation of constraints on inference in general.

3. Frequentist information can be understood as expressing additional constraints on rational degrees of belief, in addition to considerations of probabilistic coherence. This is what the OB means by Calibration.

So, the even though Bayes’ Rule doesn’t have any readily available way to incorporate a frequentist determination that \(P[a < x < b] >= 0.9\), the OB can then turn to more general minimum-divergence methods in order to weave posterior distributions that meet this constraint.

That is, OB posterior distributions are not only consistent with subjectivist information, as introduced via Bayes’ Rule, but also with complementary frequentist information introduced via minimum-divergence optimization. As pointed out in previous posts, this produces posterior distributions that avoid various otherwise troublesome sure-loss scenarios that the subjectivist Bayesian and frequentist are subject to.

I’m of the opinion that inference methods that can make principled use of the most information are generally better and are to be preferred. Under that admittedly casual criterion, OB is superior to both traditional frequentist methods and subjectivist Bayesian methods, allowing for empirical coverage guarantees that are probabilistically consistent and updateable as information is available (calculated foundationally, as Williamson prescribes). If any other statistical methodologies arise that produce inferentially relevant facts, any responsible statistical inference should incorporate those facts or be recognized as incomplete, including OB.

All this said, I’ll reiterate a previous comment that I think holds true: A Bayesian can incorporate frequentist information by calibrating their posteriors as the Objective Bayesian does, but a frequentist cannot make use of Bayesian information. That is, the bridge from probabilities-as-frequencies to probabilities-as-degrees-of-belief goes one way. If this is true, then the only responsible inference methods must ultimately be Bayesian.