Ethics and the Statistical Use of Prior Information
1. The choice to not use all available information
Debates about statistical foundations can be annoying to practitioners but are important in that foundational claims are used to make general recommendations for practice.
Some content is only viewable by ASA Members. Please login or become an ASA member to gain access.
Gelman’s view on the proper entry of background knowledge is actually in sync with Sir David Cox’s position. Beyond what might be extracted from the snippet from the highly informal, recorded (Cox-Mayo) exchange to which Gelman refers (p. 3), Cox has done at least as much as anyone else I can think of to show us how we might generate, systematize, and organize background information, and how to establish the criteria appropriate for evaluating such information. But even in our informal conversation, Cox is clear about using prior knowledge in analyzing the data.
MAYO: But they should use existing knowledge.
COX: Knowledge yes. Prior knowledge will go into constructing the model in the first place or even asking the question or even finding it at all interesting. It’s not evidence that should be used if let’s say a group of surgeons claim we are very, very strongly convinced, maybe to probability 0.99, that this surgical procedure works and is good for patients, without inquiring where the 0.99 came from. It’s a very dangerous line of argument. But not unknown.
When Cox says in our conversation:
“In fact you have very clever ways of making sure that your analysis is valid even if the prior information is totally wrong. If you use the wrong prior information you just got an inefficient design, that’s all” (p. 105),
he is using “prior information” to refer to various uncorroborated beliefs, conjectures, hunches, whether or not they are influenced by policy, ethical, or other values, and for which no evidence is given. That is why he mentions the possibility that “the prior information is totally wrong”.
Ironically, it is Gelman who declares in this article that “A Bayesian wants everybody else to be a non-Bayesian”:
“No funny stuff, no posterior distributions, just the likelihood. . . . I don’t want everybody coming to me with their posterior distribution—I’d just have to divide away their prior distributions before getting to my own analysis.”
I discuss this in a series of posts on my blog beginning http://errorstatistics.com/2012/09/12/u-phil-how-should-prior-information-enter-in-statistical-inference/
Our exchange may be found here: http://www.rmm-journal.de/downloads/Article_Cox_Mayo.pdf