Bayes' Theorem Part 1: Belief in ESP

Bayes' Theorem can be used to determining how strongly evidence should affect a given belief. This methodology is explained and defended in the book The Logic of Science by E.T. Jaynes (see here). Jaynes argues that Bayes' Theorem can be derived from desirable postulates of plausible reasoning. That is, Bayes' Theorem in probability is simply a special case of a more general form of inductive logic.

In this post, I intend to introduce Bayes' Theorem by showing its application to assessing evidence in an unusual area. This post relates to the debate between William Lane Craig and Bart Ehrman found here. I think that Dr. Craig was technically correct in his critique of "Ehrman's Egregious error", but this fact does not help Dr. Craig's case as much as he would hope. But that discussion will have to wait for another post. My main focus here will assessing the question "Are ESP skeptics guilty of an anti-supernatural bias?"

This post will utilize some math. The ideas expressed here could be given without the mathematics. However, I find that putting the ideas in a mathematical framework forces me to be more disciplined in my thinking. It can make the discussion more concise, and makes my thinking more scrutible. It gives critics a very clear idea of what evidences are moving my beliefs and can sharpen critcism of my position.

Much of this post follows Chapter 5 in Jaynes' book. There is a published report of apparent telepathic events (Soal, S. G., and Bateman, F. Modern Experiments in Telepathy. New Haven: Yale University Press, 1954.) One card guessing experiment was designed to produce a probability of a correct selection of 20%. Mrs. Stewart reportedly predicted the correct card 9410 times out of 37100 trials (she was correct 25.36% of the time). This is over 25 standard deviations away from the expected value. I will try to examine how one could interpret this data in light of Bayes' Theorem.

First some background on Bayes' Theorem. (This description follows the Wikipedia article here). Bayes' Theorem often written as:
P(H | E) = P(E | H) P(H)
P(E)
Where
  • P(H | E) is the probability/plausibility of the hypothesis H in light of evidence E.
  • P(E | H) is the probability of observing Evidence E given that hypothesis H is true.
  • P(H) is the probability that a Hypothesis is true based upon background assumptions
  • P(E) is the probability of observing the evidence regardless of the truth of the hypothesis

The denominator is often rewritten using an identity is probability,
P(E) = P(E|H1)P(H1) + P(E|H2)P(H2) + . . . + P(E|Hn)P(Hn), where H1, through Hn are mutually exclusive hypothesis. In the case of two hypotheses, ~H is considered the negation of the primary hypothesis, P(~H) = 1 - P(H) and Bayes' Theorem is written as:
P(H | E) =         P(E | H) P(H)        
P(E|H) P(H) + P(E|~H)(1- P(H))
How does Bayes' theorem indicate that we should assess the likelihood that Mrs. Stewart has exhibited ESP, assuming that either ESP or random chance are the only explanations? The probability of selecting r correct cards out of n trials is given by the binomial theorem. (The formulas I used to compute the specific values P(Data|HP=p) are given at the end of the post.)

HypothesisDescription P(Data|Hypothesis)
HP=0.2 Mrs. Stuarts' results are a
result of pure chance.
2.003 × 10-139
HP=0.2536 Mrs. Stuarts' is able to predict
cards at a rate of 0.2536 which is
indicative of ESP
4.76 × 10-3

To find how much beliefs move as a result of evidence, it is necessary know where we start prior to the evidence. Suppose I am initially very skeptical of any claim to telepathic power. My a priori estimate that ESP has been genuinely demonstrated is about one in a billion, compared to the likelihood that the observed data is due to random chance. In light of these prior beliefs, I would assign P(HP=0.2536)= 10-9 and P(HP=0.2)= 1 - 10-9. Plugging these values into the formula for Bayes' Theorem gives:
P(HP=0.2536 | E) =         (4.76× 10-3) (10-9)            .
(4.76× 10-3) (10-9) + 2.003× 10-139(1- 10-9)

= 1.0 to the limits of floating point precision

Thus, with the assumption of two hypothesis, a telepathic event is nearly a certainty. (At least we know that this event is not well explained by chance). Despite these calculation, I personally do not believe that Mrs. Stewart exhibited telepathic powers. If I believed that the two offered hypothesis were the only possibility, perhaps I should be persuaded. However, this use of Bayes' Theorem ignores several possibilities that come to most peoples' minds. There is always the possibility that there was deception somewhere in the reporting chain. The researchers could have failed to report data on days the Mrs. Stewart did not do so well. Mrs. Stewart could have also noticed a reflection that the researches missed, etc.

Let's offer a third hypothesis that includes the possibility of deception by somebody in the reporting chain. I think that deception would explain the data as well as an actual telepathic event. I think reasonable priors are 1 in a billion for ESP, 1 in a thousand for deception someplace with still the most likely possibility that the events can be explained by chance. I will also assume that deception could explain the data as well as the ESP hypothesis.

HypothesisDescription P(Data|Hypothesis)
HDeceptionDeception Occurred SomewhereAs explanatory as ESP:
about 4.76× 10-3

My prior probabilities for ESP are still 1 in a billion. My prior probability that someone in a scholarly article would be the victim of (or purposely perpetuate) a fraud is about 1 in a thousand. My prior probability that the event can be explained by natural processes and not deception makes up the remainer (1 - 10-9 - 10-3). Utilizing three hypothesis changes the denominator to P(E|HP=0.2536) P(H0.2536) + P(E|HDeception) P(HDeception + P(E|HP=0.2) P(H0.2). Again, the last term is so small that underflow will occur, so I will ignore it in the following calculation. With the new hypothesis considered, the probability of ESP is now:
P(HP=0.2536 | E) =         (4.76× 10-3) (10-9)
(4.76× 10-3) (10-9) + 10-3(4.76× 10-3)

= 9.99999× 10-7

The probability of ESP is now about 1 in a million. Utilizing Bayes' Theorem with these values indicates that the probability of deception is very large (about 0.999999). When dealing with extemely unlikely event, seemingly unlikely alternatives can have a dramatic effect on the plausibility assessment.

What I have attempted to do here is essentially put the parable of "The Boy who Cried Wolf" into a Bayesian framework. It is possible for someone to tell the truth, not be believed while the people doubting the claim are entirely rational. It gives insight explaining why it is so hard to establish unusual claims on the basis of testimony. That is not to say it is impossible. Note that in my example, the plausibity of ESP moved by a factor of a thousand. However, the various possibilities of deception anywhere along the reporting chain make it very difficult to convince one of a highly unusual claim. It raises the question, "Can ESP be established on the basis of testimony?" I think it can, but the controls would need to be extensive.

This form of reasoning does capture the essence of why I do not believe most ESP claims. It is very reasonable to believe that magicians are not using supernatural powers even when you don't know how they are performing their trick. Is my rejection of Mrs. Stewarts' ESP claim justified here? Am I displaying an unjustified anti-supernatural bias?
__________________________________
To compute my results, I used the following formulas. (In other word, only read this if you want to examine the math.) The probability of selecting exactly r of n on the hypothesis the selection probability is p is given by the equation:
P(D|HP=p) =   n!   pr(1-p)n-r 
n!(n-r)!

The factorials are too large to used directly with standard floating point processing, so using the Stirling approximation for large factorials:
n! ≈ nn e-n sqrt(2 π n)

And the identity
x = exp( ln(x)) where x > 0 

after some algebra yields:
P(D|HP=p) ≈ exp{-0.5 ln(2π)+(0.5+n)ln(n)-(0.5+r)ln(r)...
+ (r-n-0.5) ln(n-r) + r ln(p)+(n-r)ln(1-p)}

This expression is amenable to floating point computations. I utilized a different approximation than Jaynes did. He utilized an "entropy" approximation that gives the same answers as my formulas do. However, the number he used in his equation 5-7 appears wrong, (this difference doesn't affect our conclusions).

Edited Aug 1 to fix various typo's

4 comments:

John W. Loftus said...

This is just an example of what's to come next. It'll be interesting, and I know you are excited to share it, Bill. It's interesting that Christians might agree with your reasoning here (I don't know) but will disagree with what you'll share later.

exapologist said...

I agree that Craig was technically correct in saying that Ehrman made a mistake in the way he put his point about probabilities. However, once we agree that posterior probability and not prior probability is the relevant sort for assessing the case for the resurrection, Ehrman would still be vindicated: the posterior probability of the resurrection is still less than 1/2. Indeed, we can push this point by adding the following piece of data that Craig fails to mention (but of which Ehrman argues at length in his book, Jesus: Apocalyptic Prophet of the New Millenium): the fact that the gospel narratives repeatedly attribute to Jesus a false prediction of the immanent end of the world. If we add this piece of data to the equation, the probability of the resurrection is certainly less than 1/2. For what is the likelihood that God would vindicate a false prophet? Hmmm... Game, set, match.

Daniel said...

Craig was borrowing the method from Swinburne. Swinburne's method was thoroughly critiqued by Mark Chu-Carroll, a math expert, at Good Math, Bad Math (scroll through comments), and also derided properly in another post.

I also had a protracted discussion about this at my freethought group blog (scroll through comments) with our philosophy prof advisor.

The major point that it comes down to, and the reviewer of Swinburne's Resurrection of God Incarnate made this same point, is:
However, I found it difficult to see why these reasons make it as likely as not that God would become incarnate if he exists. Swinburne himself discusses a couple of plausible alternatives to becoming incarnate as the best act for God to do, and readers may think of several more. He endorses a form of the principle of indifference applied to God’s actions, and thus one would expect Swinburne to conclude that the probability of God becoming incarnate (given that he exists) is about 1/n, where n is the number of equally best acts that God could do (p. 34). Given this, it is difficult to see why Swinburne concludes the probability of God becoming incarnate, if he exists, is 1/2 instead of 1/3, and many readers will assign an even lower probability.

Specifically, why does n = 2 for any of the events Swinburne lists? No convincing reasons are given, and when we consider "what would God do?" the assignment of n becomes an exercise in absurdity.

The important thing to remember with Bayes is that it's all about the assignment of probabilities. Calculating the end result using all the fancy tricks in the world doesn't add one iota of truth value to the question of how accurate your assignments of probabilities are. That's why Dembski's UPB and NFL are "written in jello". For UPB, he attempts to impose the restriction of randomness upon given systems in which it does not apply (eg chemical reactions are not "random" -- they are thermodynamically and kinetically controlled), which is the same mistake Berlinski makes in assigning probabilities in his work: these just do not reflect physics and chemistry. For NFL, he attempts to confuse and obscure the issue by complaining that natural selection may not outperform random search space algorithms. What this obscures is the fact that any selective algorithm explains how evolution works, and evolution does not need the "best" or "fastest" algorithm, simply one which functions, and corresponds to the genetic rates we observe in nature.

People like Dembski and Swinburne (and Berlinski) are perfectly capable of "plug and chug" -- the issue is whether or not what they've "plugged" is bullshit.

Bill Curry said...

Daniel,

To quote James Bernoulli (1713) regarding Bayes’ formula. “I cannot conceal the fact that in the specific application of the of these rules, I foresee many things happening which can cause one to be badly mistaken if he does not proceed cautiously…” (From Jaynes)

For example, this post gave an example in which under the same data, one could be convinced of ESP or not depending upon the alternative hypothesis considered. I understand that there are pitfalls to applying the equations, but I also think that people make the same mistakes without putting numbers to their assessments. Further I think avoiding numbers people tend to do things with their assessments that they certainly would not allow opponents to do. In other words, the pitfalls are already there and the math may help to expose them.

So even though I think Swinburne’s analysis is way off, I actually think he has shown some courage in trying to put his numbers into Bayes’ Theorem. By doing so, he has made it easier for me to see 1) how he is framing the alternative hypothesis 2) what he thinks reasonable prior beliefs should be 3) his assessment of how much the evidence should move his belief. I think that his argument is very open to criticism in all three areas. One thing that immediately jumped out to me is that he didn’t seem to assign weights to the constituent historical evidences.

So even though Swinburne’s argument is well deserving of criticism, I have to give him credit for being so open about his reasons. I wish more Christians were so open about their beliefs. I have often been told that when you weigh the cumulative case for the resurrection, you end up being a Christian. Swinburne is actually trying to utilize valid methods for weighing evidence. The Christians I know personally will not change their belief even when all their supporting evidence is undermined. If Swinburne acts as a consistent Bayesian, he has not given himself that escape route.

Jaynes implies if one’s argument cannot map to one utilizing Bayes' Theorem, one is violating at least one of Jaynes 'postulates' of plausible reasoning. I would agree that the assigning of specific numbers (probabilities) to indicate degrees of belief is subjective, and I wouldn’t try to give the results airs of numerical accuracy. But I see Bayes' Theorem mainly as a way in which fair-minded thinkers can discipline themselves.

I do agree with many of the specific criticisms you’ve offered of Swinburne’s assignment of priors, etc. But I want to be clear that I think there are lots of good reasons to proceed as he did.