On Atheists & Blind Faith, Or, False Arguments 27, 28 & 29: Why Prayer Studies Are Not Credible
Posted in Blogosphere, Faith, False Arguments, Logic, Medicine, Religion, Responses, Science, Skepticism, Thinking Critically on | 12 minutes | 46 Comments →So I locked horns with PhillyChief and John Evo, again, this time it was over the following comment from PhillyChief – who if I remember correctly – claims to be a scientifically-minded rationalist atheist:
Prayer helps no one but the one praying, providing a euphoria and calming effect, which could be comparable to ejaculating.
–PhillyChief
I felt that was an odd statement for a scientifically-minded rationalist to make, but was not surprised that it came from a sarcastic atheist who claims to be "almost always right", and so I replied,
How would you know? Where is that "demonstrable evidence" you're so fond of? Aside from being grossly unscientific, statements like the above appear contradictory alongside appeals to soft atheism as you've recently made on my site.
–cl
At this point John Evo chimed in, submitting some link that was presumably to some prayer study that he felt somehow qualified as the evidence I asked for, but some silly form asking for private information had to be completed in order to read it – and I don't want to join some organization just to refute what is really basic and obvious scientific misunderstanding. As I said multiple times in the thread, John Evo and PhillyChief are more than welcomed to cite any pertinent facts themselves.
To make a long story short, in our discussion I essentially challenged the scientific credibility of all prayer studies. Now I'm not normally a fan of absolute quantifiers, but in this case I will justify my use of the word all. In fact, if I were still screenwriting and had extra money sitting around, I would gladly offer a Randi-esque incentive of at least $10,000 to anyone who can prove the following arguments wrong: Philly's aforementioned comment is inherently unfalsifiable, and sans invocation of counterfactuals no scientifically reliable prayer study is possible.
Atheists and skeptics are rather fond of criticizing and ridiculing believers for relying on blind faith regarding their beliefs, yet I felt Philly and John Evo rely on nothing more than blind faith, anti-cl bias, or the typical knee-jerk misunderstanding of science we've rightly come to expect of a certain subset of believers. Of course sheer stupidity always remains an option, but I think both Philly and John Evo are intelligent guys, and I feel safe excluding that idea.
The "demonstrable evidence" part of my response refers to another thread where Philly moved the goalpost to "demonstrable in part" and had to equivocate pretty badly in order to save face after being called him out for an appeal to telepathy. If you've got the time, read that one, too. Much more than just another chance to roast one of my loudest critics, the errors in methodology Philly exemplified in that post actually relate directly to this one: Philly's aforementioned comment assumes there is demonstrable evidence regarding prayer studies – yet even if he were forced to equivocate again to "demonstrable in part" – he could not sustain his claim. That is the true basis of my aforementioned "10%" comment, and I can prove this beyond any reasonable doubt.
Misunderstanding me as he does on occasion, Philly replied:
No one KNOWS WITH ABSOLUTE CERTAINTY what anyone else is thinking. Here's an example: I swear I think you're a fucking asshole, cl. Do you know with absolute certainty that's what I'm thinking? …If you'd like to make a case that prayer isn't euphoric or calming, be my guest.
–PhillyChief
..to which I replied,
Why would I? Your statement is one-sided and only applies to a certain subset of people, and shows as much ignorance of sound science as the original statement cited. Point is, making claims that cannot be substantiated by evidence is a departure from rationalism, and with statements like these, you're a frequent flier. The 100% certainty argument is not the argument I'm accusing you of making. I'm accusing you of making a claim you can't even be 10% sure of. And what's this "reasonable degree?" Is that sort of like "demonstrable in part?"
–cl
Still misunderstanding me, Philly tries another angle:
Ok, then what percentage of Christians praying would I have to observe then? From your statement, all I can tell is less than all but more than 10%.
–PhillyChief
I'm fully convinced that Philly never understood this, but anyways, let's get to it.
False Argument #27: Prayer studies are scientifically credible – In order to even debate this, it seems we need working criteria of just what constitutes a credible study. First off, a credible study needs a falsifiable claim. I hinted at this much in the following comments to both of my detractors:
[Y]our statement suggests that you misunderstand the scientific method and falsifiability. …Can science reliably proceed sans a falsifiable claim? …My basis for discounting prayer studies is their inherently unscientific nature, and I've always thought such studies irrelevant in the misguided attempt to prove God. You're right that "the god thing is still unfalsifiable," and that's why you're completely out of your mind to claim that the prayer thing is falsifiable. Make sense yet? …Any scientist who claims God or prayer are amenable to the scientific method is not worth their paycheck.
–cl
Somewhere around or between those comments, John Evo makes the following statement, also an odd one for someone who claims to be a scientifically-minded rationalist. It also shows how little he's actually studied prayer studies:
If remote prayer studies had ever shown a positive impact outside of statistical fluctuations you would cite the study in any discussion about the value of prayer.
–John Evo
Really? Apparently John Evo assigns the same amount of credibility to telepathy as he does to prayer studies. I'll prove Evo wrong just as easily, as the two studies we're about to discuss both purported promising results – and you won't hear me claiming them as evidence for the value of prayer.
Now, it's never fun to argue in generalities, and that's why I challenged PhillyChief and John Evo to cite these studies they continually alluded to, yet for some reason kept shrouded in mystery not unlike Christians backed into a corner by an atheist in a discussion about evolution. As of the time of this post, citations have not been forthcoming – not even when I led by example and cited an actual study demonstrating that mere words can possibly function as effective placebos, which is only one of many potential confounders that inherently skew a certain subset of prayer studies (Thomas KB. (1987) General practice consultations: is there any point in being positive? Br Med J (Clin Res Ed). 294:1200-2. PubMed).
Now let's take a brief look at two more actual studies: The first from my own liberal backyard and published in Southern Medical Journal; the second from America's conservative heartland and published in the highly prestigious Archives of Internal Medicine.
In 1999, San Francisco cardiologist Dr. Randolph Byrd conducted a study on intercessory prayer (IP) where 393 CCU (coronary care unit) patients were randomly treated with either standard medical procedures, or standard medical procedures + daily IP from three to seven alleged "born-again Christians". The results were touted as promising, but I see so many red flags it feels more like I'm a matador in the middle of an intense bullfight on 10 hits of LSD.
Presuming them all honest, how do we know they weren't one of the majority whom the Bible describes as saying, "Lord, Lord" but not having faith? Humans can be dishonest too, you know. How do we know one or more of the "born-again Christians" wasn't actually an anti-Christian attempting to purposely skew the results? On the purely medical angle, newly-prescribed diuretics and newly-diagnosed cases of heart failure were two of Byrd's categories – and the prayer group showed fewer instances of both – but aren't diuretics routinely prescribed for heart failure, essentially making these categories measurements of the same thing? What an epistemological nightmare already, right?
Also published in 1999, at first glance the Harris Study seems methodologically superior to Byrd's, yet entails its own set of problems, and by its very nature remains vulnerable to the penultimate problems which cripple all prayer studies. The scenario was similar: 990 CCU patients in a Kansas City hospital received IP from teams of "Christian" intercessors on a completely randomized, controlled, double-blind, prospective basis, again with the control group receiving standard care and the prayer group standard care + IP.
Again, how do we know one or more of these "Christian" intercessors weren't among the many whom the Bible describes as saying, "Lord, Lord" but not having faith, or anti-Christians attempting to purposely skew the results? Further, the test was structured in a manner that reveals the researchers' own presuppositions about prayer as an agency: Requesting the intercessors to pray daily for 28 days suggests the conflation of prayer with magic. If it's true that "the Lord knows what we need even before we ask," of what value are a bunch of redundant prayers? How would we know negative results weren't God's just responses to foolish chatter, which is tantamount to lack of faith?
Now that we've briefly discussed specific difficulties with two actual studies that were both touted as promising, I'd like to discuss the general weaknesses that inescapably befall all prayer studies.
In fact, in the interest of brevity, let's start with the most crippling factor first: Regardless of how we structure our tests, we cannot reasonably control for or even quantify the confounding effects of extra-study prayer. That is, any methodological advantage we can conceive of cannot possibly eliminate or even quantify the subjects' exposure to other sources of prayer occurring outside the study, or to other types of spontaneous healing that may in fact exist. How can we know that any positive results in the prayer group were the result of the intercessors as opposed to other people praying in other places around the world that may or may not be focusing specifically on subject in the prayer group? It is entirely reasonable that other people besides the selected intercessors would be praying for the subjects in both groups – for example family members, friends, neighbors or members of their congregation – and I'm open to suggestions as to how we might eliminate or even quantify this massive problem. To those who might be wondering, "Wouldn't extra-study intercessors simply amount to a bigger sample group, effectively increasing the probability of positive results?" To them I would ask, "Is padding a study in favor of positive results scientifically credible?"
We've barely begun to discuss when and where the placebo effect becomes relevant, but that's another angle we could discuss, and frankly I see it as overkill. I've got some final objections to prayer studies in general, and then I'll conclude.
As C.S. Lewis so eloquently put it, prayer – by its very nature – is a request, not a compulsion. It logically follows that any request – also by its very nature – entails the possibility of being granted, rejected – or even postponed. The actual IP studies I've seen presume that God is like a magic genie who would, could or should simply grant any and all requests from any and all Joe's or Jane's off the street who claim to be Christian. What scientifically credible study begins with such debatable assumptions about the agency it seeks to reliably evaluate?
If someone in an hypothetical control group had cancer, and the cancer goes into remission, how do we know whether such resulted from IP vs. spontaneous remission? How do we know undiscovered phenomena analogous to spontaneous remission don't occur in other forms of cumulative or degenerative diseases like heart conditions, which were the subject in both of the aforementioned studies? How do we control for the fact that Kansas City likely dwarfs San Francisco in its population of born-again Christians, thus increasing the likelihood of confounding via extra-study prayer? How do we control for people who pray for all of those who suffer from disease and suffering? Do all prayer studies accurately control for the fact that every instance of a prayer study increases the likelihood of positive results by chance? How can we control for the fact that individual intercessors are going to have individual and varying degrees of faith? For these and many more reasons we've only begun to scrape the surface of, I submit that all prayer studies are inherently flawed and no prayer studies are scientifically credible.
Hence, False Argument #28 (That prayer has been proven or that studies with credible, promising results exist) and False Argument #29 (That prayer's inefficiency has been successfully demonstrated) both flow logically from False Argument #27, and here's the bottom line: Philly's aforementioned comment is inherently unfalsifiable, and no scientifically reliable prayer study is possible without the invocation of counterfactuals – because no matter how we structure the tests – it is impossible to successfully eliminate confounders and co-dependent variables.
I reminded PhillyChief and John Evo several times that the positive claimant retains the burden of proof. They made the positive, generic claim that prayer studies were credible, but failed to support that claim with evidence, hence that claim is irrational.
NOTE: In no way complimentary towards myself, John Evo has since changed his opinion, agreeing with me and challenging PhillyChief to take a more reasoned look at the evidence. "Ask yourself if you did any research at all when cl challenged us… You will find, as you continue your learning and growth, that the advice I gave in this message was well worth considering as it is said to you by someone who thinks highly of you. But take it as you will." John Evo to PhillyChief, May 16, 2009 6:07pm.
nal
says...If the study produces negative results, then yes.
By comparing the results between the control group and the non-control group. Both groups should have similar numbers of spontaneous remissions.
This is a good argument. Devastating, in fact.
This seems to imply that God responds more or less favorably to the request depending on the level of faith of the person doing the praying. Is this way God behaves? Do the number of people praying, assuming equal levels of faith, have any effect on God’s response? Just more questions that need to be accounted for, in a controlled test.
I wonder if a proper IP test could be conceived.
cl
says...But in the case of spontaneous remissions in the prayer group, how would we know they didn’t result from prayer? Or vice versa? How can we tell the difference between remissions that result from prayer vs. remissions that result from the natural phenomenon of spontaneous remission?
Well thank you. I wonder if a proper IP test could be conceived, too.
alexis
says...I would have to agree with cl. It seems to me that with the case studies examined in this post, there are none that convince. You cannot completely eliminate all the variables and confounders of the study, therefore you cannot have a successful and unbiased outcome. For people that expect convincing answers for everything, believing and agreeing that these studies are credible would seem like an easy out.
nal
says...cl:
How can we tell the difference between remissions that result from prayer vs. remissions that result from the natural phenomenon of spontaneous remission?
By extrapolation from conditions that don’t have spontaneous remissions, like ruptured appendices. If prayer doesn’t work on ruptured appendices, then it is reasonable to conclude that they don’t work.
cl
says...The phenomenon we refer to as spontaneous remission is only known to occur in cancer patients, at least from what I’ve gleaned from journals, personal interviews and online. That’s why the spontaneous remission part of the original argument was limited in scope specifically to cancer patients.
However, if prayer doesn’t work on ruptured appendices, I still don’t think that is reasonable ground to conclude that prayer doesn’t work – for all of the other reasons outlined above that plague prayer studies in general. Prayer is request – how might we reasonably discern between denial of request and failure of agency?
nal
says...cl:
Prayer is request – how might we reasonably discern between denial of request and failure of agency?
nal:
Is it reasonable that denial of request is 100% in those conditions wherein spontaneous remission doesn’t exist and less than 100% in those conditions wherein spontaneous remission does exist? That’s too convenient for my reasonability meter.
John Evo
says...I’m willing to concede the point that scientific studies of prayer are inherently flawed and pointless, regardless of what the individual study seems to indicate. Why any “scientist” would trouble him/herself with setting up a randomized, double-blind study with control groups, etc. is beyond me. Nevertheless, they do it. The fact that these studies (in toto) do not point to a value of prayer, I concede is irrelevant.
As Philly’s original statement was “Prayer helps no one but the one praying, providing a euphoria and calming effect, which could be comparable to ejaculating”, I would suggest that he gave a perfectly fine opinion. just as when my wife tells me that her prayer group has said many intercessory prayers and it has value to the well-being of others, it is a perfectly fine opinion.
Since there is no scientific evidence and can not be, either supporting or refuting intercessory prayer, I opt as always for the default position of “refute”. Prayer then becomes as meaningless as Bertrand Russell’s teapot.
Now, I hope you find some satisfaction and peace of mind, cl. You are self-aware enough to realize you have some personality issues that non-existent intercessory prayer is not going to help you with – right? Just sayin’…. you take care now.
nal
says...cl:
Prayer is request – how might we reasonably discern between denial of request and failure of agency?
nal:
If the agent always denies requests, that is not necessarily a failure. It may be a feature of the agent.
Spanish Inquisitor
says...Whatever happened to burden of proof? Philly’s statement has an unstated but implied qualifier built into it, that requires it to be read as follows:
“In the absence of proof to the contrary, prayer helps no one but the one praying, providing a euphoria and calming effect, which could be comparable to ejaculating.”
If you stand for the proposition that IP is quantifiably beneficial, you must be able to prove it. Until there is proof to the contrary, Philly is correct.
The rest of the post is just intellectual masturbation.
nal
says...The “devastating” comment is my sincerely held opinion.
/Been over at DD’s, reading the comments.
cl
says...Thanks, nal.
SI, you’re full of it, and that’s exactly what I’ve been asking Philly. What of the burden of proof? I don’t stand for the position the IP is quantifiably beneficial in any scientifically reliable sense and you obviously have given this thing knee-jerk treatment. Philly has claimed that hitherto un-cited “studies” are credible. That is a positive claim. Philly has failed to provide even a lick of evidence for his positive claim and instead prefers to wear a rationalist’s T-Shirt and produce his Driver’s License when asked to prove he has a penis. Masturbation indeed.
Stick up for your boy all you want, but it’s obvious to the rest of us you didn’t read close enough.
Spanish Inquisitor
says...Philly has claimed that hitherto un-cited “studies” are credible. That is a positive claim. Philly has failed to provide even a lick of evidence for his positive claim
Who’s full of it? Credibility is NOT a positive claim. Credibility is subjective. Ask any judge, read any court opinion. Higher courts don’t overrule lower courts on issues of credibility. They ALWAYS defer to the lower court on matters of credibility.
Philly has no obligation in any argument or discussion, to prove that the studies are credible. He merely has to provide them. If you reject them because you think they are incredible, that’s your prerogative, but you then must present your own evidence, via studies or otherwise, to substantiate why they are incredible. You cannot simply assert that you don’t believe them, and say it’s now a stalemate.
In any court of law (or court of opinion) with any rational judge, you lose.
Philly met his burden of proof. The burden has now shifted to the other side.
A lot of your post is interesting and thought provoking. The contention that the study of IP cannot really be subject to scientific testing has some merit to it.
However, when a Christian makes an absolute statement like “Ask god and you shall receive”, then science surely has the right to test that, and show that’s it bogus. You counter
by saying “No, no no. It’s ‘Ask god and you might receive’. That’s what god promises, and science can’t measure “might”. OK. I’ll buy that. But that’s not what Philly’s studies studied.
So stick to the argument. Don’t assert that the science is invalid because it didn’t study the right thing. Accept what it tested as a valid test of a limited proposition, and stop masturbating.
nal
says...SI:
However, when a Christian makes an absolute statement like “Ask god and you shall receive”, then science surely has the right to test that, and show that’s it bogus.
nal:
Even under the quoted definition of IP, I don’t see how the scientific study can control for outside influences. Without this control, the study is not scientifically valid.
I think there are reasonable arguments that IP is ineffective, but these “studies” are not among them. I used to quote these “studies” in arguments supporting the lack of efficacy with regards to IP, not anymore.
cl
says...I’m sorry you’re having trouble seeing. I respect reasoned disagreement, but your disagreement is not reasoned.
1) Where did I say credibility was not subjective?
2) Even if I accepted your claim that Philly saying “studies are credible” is not a positive claim – which I emphatically do not – it still demands reasonable evidence. You’re making excuses for someone who won’t even cite a single study. Ask yourself how that would stand up in a court of law.
“Philly has no obligation in any argument or discussion, to prove that the studies are credible. *He merely has to provide them.* If you reject them because you think they are incredible, that’s your prerogative, but you then must present your own evidence, via studies or otherwise, to substantiate why they are incredible. You cannot simply assert that you don’t believe them, and say it’s now a stalemate.” (emph. mine)
3) Regardless of your legal profferings, anyone who makes a positive claim and also claims to be a rationalist must provide evidence for their claim when asked. Basic rules. You deny that Philly’s made a positive claim, and you also seem to deny that I’ve presented evidence challenging both actual studies and studies in general. Weird, because nobody else seems to be having any problem.
4) I agree with you that Philly also has to provide them. To date? Nothing. Not. One.
5) I’ve already presented my own evidence. You’ve discussed none of it. For example, aren’t newly-prescribed diuretics and newly-diagnosed cases of heart failure overlapping variables? I’ve substantiated my case to the point that at least one person has reversed their opinion on the matter, and every other commenter seems to agree. Note that you’re the only dissenter here – who happens to be Philly’s pal and one of my detractors – and I’m the only one who’s not an atheist (I believe).
“In any court of law (or court of opinion) with any rational judge, you lose.”
6) No. In any court of law, I question your credibility and competency as prosecutor based on the fact that you completely misunderstood the initial accusation leading to an obfuscating and irrelevant defense of your client, and I would again challenge your team to produce a single instance of a credible study, pointing to the actual studies previously demonstrated as incredible. All eyes would then fall on you.
7) Philly didn’t meet any burden of proof. He still hasn’t even cited a single study, let alone the evidence or reasoning that the study (or any studies) was (were) credible. Again – that study X was credible is not something we get to just assume, and I know you know this, so quit trying to lawyer me.
“The contention that the study of IP cannot really be subject to scientific testing has some merit to it.”
Well thanks, SI, that was kind of a shocker actually. I’ll note that you appear to agree with me, not Philly. And I’m aware you still think I’m wrong.
“However, when a Christian makes an absolute statement like “Ask god and you shall receive”, then science surely has the right to test that, and show that’s it bogus.”
Who says that’s an absolute statement? You have to consider the entire argument from scripture before you go making challengeable extrapolations such as these. Diction? If you think your literalist approach through, you’ll quickly realize either it’s incorrect, or the Bible is. I say naive, literalist approaches to scripture are incorrect, and I already know what you think about the Bible.
What you’ve done is what most atheists and skeptics do, and it’s the same exact thing most believers do with science – make it say something it does not. Just as you noted the unspoken qualifier in Philly’s claim – which was in fact completely irrelevant to the argument I was actually making – you ought to note the unspoken qualifiers in “Ask and you shall receive.” That statement didn’t exist in a vacuum, SI.
“Don’t assert that the science is invalid because it didn’t study the right thing.”
I haven’t asserted such. The science is invalid because what it purports to study cannot be studied scientifically in any accurate or credible manner. Prayer studies test for a magic genie – not the God of the Bible. It’s nowhere near a stalemate. Prayer studies are a joke, an rhetorical exercise. Look into this stuff before you go trying to throw your legal expertise around. Working, publishing, skeptical scientists have published volumes on exactly this. Start here:
Blind Faith: The Unholy Alliance of Religion and Medicine, Richard P. Sloan Ph.D., St. Martin’s Press, 978-0-312-34881-6
Speaking of addressing arguments: How can we tell whether healings in the prayer group weren’t effected from prayers occurring outside the study? How can we tell that someone who heals in the prayer group wouldn’t have gotten better anyways? Do tell. Answers to these questions would help convince me you’re giving this more thought than PhillyChief.
cl
says...nal,
It’s too convenient for mine, too. Take the Harris study for example: Is 28 days long enough to conclude that God denied vs. postponed a request? Who’s logic is that?
Spanish Inquisitor
says...1) Where did I say credibility was not subjective?
When you insisted that Philly provide a “lick” of evidence to support the “positive claim” that his studies were credible. Evidence is objective. You want evidence of credibility. Sorry, but my opinion of your credibility, for example, is, by definition, subjective. Actually, the mere concept of evidence of credibility is nonsensical.
You’re making excuses for someone who won’t even cite a single study.
Sorry, I thought he did. The one study I’m aware of, reported last year, was all over the blogs. I figured that was a given. I’m not going to look for it, though. It’s out there. There were others, I’m given to understand, also.
You deny that Philly’s made a positive claim, and you also seem to deny that I’ve presented evidence challenging both actual studies and studies in general. Weird, because nobody else seems to be having any problem.
You’re masturbating in public again. Let’s skip down to what interests me.
Who says that’s an absolute statement?
It’s the word “shall”. Indicates that if you ask god, you always get what you ask for. That’s absolute. Sorry if I sound like a lawyer, but that’s the way it is. And you have to admit that there are lot of Christians who think that way. Maybe you don’t. Maybe your thinking is more nuanced. You sure try to convince a lot of us that it is, but in the end, you align with the fundamentalists who believe all that shit.
That one prayer study I’m familiar with, and probably the one Philly was referring to, indicated that concentrated prayers directed to asking god for something specific, something clearly defined, and monitored with a control group, actually did not work, and that there was a slightly better chance that they did more harm, than good. Sure, more study is needed. No science rests on just one experiment. But one would think that if god wanted to, perhaps, convince a few unbelievers that he existed, he’d cooperate with the study, and skew the results in his favor, even if by only a little bit. But, I guess, he’d rather we wallow in unbelief.
We are talking about god here, aren’t we?
Spanish Inquisitor
says...BTW, your HTML tags are a bit inconsistent. Sometimes they work, sometimes they don’t. The comment above had the quotes in italics, and I know how to use the italics tag, but none of them showed up.
cl
says...SI, I’m really sorry about the HTML – it frustrates me even more because I’m the one getting charged fullprice for compromised service – let that be TypePad’s negative advertising. If I had time, I’d just code my own site that worked correctly. School ends this week, maybe over summer I’ll get to it…
Would you defend a client after reading one law book? I now realize you’re only aware of one study you
refuse to cite (hey, why be specific, right?), but still, I’d love to analyze the methodological soundness of the study you mention, and the fact that you won’t even tell me what study it you refer to is bunk. Yes, you ask some good questions about God and the Bible, and I’ll gladly discuss scripture and “shall” with you – but as you say – let’s skip down to what interests me:
*Credibility* means slightly different things in science and law, SI. In science, *credibility* (as I’m using it here) refers to safeguarding against confounders and overlapping variables. Yes, whether a person *believes* a given court testimony or scientific study is credible reflects their subjective opinion. What I’m getting at is this: When scientific studies have not successfully controlled for confounders and overlapping variables, we cannot declare them credible. Therefore, my claim that my cited studies are not credible can be objectively assessed. For example, again – aren’t newly-prescribed diuretics and newly-diagnosed cases of heart failure overlapping variables? If yes, it would seem the category is not a credible indicator, correct?
Spanish Inquisitor
says...But you asked Philly to not only give you the study, but to prove that it’s credible. The study, on it’s face, makes a claim and provides its evidence. It’s up to others, yourself included, to analyze and question its credibility. Not Philly. Philly believes the study and implicitly finds it credible, or he wouldn’t refer to it. You still don’t get burden of proof, do you?
I saw he cited a study on his blog, and you were interacting with the post. How can you say he hasn’t presented a study? You mean here? If you mean he has to physically copy the URL to your blog for you to assess it, that’s just plain stupid. You’re playing games.
MS Quixote
says...Hey cl,
I don’t think Philly needs to worry about ejaculating any longer, becuase you’ve emasculated his argument :)
I tend to agree with you that prayer studies are not much more than something to pass the time with. When dealing with a supposed willful agent, there’s just no way to predict what God may or may not do. For all we know, he would frustrate the study only because it’s insulting to him. It’s not as though we’re dealing with an impersonal force such as gravity, so the test conditions are inherently faulty. It would seem that there could be a dramatic occurence that justifies the efficacy of prayer, but never a falsification: kinda like the entire Theism/Naturalism argument taken as a whole. Theism is empirically verifiable, naturalism’s not. naturalism’s falsifiable, theism’s not.
As you note, these studies may be worthwhile to debunk bellhop-in-the-sky conceptions of God, as would be common within faith Theology. I’m all for that, but for the orthodox Christian God, there just seems to be no adequate methodology to test this scientifically, unless God submits to it. You’ve noticed previously, of course, that DD’s Gospel hypothesis tracks along similar faulty lines.
Now, I would argue that your previous argument of a prayer resulting in an immediate un-decapitation would provide rather startling evidence, to me anyway, and I would hope to any rational observer.
This discussion ties in to the whole verificationism/logical positivism nonsense so prevalent with internet discussions, IMO. It strikes me as a walling off of the knowable. If you want to find out if prayer works, go do it…
Tacroy
says...I would refer you to the study mentioned in this NY Times article. This was a large, well-funded study on the effect of intercessory prayer on the incidence of complications in people who underwent a certain type of heart surgery. Basically, it demonstrates that intercessory prayer does not have a measurable effect on health outcomes, and in fact being told that people are praying for you makes it more likely that you will experience complications.
Further, because people are randomly assigned to groups, it does control for external prayer – unless you want to say that God intervened in the random assignment of people to classes in the study. If He’s willing to do that, though, why would He not simply heal enough people in the prayed for group for it to be statistically significant, instead of refusing to do so?
cl
says...MS,
Exactly. I can summarize my reasons for not being an atheist in a single word: Vindication. Defined as lack of supernatural beings and cessation of consciousness upon death, atheism is a belief that cannot be vindicated. I’m glad you noted that.
To clarify, I don’t have a problem with DD’s methodology, my claim is that his so-called Gospel Hypothesis is no hypothesis of the gospel at all.
Tacroy,
Thanks for giving me a read at least, but I still disagree. There is no possible way to control for external prayer. We cannot control those outside the control groups. Citing another study doesn’t address the concerns I’ve leveled against all prayer studies.
Tacroy
says...I see you do not understand the statistical basis of modern medical trials.
The people in the study are assigned to each group randomly. On average, this will bring the effect of background prayer to an equal level for both groups.
For instance: Assume you have 100 people. 70 of them, when tested, will exhibit the effect of background prayer. 30 of them will not. We split the people into two groups of fifty, and randomly select the members of each group. Ideally (and on average, because we chose the groups randomly), each group will be composed of 35 who are prayed for, and 15 who are not. This would mean that when we measure patient outcomes in the non-prayer group, 70% of the patients would be affected by prayer. When we ask a church to pray for all the patients in the prayer group, 100% of them would be affected by prayer. This sort of thing would be measurable; when we compare the two groups, we effectively disregard the background effect and notice only that the group which is prayed for shows a 30% increase in “good” outcomes. This is the fundamental idea behind modern medical trials. (Note that although my example has a simple discrete “affected/not affected” choice, the same logic and statistical analysis also works for qualities that are continuous.)
The more people there are in the study, the more likely it is that we will reach the previously mentioned ideal average. Out of 1200 people, the odds are quite slim that people with a high incidence of background prayer will be unfairly represented in either group.
(And of course, you either didn’t read or just didn’t comment on the fact that the other 600 people who were prayed for and were told that they were being prayed for suffered more complications on average.)
Basically, if medical trials cannot account for background prayer, they cannot account for any difference in patient physiology unless you start doing some really special pleading. If you’re willing to say that God manipulates the dice, then I refer you to what I said before:
“If He’s willing to do that, though, why would He not simply heal enough people in the prayed for group for it to be statistically significant, instead of refusing to do so?”
nal
says...cl:
External prayer is a problem only if it effects one group differently than the other group. If external prayer effects both groups identically, then I don’t see how it could be a problem.
nal
says...1) Do the number of prayers have any effect on the desired outcome?
2) Do prayers from certain individuals have a greater effect on the desired outcome?
3) Do prayers directed at a specific individual work better than general prayers?
Until these unanswerable question are resolved (which is never), a proper IP study is impossible. No one knows how IP is supposed to work, so testing it is a fool’s errand. Without knowing the IP protocol, how can one test it?
Tacroy
says...So until we know how it works perfectly, we can’t figure out whether or not it works?
I don’t buy that.
Let’s say I’m testing willow bark tea, to see if it is effective at easing pain after surgery. I take 100 patients who will be undergoing, oh, I don’t know, appendectomies. 50 of them are randomly put into the “nothing” group, and the other 50 are put into the “willow bark tea” group. After surgery, I regularly dose half the patients with willow bark tea, and the other half with some other tea I claim is willow bark, but which has no actual medical effect. I have all of the patients record their pain levels using a standardized measure (like this one). I find, none to my surprise, that on average the “willow bark tea” group experiences statistically lower pain levels than the “nothing” group. Now, there are a bunch of questions this study didn’t answer, like:
1). Does the intensity of the tea have an effect on how the pain is reduced?
2). Does willow bark from certain trees work better than willow bark from other trees?
3). Does freshly brewed willow bark tea work better than willow bark tea that’s been brewed in a giant pot all at once?
But those are all beside the point – the main thrust of my study was, “Does willow bark tea reduce pain in people who have just gone through appendectomies?”, and the answer is yes. We may not know exactly how the willow bark tea helps, but we’ve shown pretty conclusively that it does help, and from here we can start branching out and finding other areas where it helps, along with trying to find out what causes it to work so well.
In this case, the main thrust of the study was “Does intercessory prayer reduce the incidence of complications in people who have had heart surgery?”, and the answer is a definite no. We can come up with all sorts of questions about intercessory prayer and corner cases where it may or may not work – but none of that matters. We know, for certain, that it does not work in this case, and from a medical standpoint it is probably not worth further investigation – there wasn’t even a hint that it has any effect at all.
nal
says...Tacroy:
Perfectly? You, and everyone else, know zero about how IP works.
4) Assume that cl, or anyone else, prays that God not interfere in any manner in any IP study, and that God grants this request.
That is an example of an external prayer that effects both sides identically, that cannot be controlled for, and effects the inferences drawn from the study’s outcome. Ridiculous you say? You’re right. But it’s possible and that’s why the whole idea of the scientific study of IP is ridiculous.
With 4) I’d like to reinstate my “devastating” comment. Until I change my mind again.
Tacroy
says...Asking God not to heal sounds like asking salmon not to swim.
You don’t even have to include prayer in this, though: you can just come right out and say that God refuses to heal anyone He knows will be in a medical study. (The only difference between this and your #4 objection is that your objection makes God something of a Nuremberg defendant – “I was just following prayers!”)
Either way, it leaves you in a pretty bad place. You’ve got to choose one of two things: either God is willing to change His normal behaviour in order to keep us from better knowing Him, or He simply does not answer prayers.
If
He is willing to do the former, what motivation does He have in hiding from us? There is no other interpretation of the evidence, in that case; He is specifically hiding. It’s not the result of original sin, or of any of those other theological rationalizations. This serves no purpose beyond hiding.
If He does the latter, why pray at all? Meditation has the same benefits, without the false hopes. Further, God not answering prayers is more consistent with the atheist position than it is with the theistic position – something that doesn’t exist doesn’t respond.
Finally, allow me to quote myself once again:
nal
says...Tacroy:
That’s a possibility. And that possibility also makes any prayer studies useless. Many contend that God does not grant any IPs at all, and that possibility also makes any prayer studies useless. Since no one knows how God works, prayer studies are pointless.
It doesn’t leave me in a bad place at all, I’m an atheist.
Tacroy
says...Anyway, English is dumb; it doesn’t make a differentiation between “you” and “you-plural”. That was a “you-plural”, referring implicitly to the set of people who need to make the decision. You’ve already made it, so it’s easy for you – God simply doesn’t grant IPs, because He doesn’t exist.
Other people, however, cannot believe in a God Who granted IPs for the subjects of this study, because He clearly did not. They must therefore decide between a God Who explicitly hides Himself, or a God Who does not answer IPs. As far as I can tell, there are no other options.
If God does not grant IPs, then that is more validation for the atheist hypothesis; that which does not exist clearly does not grant prayers. On the other hand, a God Who exists yet refuses to use an insignificant portion of His power to ease suffering needs even more explaining.
So no, prayer studies are not pointless. They give us a clear view of what God is not doing, even though we may think He is doing those things. This forces us to re-evaluate our internal model of God, to make sure that model matches the observed reality. It’s not reality’s fault that whenever this happens, the gaps into which God can fit shrink.
Tacroy
says...Bah, I had something before the body of the post but I ended up removing it. You can disregard that “Anyway” before the “English”
cl
says...Thank you both for putting brain power into this. Confirmation is always pleasant, but in the final analysis, I don’t really care whether we agree or disagree so much as that we made each other think. It’s the contemplation and raw processing that gets me going.
Regarding external prayer (EP) – sure – we can simply assume or presuppose that EP affects both groups equally – but how is that rational or scientific at all? Then, let’s say we all somehow agreed such could be both rational and scientific – even still – we’d have conditions where distinguishing prayers healed by IP vs. prayers healed by EP becomes impossible. None of that sounds the least bit scientific to me.
Tacroy,
I understand your reasoning, and I hate to seem like I’m simply dismissing it – but I have no choice. I can agree with you that we don’t have to fully understand how something works to test whether it works. OTOH, is willow bark tea purported to be conscious? Does God fall under the category of things which proceed sans consciousness, which would make them amenable to the process of methodological naturalism?
I feel you offer a false dichotomy in your 28th and 30th comments. For example, there are other logically permissible options besides divine hiddenness and denial of request that would justify apparently negative results without precluding God.
I disagree. Saying we can know what God does and does not do from a couple of tests is like saying we can know what a person does every time they go out drinking from spending a couple of nights with them at various bars. Honestly, the so-called scientific study of IP really does seem pointless to me, and I’m not just trying to be obstinate or minimize those who disagree.
How so?
nal,
I would say, since IP is inherently unfalsifiable, prayer studies are pointless – but that’s just me. Incidentally, I had a feeling you were going to change your mind – and I’m not trying to be rude or cocky or anything like that at all – I just feel that you thoroughly understand the argument against IP studies. The ‘external prayer affects both groups equally’ assumption was just another bump in the road.
Tacroy
says...Explaining how this works was the purpose of post 23. We are not simply “agreeing” that external prayer affects both groups equally; through a bit of applied statistics, we are making it extremely likely that such external effects affect both groups equally. This is how medical studies control for things like the mold content of your walls or the pollen levels near your house.
So methodological naturalism only applies to things that act without conciousness? You better let the people working on the DSM-V know about this, they’re going to want to stop wasting time using these very methods to research and refine the criteria for diagnosing mental illnesses – criteria that, incidentally, are used to successfully treat people who have mental disorders.
As I said, I cannot think of any. Either intercessory prayer works and God simply chose to hide himself when this study was performed, or intercessory prayer does not work. The only other option I can think of is that intercessory prayers are so ineffective as a healing modality that, although they exist, they would not show up in this study. The obvious objection to that lies in the word “ineffective”: if they are that rare, they are probably indistinguishable from random noise. If you’re willing to propose that maybe God has the same efficiency at healing people as a placebo, then be my guest.
You know that they went out to various bars on those nights. We know that God did not preferentially heal anyone involved in this study. These are both knowledge of how certain beings act.
If you believe that the person normally does nothing but stay at home and argue with people online during the evening, then you know one of three things:
1. What you believed that person does is false in general
2. They changed their behaviour specifically because you were present or
3. They just randomly decided to go out.
If you hang out with them for more than one evening, the chances of #3 being true go down. If you hang out with them for 1200 evenings, then chances of #3 being true are infinitesimal.
Tacroy
says...Explaining how this works was the purpose of post 23. We are not simply “agreeing” that external prayer affects both groups equally; through a bit of applied statistics, we are making it extremely likely that such external effects affect both groups equally. This is how medical studies control for things like the mold content of your walls or the pollen levels near your house.
So methodological naturalism only applies to things that act without conciousness? You better let the people working on the DSM-V know about this, they’re going to want to stop wasting time using these very methods to research and refine the criteria for diagnosing mental illnesses – criteria that, incidentally, are used to successfully treat people who have mental disorders.
As I said, I cannot think of any. Either intercessory prayer works and God simply chose to hide himself when this study was performed, or intercessory prayer does not work. The only other option I can think of is that intercessory prayers are so ineffective as a healing modality that, although they exist, they would not show up in this study. The obvious objection to that lies in the word “ineffective”: if they are that rare, they are probably indistinguishable from random noise. If you’re willing to propose that maybe God has the same efficiency at healing people as a placebo, then be my guest.
You know that they went out to various bars on those nights. We know that God did not preferentially heal anyone involved in this study. These are both knowledge of how certain beings act.
If you believe that the person normally does nothing but stay at home and argue with people online during the evening, then you know one of three things:
1. What you believed that person does is false in general
2. They changed their behaviour specifically because you were present or
3. They just randomly decided to go out.
If you hang out with them for more than one evening, the chances of #3 being true go down. If you hang out with them for 1200 evenings, then chances of #3 being true are infinitesimal.
nal
says...cl:
The people doing the EP would have no way of knowing which of the two groups was the target group. It is therefore reasonable to assume that the EP would be directed at or not directed at each group equally. If you have a counter argument, I would be glad to hear it.
cl
says...Tacroy,
I understand the logic, and still disagree because any data we get is going to be irrecoverably unclear. Of people that get better, how are we to know they wouldn’t have gotten better anyways? How do we know those who got worse wouldn’t have gotten worse anyways? How are we to know someone else didn’t curse them? Honestly, how can we make any rational conclusion that does not rely on post hoc reasoning? Where is our necessarily demonstrable causal link – which was so important to everybody in our discussion on miracles – but seems to have lost all import here?
Again, this is a textbook false dichotomy that – like DD’s GH – completely overlooks valid options. Stop thinking in such constricting absolutes, and if you’re tempted to say, ‘What are these other options?’ then I suggest rereading the OP.
You slightly misunderstood my MN statement: Is willow bark tea purported to have mind that can choose when and when not to ease pain after surgery? If it were, what problems would that raise in making any firm conclusions as to its efficacy?
nal,
Neither would folks praying to Buddha, Allah, Shiva, Vishnu, Moroni, Satan, Beelzebub, Molech or any of the Greek Gods, right? As I said to Tacroy, I understand the logic, and still disagree because any data we get is going to be irrecoverably unclear. Of people that get better, how are we to know they wouldn’t have gotten better anyways? How do we know those who got worse wouldn’t have gotten worse anyways? How are we to know someone else didn’t curse them? Honestly, how can we make any rational conclusion that does not rely on post hoc reasoning? Where is our necessarily demonstrable causal link – which was so important to everybody in our discussion on miracles – but seems to have lost all import here?
Tacroy
says...Actually, I don’t think you do understand the logic behind randomized controlled trials. If you did, you would see how they answer all of those questions you posed. This is basic statistics. Look up the Central Limit Theorem, it might help you really understand what goes on from a mathematical perspective.
Here’s how it works, in case you don’t want to read the Wikipedia article: you can assume that the health outcomes of a given patient is a random variable that will land somewhere between “healthy” and “dead” with certain chances. When you take all the health outcomes of all the patients, their combined health outcomes will converge on a Gaussian distribution, and it gets closer as you add more patient outcomes. This is the Central Limit Theorem. The thing about the CLT is that the patient’s individual statistical chances do not change this fact. If patients A-ZZ will get better with a probability described by distributions Ad-ZZd, when you plot them together their outcomes converge on a Gaussian distribution. All of the questions you’re asking are basically perturbations of an individual patient’s probability distribution, which does not affect the overall distribution. The only way in which one of your objections would affect the overall distribution is if it was somehow preferentially applied to one group and not the other; that’s the sort of thing that would confound an RCT.
Further, one of the interesting things about an RCT is that you don’t need a demonstratable causual link – that’s why people keep on doing RCTs on acupuncture, for instance. There’s absolutely no evidence for anything like “qi” or meridians, and yet a sufficiently large RCT can still detect that the procedure works no better than placebo. An RCT can measure an effect even though we don’t know how or why or even whether or not it works. Indeed, there’s some people in the medical community right now who are calling for a return to “evidence based” medicine, because so many researchers perform RCTs on things that we have no reason to believe would work.
It’s only a false trichotomy (I did mention a third choice immediately after the bit you quoted) if there’s more than three choices. What’s the fourth one? Keep in mind that for it to be a real fourth choice, it can’t boil down to one of the first three; for instance, claiming that Satan opposes God’s healing reduces to the third option – that for whatever reason, God is so ineffective a healer that this RCT could not detect His work.
Doesn’t matter. Some treatments are more effective in some people, for whatever reason. This can be detected. (for instance, reducing the estrogen in women who have had certain types of breast cancer will reduce the chances that the cancer will come back)
If the willow-bark tea had a mind of its own, and only chose to reduce the pain in people who it liked, we could detect this, and figure out what characteristics it likes. We can also assume that out of a population of 1200, the willow-bark tea would like at least a few people – and if it doesn’t, it’s ineffective as a pain relief agent. If it turns out that willow-bark tea doesn’t like people who are in studies, then it’s just trying to hide from us.
cl
says...Sn = X1 + … + Xn is a fairly straight forward and undeniable proposition. Is there anything I can do other than agree with you that would convince you that I do understand the logic behind RCT’s? It seems to me that you’ll continue to claim I misunderstand the logic until I concede your point. Is that the case?
So now we can detect who God likes? Even if I agreed we could do so with God – which I do not – the conclusion that willow-bark tea either doesn’t work or was hiding wouldn’t be as sustainable then, would it?
IMO, you jump to conclusions. Why even do science anymore with all the assumptions you suggest we take for granted? Lastly, the other options I alluded to are contained in the OP.
Tacroy
says...Of course there is; you could provide a concrete objection that is actually valid. “Maybe someone was cursing one of the patients” is not valid; “maybe someone was cursing the patients that were in the ‘prayed-for’ arm of the study, and this curse somehow exactly cancelled out the effect God would have” is a more valid, though monumentally improbable, objection. It still doesn’t account for the fact that 59% of the people in the “prayed for and know about it” arm of the study experienced complications – as opposed to 51% and 52% of the people in the “prayed for and don’t know about it” and the “not prayed for and don’t know about it” arms of the study – but it’s closer to being a reasonable objection.
(Also, as a side note: if you want to quote a theorem to prove you understand it, don’t choose one of the minor definitions – in the definition of the CLT, Wikipedia lets Sn = X1 + .. + Xn only because it makes the definition of Zn more concise)
Yes indeed, we could detect who God prefers to heal with a large enough study, as long as he heals enough people that this effect is distinguishable from pure blind luck. However, it seems He likes very few people; whatever healing powers He possesses never seem to make a difference in the studies we’ve performed so far – after all, religion is almost never a confounding
variable in medical studies, even the enormously huge ones, unless it’s something like “this sect refuses blood transfers and so has a higher mortality rate”. The only conclusion we can come to, therefore, is that if God is healing people, He either does so very inefficiently, or withholds His healing powers from people in studies.
(as another side note, we already know who God doesn’t like – amputees.)
Perhaps I do jump to conclusions with my assumptions – and yet, those same sorts of assumptions that led to over the counter Claritin, Ibuprofen and various other medical interventions. If this discussion were not about prayer, but were instead about homeopathic remedies, would you be so quick to say “Assumptions!” when I say that we can assume a homeopathic remedy that shows no effect on a group of 1200 people when compared to a placebo is either ineffective as a remedy, or does not work on people in studies?
Oh but I asked for options that don’t boil down to one of “God’s hiding”, “God’s ineffective” or “God doesn’t answer IPs”. Let’s see if any of those other options in the OP fall into these categories, shall we?
“God doesn’t answer IPs” – only real true Christians can get results, and we all know how many of those there are. Further, the study I referred you to asked entire churches to pray. Are you saying that there was not at least one faithful Christian in those congregations?
“God is ineffectual” – after all, if an “anti-Christian” (I don’t even know where you got that term) can keep God from healing, then He’s not much of an omnipotent being. I bet there’s at least one really demented person out there who continuously ask God to not heal anyone – so clearly, He can never heal at all.
Ooh, this one is a trifecta – “God is hiding”, because if we ask Him to do things He won’t do them; “God doesn’t grant IPs”, because He already knows what we need; and “God is ineffective”, because repeated prayers come to His ears as foolish chatter – and who hasn’t prayed over and over again for something they really want?
So here’s what it boils down to: you don’t have any sort of real objections to randomized controlled trials; you also don’t have any options to describe God’s behaviour in these prayer studies besides “hiding”, “ineffective”, or “doesn’t grant IPs”.
Keep in mind that it is an indisputable fact that out of 1200 people, the half that was prayed for did not experience fewer complications than the half that was not. You can provide whatever rationalizations you want, but that does not change the underlying truth: prayer did not play a significant role in the health outcomes of those people.
nal
says...cl:
Know? You don’t know. What you get is probabilistic outcome. Depending on the variables in the test, you could say that you are 99% (for example) confident that IP did/didn’t have an effect.
Isn’t that what the tests are designed to find, whether IP has any effect?
cl
says...Tacroy / nal,
Check back in a day or two. I’ve been pretty darn busy over at DD’s today, and now I’m too burned out to deal with comments on my own blog. I want to give your objections a fair hearing, so I’m going to wait until my mind is refreshed a bit. Thanks for sticking with this.
cl
says...Tacroy / nal,
Check back in a day or two. I’ve been pretty darn busy over at DD’s today, and now I’m too burned out to deal with comments on my own blog. I want to give your objections a fair hearing, so I’m going to wait until my mind is refreshed a bit. Thanks for sticking with this.
John Morales
says...Datum: a study that apparently is not credible to cl, from Harvard Medical School, Boston.
http://www.ncbi.nlm.nih.gov/pubmed/16569567
cl
says...Thanks John, but no need to assume. I’ll give it a look to determine whether or not I think the linked study is credible in the future. Kinda busy on other things right now.
John Morales
says...No worries, no hurry.
BTW, I assumed you would not find it credible based on the post title including “Why Prayer Studies Are Not Credible”.
This is a prayer study.
For more detail, see the manuscript.
(Yes, it was funded via the John Templeton Foundation)
cl
says...John,
It would be intellectually dishonest to deny that a prayer study might actually have overcome the difficulties I claim render all prayer studies incredible. If that’s the case, I’ll be happy to make that concession.