Now Taking Suggestions For A Credible Debate Scoring System
Posted in Debate, Philosophy on | 1 minute | 23 Comments →
Well this whole PZ Myers Memorial Debate sure sparked quite the fiasco, but it’s really got me thinking. A few commenters both here and at VoxWorld have tossed out some pretty decent ideas as far as judging debates are concerned. If you were to judge a debate, what would you look for? What sort of things would you award or penalize? Have you seen any successful debate scoring systems before? What sort of scoring do you think would be fair? Based on what we’ve seen in the recent judging, what sort of things would you advise for or against? Where did the judges do well? Where could we have done better? Was there anything you wanted to see, but didn’t? Let’s see what we can come up with.
Heuristics
says...For every round award points for arguments and counter arguments.
If an argument is given in a round and a judge finds the argument to be relevant to the debate and coherent a point should be given. Likewise if an argument is given but it is incoherent or irrelevant a point should not be given. Notice here that the judge should still give a point even if the judge does not agree with the premises or the conclusion of the argument.
Restrict the maximum number of points that can be given out to 3 per round (so that there is no flooding of arguments). Do the same for counter arguments.
FORMAT:
A debate topic is chosen (and the relevance of the arguments is judged relative to it).
In round 1 each person gives arguments where they argue their case.
in round 2 they give counter arguments to the opening case of their opponent.
in round 3 they give counter-counter arguments
If this form is broken no points should be given. Example: if in round 3 new arguments are introduced this should not be given points by the judges, it should just be ignored.
After 3 rounds bonus points should be given if the opponent dropped an argument (they failed to give counter arguments). This should not be per-judge, the judges should award this as a group.
In the end, the one with most points wins (the point of giving out points per argument is to avoid the issue of the judge having a personal favorite).
Alrenous
says...Common sense is correct in this case. The integrity-having scoring system is one where each correct point gets +1, each unsupported or irrelevant point gets 0, and sophistry, fallacies, and other corruptions get -1.
Though come to think I don’t want to conflate debate integrity with logical integrity. So I’m going to use a 2D system. One for points and one for rating how epistemically reliable the debate is likely to be.
Crude
says...I vote for: None.
In fact, put me down as voting against debates. Conversations are superior, happen more rarely, and obtain only in the best situations.
Too many goddamn debates nowadays.
TheistDude
says...cl, not being a vox apologist, but, you do know he never said his scoring system should be adopted, and was merely saying how the debate turned out so far, right?
cl
says...Heuristics,
My original Round One write up experimented with a scoring system pretty similar to what you layed out. I’d propose limiting opening arguments to three. This would prevent flooding of arguments and also allow for more flexibility, point-wise, if necessary. Would you group all fallacies together and give each a “-1” or would penalties vary by severity? If the latter, which types of fallacies do you think merit more penalty than others?
Alrenous,
I tend to agree, although I’m open to awarding more than a single point for rewards and penalties. I think that might allow for a more nuanced score. Any thoughts?
Crude,
I think there are legitimate pluses and minuses for each. What I like about conversation is that it happens more or less spontaneously and tends to wander, which means one has to be quick on one’s feet (as opposed to having days or weeks to prepare formal arguments). OTOH, conversations don’t have as much accountability as debates, and debates tend to be a little more rigorous. So, I think both formats have advantages that can potentially yield breakthroughs, or, if nothing else, interesting conversations and debates. Would you ever a judge a debate?
TheistDude,
No worries. It takes more than simple questioning to deserve that label, i.e., deleting any and all comments that don’t portray Vox Day in a positive light. Yes, I realize he never actually said “this is the scoring system” but I think he compromised the integrity of the whole thing and really, as a contestant, I think he should have left all mention of scoring to the judges. Or at least spoke with us before announcing his scoring system. Or, he could’ve waited until after the debate to say, “Hey, here’s how I scored it.” None of those things would have bothered me. It just kinda cheapened the whole thing IMHO, to the point where I didn’t really want anything to do with it anymore. It’s no big deal, they’ll get Markku and move right along.
If you have any suggestions regarding the judging questions above I’d love to hear your take.
Alrenous
says...Come to think, I’m more on Crude’s side in that debates tend to cost more than they’re worth. Nonetheless, a good scoring system interests me epistemically.
I was thinking about non-sequiturs. It’s an honest mistake, so maybe one point for that, pushing really good points up to two?
Also, there’s suggestive arguments vs. conclusive arguments and egregious fallacies vs. natural errors.
But I my system gives the non-sequitur a zero. One point for a supported argument, minus one point for a fallacy.
There’s always going to be fudge and judgment. Basically every debater thinks they were the winner; else they’d have to change their mind. So if you really think a point deserves extra points, break it down into sub-points and award one for each. Exploit the judgment factor instead of fighting it.
And yeah, it’s by this system that both Vox and DS are way in the negatives. I think Vox is in a deeper hole but I’ma do a rough tally and I’m sure it’ll surprise me in some way.
TheistDude
says...“If you have any suggestions regarding the judging questions above I’d love to hear your take.”
A scoring system similar to what they do in Boxing matches would be the best. Here, at least, people can see not just who won a round, but how much better the winner argued his case.
Crude
says...cl,
What I like about conversation is that it happens more or less spontaneously and tends to wander, which means one has to be quick on one’s feet (as opposed to having days or weeks to prepare formal arguments). OTOH, conversations don’t have as much accountability as debates, and debates tend to be a little more rigorous
Well, a lot of conversations are just debates. In fact, especially on these topics, having an actual conversation is downright rare. The Cult of Gnu doesn’t “do” conversation, nor do a sizable number of Christians. And the moment you call something a debate, you pretty much rule out the ability to say “Hey, that’s a good point.” or pursue a line of thought.
I’d be willing to agree that in some contexts a debate can contribute something. One side-effect is you can at least suss out guys who completely melt-down when challenged. (Like… man, don’t say his name – he shows up, like Beetlejuice. That jackass in the hat.) But, especially on the internet, and especially on these topics, it’s done to death. In my ever humble opinion.
Would you ever a judge a debate?
Probably. Would depend on the debate, the participants, and the format. I’d even debate, but I’d prefer conversation and some back and forth over the debate format every time. And I’d only debate someone I personally had intellectual respect for.
Peter Hurford
says...Personally, I like the format done in The Wanchick-Carrier Debate at Infidels.org.
Basically, the two debaters agree on a joint statement about the precise topic they are debating, and the precise statements each side will be defending.
Then a side (or both sides) creates an opening statement outlining arguments that demonstrate their position. After this, no new arguments may be made. The remaining is a rebuttal to opening, counterrebuttal, and counter-counterrebuttal on the arguments made during the opening statements.
On an argument by argument basis, judges say who they think won that specific argument, and points are averaged among all judges.
cl
says...Peter,
Thanks for your input. Check your email, if you haven’t already…
joseph
says...Thanks for the link Peter Hurford.
TruthOverfaith
says...As long as our virgin born, blood sacrificing, never-had-an-erection Lord and Savior is glorified, I don’t care what the scoring system is.
Praise Jeeezus!!!!
TheistDude
says...Someone please give TruthOverfaith the attention he seems to be begging for.
joseph
says...“Atta boy, try not to pooh on the rug”
Daniel
says...Suppose one were to make a fallacious or unsound argument and the opponent fails to recognize it, or misses the problem. Should the judge deduct points from both sides such that it is a wash?
cl
says...TheistDude,
NO, please don’t. There’s enough of these types at VoxWorld. This person should go there, where they’ll get much more mileage out of their trolling.
Daniel,
Howdy.
I’m inclined to say no. Let’s say we’re debating phil. of mind, and “Bob” offered something like, “materialism is probably true because the majority of neuroscientists accept it” as an argument. As a judge, I would penalize that argument for both lack of validity and presence of fallacy [appeal to authority]. I wouldn’t penalize Bob’s opponent for failing to note the lack of validity or presence of fallacy, but I would grant Bob’s opponent points for successfully rebutting Bob’s argument. After all, if Bob’s opponent wishes to use their time supporting their own arguments as opposed demonstrating the errors with Bob’s, they shouldn’t be penalized for that approach. What do you think?
The tougher questions for me involve identifying when a claim is fallacious. On the one hand, I definitely think debaters should be able to cite scientific consensus as support for their arguments. On the other hand, any given argument needs to be supported with more than scientific consensus, which is known to change, often to the point of 180 degree reversal. I mean, look at the new CERN data in favor of tachyons. If ever there was a statement accepted as gospel truth by most scientists and laypeople, it’s the so-called speed limit of light. It seems tempting to grant the truth that nothing can travel faster than the speed of light, but is it really a truth?
Ultimately, I think debaters should be awarded when their arguments respect cross-disciplinary consensus. This is why I rejected Dominic’s argument that “causality might not always be linear.” It’s similar to the speed of light thing. Sure, it *MIGHT* be true that causality is not always linear, just like it *MIGHT* be true that tachyonic speeds exist. But judges shouldn’t award points for mere plausibility, they should award points for high “probability” supported by cross-disciplinary consensus.
Daniel
says...cl,
So I guess it would work out such that if one were to make a fallacious argument, one would be penalized and also open up the opportunity for ones opponents to gain points by calling attention to the fallacy.
I also agree with you that identifying an informal fallacy can be somewhat tricky. The fallacy of arguing from authority should be referred to as the argument from illegitimate authority, as I don’t take the appeal to authority itself to be intrinsically fallacious. An authority is illegitimate if the authority is (a) irrelevant, (b) unreliable, (c) unnecessary, (d) dogmatic, and/or (e) uncritical.
Most of our knowledge is passed on to us from authorities rather than from direct experimentation and observation. I’d have to admit that almost everything I know is second hand. But, that’s ok because argument from authority is, in fact, a weak form of inductive reasoning, i.e. in the past such experts were shown to be reliable and credible when speaking about the X, so in speaking about Y, which is similar to X with regard to a,b, and c, their assessment is trustworthy. It is not the best source of epistemic certainty, but for a mere mortal with three score and ten on this planet, it will have to do.
I am not sure what to make of this neutrino business, but it does sound like we are going to have to reassess or refine what “C” means! I think it perfectly illustrates why science is so powerful and “scientism” is so stupid –the latter I understand as the unbridled worship of science as an omniscient source of all knowledge (note that such a person is likely to take science as an authority dogmatically, uncritically, and apply the authority’s claims to irrelevant fields of inquiry, e.g. ethics, and metaphysics).
Finally, I agree with your assessment that Dominic’s use of “might” is fallacious. “Might” is a classic weasler that trivializes most claims. The only exception I can think of is if you are making a point using a modal system of logic where logical possibility, impossibility, or necessity are the points in question.
cl
says...Daniel,
I agree with you on scientism. It’s just a [hopefully] dying fad, a pendulum swung too far the other way.
That sounds like a usable metric. I would also penalize for poorly-worded claims, i.e., the difference between “nothing can go faster than the speed of light” vs. “the current scientific consensus is that nothing can go faster than the speed of light.” The former may or may not be true. The latter is undeniably true. As much as possible, opponents should aim for undeniably true claims.
I didn’t say it was fallacious, I just said I rejected it because of its irreverence for cross-disciplinary consensus. Though, maybe I’m wrong. Which fallacy do you think he committed?
Daniel
says...Right, I think that an appeal to cross-disciplinary consensus would strengthen an appeal to authority. However, I took the use of the word “might” to be an instance where Dominic was linguistically hedging his bets by watering down his claim so that it is trivially true. Depending on context, adding qualifiers like “might” can be legitimate. The key is that if he is going to appeal to logical possibility in his premise, his conclusion cannot suddenly have a stronger modality attached to it. Modalities can be very misleading.
In terms of taxonomy, some logicians call something like this weasling (c.f. Moore & Parker 2009, Critical Thinking, 154-156) or innuendo.
However, as I go back over the debate, I don’t see Dominic using the word “might” as a weasler. That is to say, my point only relates to the use of “might” and not to how you actually evaluated Dominic’s argument. So he is not guilty of committing this fallacy as far as I can tell.
cl
says...What saves him, in your opinion? You wrote,
Didn’t he use the “might” as a supporting argument for his claim the cosmological argument fails? Wouldn’t that be a “stronger modality?”
Daniel
says...As I read him, he is saying that the cosmological argument depends upon it being axiomatically true that all causes precede their effects in time. He then provides counter-examples to try and disprove this axiom. So the passage I am looking at right now is:
Dominic is trying to disprove the claim “Necessarily all causes precede their effects in time” by providing evidence for “It is possible that some causes do not precede their effects in time”. There are two senses of possibility operating in his argument: epistemic possibility, and metaphysical possibility. His evidence makes it somewhat epistemically possible that it is metaphysically possible that some causes do not precede their effects in time. But I don’t think this succeeds in actually disproving the cosmological argument. The problem is that we must assess how likely it is that these are legitimate examples of non-linear causation, and that is a difficult task. We might say that the degree to which the cosmological argument is put into question is the degree to which it is epistemically necessary that it is metaphysically possible that some causes don’t precede their effects in time. So he cannot conclude from the plausibility of these counter-examples to a stronger conclusion that it is known or certain that non-linear causality is metaphysically possible or that it is known or certain that the cosmological argument fails. So if he asserts a strong conclusion, he has fallaciously overstated his case by shifting from the possibility that his examples truly exhibiting non-linear causation to the claim that they actually succeed in doing this.
HOWEVER, Dominic says that his critique would do damage to all forms of the cosmological argument because they all depend upon linear causation. Unfortunately, WLC’s Kalam Cosmological Argument not only does not depend upon linear causation, it explicitly rejects temporal linear causation in specifically the case of God creating the universe. Craig is actually in Dominic’s position of providing examples of non-linear causation. According to Craig, God created time, so the kind of causation must not be temporally linear at all. Since Craig is explicit about this and Dominic cites Craig directly, I think it is fair to say that Dominic misrepresents Craig’s claims, or misunderstands Craig’s arguments. Ironically most atheists and skeptics attack Craig for positing non-linear causation, so I am surprised that Dominic would make this move. So, I would say that it is at least false that all defenders of the cosmological argument believe that linear causation is axiomatically true.
Needless to say, there are a few problems with his argument, as I see it.
Alrenous
says...Now I want to derive the correct method for using the ad authoritam.
Epistemically, how should I use authority?
Stephen Hawking knows a lot more about black holes than I do. Therefore, since he thinks singularities and event horizons exist, so should I.
Seems reasonable to me. Rather than undeniable, the opposite; the statements are all falsifiable, and hence supportable and confirmable.
Therefore, in debate. “Hawking knows more than you do, therefore you should believe as he does.”
My reply in that case would be, “Hawking has not run a dynamic model of black hole formation. It is true that the event horizon is a solution to General Relativity, but forming one is impossible – as density approaches the critical density, time dilation increases without limit, meaning it would literally take eternity for the in-falling matter to hit the critical density.” Again, falsifiable. (I believe in whta I call grey holes; objects arbitrarily similar to black holes, but without a singularity.)
Seems to work.
Daniel
says...Alrenous,
From the judging perspective, I would say that the use of relevant expert authority is legitimate. After all, the whole idea of a debate is to bring experts together and decide whose testimony and arguments stand up. That is to say, the whole point of a debate is for the debaters to make a convincing case that their authority is trustworthy. So suppose person A were to off-handedly reference Hawking’s theory of black holes without any references and without demonstrating competent mastery of the theory itself, that would weigh very lightly against an person B who is herself an expert theoretical physicist propounding a contrary theory to that of Hawking.
This does not mean that the more expert debater wins by default. Consider the debates WLC has had with Vic Stenger and Francis Ayala. Craig was debating physics with a physicist and evolutionary biology with a biologist and yet he was able to put in a good showing for those debates (perhaps even winning them).
Craig is successful not because he simply quotes experts. Often he will quote authorities sympathetic to his opponent’s position, or even quote the opponent himself. Craig’s expertise as a philosopher is in logical, linguistic, and conceptual analysis. So he is able to make effective use of authorities to reach conclusion that even those authorities might not explicitly or implicitly accept. The philosopher is the expert analyzer and synthesizer and so has a certain authority that the biologist and physicist lacks.
So long as the appeal to authority is relevant, up to date, well researched, and the speaker clearly comprehends what the authority is saying, it is the burden of the opponent to give us good reason to doubt the truth of those claims.