DBT01: Call For Judges, Scoring Suggestions
Posted in (A)Theist Debate League, DBT01, Debate on | 1 minute | 51 Comments →
This post is an open discussion for the confirmed judges (Daniel Vecchio, Matt DeStefano and Andrés Ruiz) in next week’s debate between Peter Hurford and I. We also have a tentative fourth judge (Adamoriens), who I will be happy to implement should we be able to locate a willing fifth judge (to preserve an odd number of judges, with Peter’s consent of course). Feel free to volunteer yourself or suggest somebody else, but keep in mind the fifth judge must be a theist (since Adamoriens would effectively qualify as an atheist in this debate). I’ve got a few people in mind, we’ll see what happens. Regardless of whether we end up with three judges or five, this debate is scheduled to kickoff Wednesday February 15th, so we should all talk about scoring. Ultimately, it is up to the judges, but I want the thread open so we can get as much feedback as possible.
We talked a little about scoring debates a few posts back, everybody might want to read that first.
Andres
says...I’ll bite first.
Having read the thread CL suggests, I liked the layout suggested by the first commenter:
“For every round award points for arguments and counter arguments.
If an argument is given in a round and a judge finds the argument to be relevant to the debate and coherent a point should be given. Likewise if an argument is given but it is incoherent or irrelevant a point should not be given. Notice here that the judge should still give a point even if the judge does not agree with the premises or the conclusion of the argument.
Restrict the maximum number of points that can be given out to 3 per round (so that there is no flooding of arguments). Do the same for counter arguments.
FORMAT:
A debate topic is chosen (and the relevance of the arguments is judged relative to it).
In round 1 each person gives arguments where they argue their case.
in round 2 they give counter arguments to the opening case of their opponent.
in round 3 they give counter-counter arguments
If this form is broken no points should be given. Example: if in round 3 new arguments are introduced this should not be given points by the judges, it should just be ignored.
After 3 rounds bonus points should be given if the opponent dropped an argument (they failed to give counter arguments). This should not be per-judge, the judges should award this as a group.
In the end, the one with most points wins (the point of giving out points per argument is to avoid the issue of the judge having a personal favorite).”
I’d personally suggest a -1 for any fallacious reasoning used in the debate, thus it’d go something like:
+1 for a valid and relevant argument
+0 for an irrelevant (even if valid and/or sound) argument (I’m thinking something along the lines of: This is a debate on whether gratuitous suffering exists. Thus, if CL would offer an argument along the lines of “evil implies an objective standard, atheism can’t supply that standard, therefore God…” then I’d award zero points for that given that it isn’t relevant to whether or not gratuitous suffering exists, even though *on the whole*, it might be a relevant argument to the problem of evil, but it has nothing to do with whether or not gratuitous suffering exists, thus would get no point.
-1 for an obviously fallacious line of reasoning, whether intentional or not.
Those are my thoughts. Feel free to chime in and modify as you see fit.
Andres
says...I’m also wondering, to what extent should the judges reveal their voting methods?
Someone mentioned that one ought to limit the points in each round to 3 so that the debaters don’t flood the debate with too many arguments and thus making it harder on the opponent to respond to all of them. Ideally that’s fine, but should the debater know that? Suppose you simply can’t give less than 3-4 arguments for your position, why should the format prevent you from giving as many arguments as you need to make a good cumulative case for your position?
I’m thinking for example on any theism debate. You simply can’t establish the existence of God with one argument (well, the Ontological argument can), most theists grant that any one given argument doesn’t establish their brand of theism. Only a cumulative set of arguments does. Why should the theist not be allowed to present the strongest case he/she can muster as opposed to being restricted to 3-ish arguments. If these 3 arguments can’t do the job, then it seems the non-believer has a leg up on the theist.
Just a few random thoughts.
cl
says...Well, it doesn’t matter to me whether I’m aware of the judging criteria or not. My only concern is that all judges agree and are on the same page. Although, if interlocutors are aware of the criteria, that might help them craft more effective rounds. OTOH, if they aren’t aware of the criteria, they might end up straying quite far from what the judges wanted.
If I had to pick one of the two, I’d like to be aware of the criteria by which our arguments are judged.
RE: argument flooding, I definitely see the wisdom in limiting points (as to prevent argument flooding), but at the same time, if somebody can craft 6-7 coherent, relevant arguments as opposed 3-4, I tend to feel like they should be rewarded for that. There’s also the option of enforcing something like a “Minimum Argument” rule, meaning that each interlocutor must produce at least 3 arguments.
Anyhow, thanks for your thoughts, I look forward to opinions from the others.
Daniel
says...Andres,
I think your proposal is quite good. As to your question of whether judges should reveal their voting methods, I would say yes. I think transparency is important here.
You ask:
I agree that points should be limited to three in each round. But, if there are extra arguments and they remain unaddressed throughout the rebuttal period, they might function exclusively as a tie breaker. I think this might encourage the full development of a cumulative case, and the subsequent rebuttal of those arguments. If a debater chooses to take up space with extra arguments, he or she takes the risk of not properly developing those three that definitely will count. If the a debater chooses to not address his or her opponent’s extra arguments, but focuses on the three, then he or she runs a certain risk that those extra arguments may still tip the scale, should all of the rebuttals be countered sufficiently. I think this would add an interesting bit of strategy to the debate. What do you think of this compromise?
Also, I think we all will agree when a formal fallacy is committed, but the detection of an informal fallacy is something of an art. If we are going to take points away for informal fallacies, I think we should operate with a consistent list of clearly defined informal fallacies (perhaps from an agreed upon logic text, or online resource). Otherwise, I fear that our biases might influence how we interpret a certain arguments. For example, it is sometimes disputed as to whether one is committing the genetic fallacy, or merely pointing out that the origin of a certain asserted belief is merely cultural/psychological and so lacks proper epistemic warrant. Informal fallacies often have perfectly reasonable cousins. I think it will help with the objectivity of judging, if we are all operating with and appealing to the same set of definitions.
A good site is: http://www.nizkor.org/features/fallacies/
cl
says...Perfect. I think we’re onto something here. How about this idea: rollover word count. If I’m allotted 2000 and I only use 1500, then, should my opponent flood the floor with arguments, I have cushion to respond. I definitely wouldn’t want to award / penalize points with regard to word count, but if a debater can make the case with less words, why shouldn’t they have that luxury?
Then again, it’s up to the opponents to agree on the format, and rollover word count seems like a formatting decision, not a judging decision. So, Peter: would you like to try this format modification? I think it will really up the quality of the debate. It will encourage debaters to respect word economy and articulation.
Matt DeStefano
says...The format Andres suggested would work for me. What could we have for a theoretical tie-breaker? Should we give partial points for arguments we deem as weak, or rebuttals that are weak?
I like the idea of judging formal fallacies, but I’m not sure, even after agreeing to a definition of informal fallacies, we will be able to be consistent among judges in deciding between fallacy and fair play.
Stephen R. Diamond
says...Isn’t this scoring system based on an assumption that the strength of the case is proportional to the number of plausible arguments? What if you have only one argument, but it’s a sockdolager?
Peter Hurford
says...Cl, I’m fine with the change of the format to reserve the balance of words not used for future statements.
I personally like the judging used for debates on the Internet Infidels, though there are some obvious improvements to be made. Their format is, paraphrased:
* The judges will instructed to assess who won based on who was able to better defend their own and rebut the other’s arguments in this particular debate, regardless of whether the judges themselves agree or disagree with those arguments or conclusions.
* Even a fallacious argument will be counted as a successful argument if it is not effectively rebutted.
* A judge may declare neither party the winner, which rates a score of zero. Or a judge may declare one party the winner and assign a score between 1 and 4 as follows: {1} only barely won the debate, {2} won the debate by a significant margin, {3} won the debate by a large margin, {4} won the debate by a decisive margin.
See http://www.infidels.org/library/modern/richard_carrier/carrier-wanchick/assessment.html for an example assessment for an example.
But I’m honestly fine with whatever the judges come up with and think is fair. I would, however, like to know the criteria ahead of time, and see that a single criteria is agreed to by all judges.
Matt DeStefano
says...I’ll throw my support behind the Internet Infidel’s version that Peter has shown us. It’s much more open-ended and allows us for a better and more accurate assessment of the debate.
cl
says...Y’all are the judges, but I disagree with Matt and I see many flaws with the II format. I felt like Andrés’ and Daniel’s suggestions were less subjective. For example, Peter wrote,
Well, how do we define “better defend?” I’m trying to avoid judgments assessed on gut feelings, and for that reason I think we need a numeric metric.
No. Hell no. Fallacious arguments should never be counted as successful. That’s weak.
I like that, but it only pertains to declaring a winner and the margin of win. Personally, I think the best possible judging system will involve some of what Daniel and Andrés said, combined with the best from what Peter left.
All I ask is that somebody put all the criteria down in writing before the debate commences on Wednesday. If we need more time, that’s fine with me, although I can’t speak for Peter. I think the best-case scenario would be to just nail down some specific criteria / scoring rules and go for it. We’ll learn what works and doesn’t as we go, too.
Stephen R. Diamond,
Not really, because there’s no reward for a higher number of arguments per se. OTOH, points should be given for relevant, cogent arguments, no matter how many are produced. But I think I see your concern: how do we judge a sockdolager against three inconclusive arguments? We would certainly need some sort of metric for that. Any suggestions?
Adamoriens
says...I’m even less involved then the rest of you, but it seems to me that cl’s and the secular web’s scheme try to impose standards in different places. How about combining them:
+1 for formal validity.
+1 for informal validity and soundness (premises support conclusion and relevant reasons given for factual premise(s)).
+0 when informally invalid/relevant reasons not given for factual premises.
-1 for formal invalidity.
-1 for failure to counter formally valid, informally valid and relevantly-supported argument from opponent.
-0 for failure to counter formally-invalid, informally invalid or irrelevantly supported argument from opponent.
Maximum points per round: 6
Margins of success by either party at the end can be determined by the interval between their total points. Say, a significant win would be an interval of 6 etc.
In this way, the first post by either party is rewarded up to the points available for 3 maximally-successful arguments, and the same for subsequent rounds. However, subsequent posts by both parties will also be scored for whether they dispute their opponent’s preceding argument’s formal/informal validity or factual premises. So, the accumulating penalties for continued failure to dispute an opponent’s argument are properly penalized, which is a good motivator to read your opponent’s arguments closely. Additionally, since neither party (may not) know how the judges will assess his/her opponent’s arguments, both parties will be rationally motivated to address all arguments from the other party. If they do know how the judges scored their opponents on the preceding round, they will know how much space they ought to devote to disputing those arguments. If the judges are unanimous that they were good; dispute them. If all agree they were bad; ignore them. If there is disagreement among the judges, or you don’t have time to read judge scoring and commentary; dispute them, just to be safe.
This also provides a natural restraint on entirely new arguments being presented; if you’ll only be scored for a maximum of 3 good arguments, and penalized for a failure to address opponent’s arguments, you’ll rationally devote your energies to counter-arguments rather than entirely new arguments.
Of course, this all ensures that each party present 3 arguments to begin with. Since arguments can be interlocking, it might be difficult to distinguish one from another; also, 3 arguments is a purely arbitrary amount.
But this seems like a regular scoring system with appropriate motivations built in. What say you?
cl
says...Nice adds, Adamoriens. I’m confident we’ll end up with something workable.
Peter,
Do you think we should discuss definitions and premises openly here? I do. For example, if you go with your standard definition of “needless suffering” as “suffering that does not exist because of a higher good,” what, precisely does “because of” mean?
I think we need finely-articulated, clearly-understood terms to have the best debate possible.
cl
says...Yeah I dunno… the more I think about it, the more I question whether we should limit the number of arguments. We definitely need parameters but we don’t necessarily need restrictions.
But, again, it’s up to the judges.
Daniel
says...I think this is a good compromise and I am willing to support it.
I am still on the fence about limiting arguments. Perhaps my tie-breaker compromise is too convoluted, so I’ll table it for now. As I see it, points are awarded to formal validity and also for soundness, i.e. good warrant is provided for the premises. I understand this to mean that a given argument can rack up more points if each of the premises is defended well. So it stands to reason that lumping together a bunch of valid unsupported argument might land you a few points, but it seems like poor overall strategy. You’re not going to land much more than a point for each valid argument, and unsupported arguments are just fodder for your opponent, who could successfully rebut you by simply presenting an unsupported, but valid counter-argument. I suppose if flooding the gates becomes a problem in the future, we could add an upper limit for med-length debates (I would tend towards a limit of four to five).
Peter Hurford
says...Ok, here’s a couple of considerations:
I agree that I would like to see agreed-upon criteria decided before my initial opening statement is published.
My goal for the judging criteria is not to something immensely complex like a ten-point scale of formal validity plus three points for every proper citation in the Chicago format. I think that can get out of hand really quickly.
Instead, I’m just looking for something that judges can agree upon and that we can be reasonably expected to follow as arguers to know what to do. I’d really just like to see judges comment on how the debate is going and where they see current flaws, as well as give an assessment of who is winning and why. I’m fine with it being a bit more vague however. And as long as the judges defend their assessment properly I think the decision even could be allowed to be potentially biased, since this bias should be corrected by having an even amount of atheist and religious judges.
And it’s not like the debate is going anywhere. It can be reassessed holistically at anytime should a better method be discovered a few months from now.
~
While I can’t speak for Internet Infidels, I think this was included because atheists and Christians have differing opinions over whether an argument is really fallacious, and it’s difficult to resolve who is right.
I can see this being a problem in our debate as I anticipate a big question will be whether or not I am committing the “argument from ignorance” fallacy with my conclusion that some suffering is needless. I’d hate for that determination to be made by judges, especially since it’s at the very heart of what I need to argue.
You’d obviously get the opportunity to rebut any fallacies I make, and obviously I will not set out to intentionally commit any fallacies.
~
The two problems here are that I don’t intend to use formal arguments with clearly delineated premises (P1, P2, therefore C3), nor do I intend to make three separate arguments. I’ve already written my opening statement, and I found it very restrictive and annoyingly difficult to try to make my case that formally.
~
Sure. Me too.
~
This time I’m using needless suffering as anything that is not logically required for a higher benefit. I’ve dedicated a section of my opening statement to explain in more detail what I mean.
As an example, perhaps it could be argued that the Holocaust was logically required for the higher benefit of humanity’s ability to make free choices. The higher benefit part comes from the fact that we’d rather endure the Holocaust than not be able to make free choices, and the logically required part comes from the Holocaust being an unavoidable consequence of these free choices — the only way for God to have prevented the Holocaust would have been to remove our ability to make free choices.
I’d be happy to hear your thoughts on the issue.
Peter Hurford
says...Another thing I liked from the Internet Infidels was the jointly written statement about what precisely was being debated, and what precisely was to be accomplished by either side. Maybe we could write one of those?
Andres
says...Would anyone be interested in laying out a skeleton of the rules, have the rest of the judges look at it, make any possible additions to it or quibble over some point, revise, and come to some sort of agreement (hopefully), and then send it to CL and Peter for final approvals?
While all these posts are interesting, and it’d be great to have extensive back and forth on this, since the first entry is due on Wednesday we should try to get a move on this and make some progress before then.
Adamoriens
says...I see your point, Peter. There’s a ton of persuasive arguments which are formally invalid, and certain epistemic situations may make your thesis informally correct and yet formally flawed.
Daniel
says...This is true. One might confuse a perfectly good inductive argument with something like the fallacy of affirming the consequent. But I think context should make this clear, hopefully. In the end, we have three judges.
I actually think Adamorien’s suggestion might serve as a beginning of a skeleton. Should we begin with this and flesh out something more specific?
Matt DeStefano
says...Adamorien’s amended rules seem to be fairly workable. I’m worried about too much concentration on formal validity and not enough judgment about justification for epistemic warrant of a given premise or conclusion. It seems that this does a slightly better job than the original suggestion, but I worry that it’s still too rigid.
If both of them only make valid arguments, make sure to attempt to respond to each argument their opponent presents, and simply gesture at warrant for accepting the premises of their argument — it seems like the margin of defeat will be relatively small. Infidels seems to augment this by giving a 4 pt spread for “winning” a specific argument and counting this towards a running total.
I understand the desire for objectivity, but it might not give the judges room for properly calling it as we see it. Having said all of that, I’m rather inexperienced at judging debates through such a medium, and will gladly give my consent to anything the other judges, cl, and Peter can agree to.
Peter Hurford
says...I’d just like to reiterate that I’m personally far more interested in hearing the judge’s analysis of our interactions rather than getting a formal score. I enjoyed reading the commentary on the PZ Meyers Memorial Debate while it lasted, and thought that was valuable independent of the stupid scoring system.
Adamoriens
says...You could roll deductive and inductive validity into the assignation of one point, and epistemic warrant for premises as another. Perhaps the best one can do is prevent well-intentioned people from going off the rails, rather than attempt to make the structure impervious to sociopaths.
Adamoriens
says...I see where you`re coming from, Peter. But any personal reflection about plausibility can run somewhat free of our assessed score. It seems like it`s not a debate judge`s job to dispute someone`s epistemic warrant (except, possibly, where it is a sub-argument to be scored)so long as it`s presented and relevant, though each of the judges will inevitably reflect on their attitude toward the epistemic warrant. We`re all interested in the subject matter, after all.
Daniel
says...So are we agreed to a 6 point max per round? If the concern is that formal validity is overly emphasized, we could increase the possible points for epistemic warrant, and limit the amount of points one can earn for validity.
cl
says...I would like to see a clear explanation of “formal” vs. “informal” validity, if we are going to even use those terms. I prefer the traditional “sound” and “valid” criteria, personally.
A valid argument (premises necessarily entail conclusion):
P1: Women have penises.
P2: V has a penis.
C: V is a woman.
A sound argument (premises necessarily entail conclusion, and are true):
P1: Men have penises.
P2: V has a penis.
C: V is a man.
Also, whatever we decide on, can somebody write up the points of agreement WRT judging? If nobody’s done that by end of day tomorrow, I guess I’ll have to give it my best shot.
Whatever happens, Peter and I will wait for consensus before we post anything.
Adamoriens
says...If validity can be assigned one point, and support for premises another, we won’t have to distinguish between different types of validity beforehand.
Daniel
says...A Modest Proposal Based on Concerns Raised:
I think the most controversial aspect of this proposal is that I would like to expand the possible points per round to a maximum of 12. This will allow a point to be awarded for validity, and up to three for epistemic warrant per argument, putting more emphasis on the truth of the premises and less on formal validity. Inductive arguments can earn a maximum of three points and will be evaluated in much the same way as we evaluate the justification for premises in deductive arguments (according to a three point scale of strength). This means that deductive arguments are more valuable, but not by much. One could fit in three sound deductive arguments, or four strong inductive arguments to gain max points.
Points are award each round and are based on the following criteria:
1. 1 point will be awarded for a formally valid deductive argument. An argument is formally valid if and only if the conclusion follows from the premises necessarily and inescapably. Examples of valid rules of inference include Modus Ponens, Modus Tollens, Disjunctive Syllogism, Constructive Dilemma, Hypothetical Syllogism, Addition, Simplification, Conjunction, and 15 varieties of valid categorical syllogisms as found in Copi, Cohen, and McMahon’s Introduction to Logic, 14th edition. Rules of logical equivalence may also be employed to preserve validity, though not necessary to explicate so long as validity is preserved.
2. Up to 3 points can be awarded for presenting epistemic warrant to accept the premises as true in a deductive argument. Arguments that provide relevant substantiated facts, relevant authority, relevant and substantiated empirical evidence, cited studies, etc. will be evaluated as strong and will be awarded 3 points. Evidence that is considered to be riddled with weaker/unverified/unverifiable or irrelevant data/evidence will earn 0-2 points.
3. Inductive arguments earn no points for validity (as they are not formally valid). They will be assessed according to the criteria found in (2), i.e. in terms of varying degrees of strength. Inductive arguments should be labeled as such so that criteria of validity are not applied to them.
4. In subsequent rounds, arguments may only be introduced to buttress existing arguments or refute an opponent’s argument. They will be evaluated according to criteria found in (1) or (2). New arguments in support of the question of the debate will not be awarded points and will be ignored by the judges.
5. The maximum amount of points one can earn is 12.
Points will be deducted based on the following criteria:
1. A point will be deducted for each instance of a formal fallacy. If a formal fallacy occurs, no points can be earned for the epistemic warrant to accept the truth of the premises, even if that evidence is strong. Example of a formal fallacy would be something like affirming the consequent, or the fallacy of an undistributed middle term.
2. In subsequent rounds, points will be deducted for failure to counter a deductive argument, or inductive argument. To avoid this, countering a deductive argument can be achieved by proving the argument invalid, or by giving epistemic warrant to think at least one of the premises is false. Countering an inductive argument requires one to provide epistemic warrant to think that the opponent has presented false, or weak data to support the inference, or by introducing some fact or evidence that mitigates the strength of the inference.
3. Point deductions should be applied after the amount of points earned in the round has been determined. Once the maximum amount of points is achieved, further points cannot be applied to offset deducted points.
4. A point will be deducted for every 50 words over the word-count limit allocated per round. If one falls under the word-count in a round, the remaining word-count will roll over into the next round.
5. The minimum score one can achieve in a round is 0.
These are just some suggestions based on concerns already raised. I would like to hammer out what is expected of the judge in terms of analysis, and when our reviews are expected (before the start of a new round, or at the end)?
Thoughts? Counter-proposals?
Peter Hurford
says...I accept the above criteria provided you’re perfectly ok with my argument not including a formally stated argument with a clearly stated P1, P2, C3.
Daniel
says...I am fine with that. Such things are a matter of personal style as far as I am concerned.
Matt DeStefano
says...I like Daniel’s proposal, although I’m a bit confused about the point discrepancy between inductive and deductive arguments. Does this mean the only way to get the entire 12 points is to present three deductive arguments, or can 4 points still be earned by inductive arguments?
I think presenting our reviews before each round will be beneficial for both those reading the debate as well as the debaters themselves. Thanks for putting that together, Daniel.
Andres
says...Hey all, just a quick question. What kind of review would be most adequate for each round? Do you want points and explanation of those points or would you also like us to interact with the arguments?
Having read the first post there’s a lot I could say, though I’m not sure what’d be most appropriate.
Andres
says...Im also concerned about the points. Peter included quite a few arguments in his first post, but there is only one general argument he makes for gratuitous suffering, the rest of the arguments are mostly there to support a given premise or, in this case, to pre empt possible theodicies CL might whip out. Im not sure how to score say, his comments on free will or his argument that say, a soul making theodicy is no good.
cl
says...Well, for whatever my opinion is worth, I’d suggest we shy away from having judges interact with arguments, if by “interact” we mean actually discuss them with interlocutors or readers during the course of the debate. OTOH, I am very interested in hearing the judges’ reasoning for why they accepted or rejected a particular argument. So I would welcome justifications for decisions made, and frown towards anything that could be construed as “assistance” if that makes sense.
Peter Hurford
says...I agree with CL.
Andres
says...Well, this is tough. I have an objection to Peter’s definition of gratuitous suffering.
Since part of his argument hinges on it, Im not persuaded by it. After that, how do you proceed? I need to discuss my objection so that any point deduction won’t be seen as arbitrary.
Matt DeStefano
says...That makes sense. No “If I were arguing” and more “I don’t think they provided enough justification for”. Interacting with the arguments would basically have us assume the role of debater.
On par with Andres, I’m confused about points and this is why I had worries about such a strict objective measurement for debates. Depending on how you want to define an argument, there could be only one or more than six. If we’re totaling to 12, how are we supposed to use this metric?
Adamoriens
says...I see Peter’s argument like this:
The most plausible theodicies we have fail to explain certain instances of evil.
There appears to be needless suffering.
Therefore, there is needless suffering.
The epistemic warrant for this argument is made up of several sub-arguments. His rebuttals to theodicies support the first premise, the second premise follows, and hus response to skeptical theism etc is intended to support the third premise.
The main problem is rewarding inductive and deductive validity differently.For example, if we add the premise,
If there’s apparently needless suffering, there”s needless suffering.
This premise adds no new epistemic warrant to the argument, but it changes it from an inductive to a dedective argument, allowing a point to be awarded that wouldn’t have been otherwise. Adding enough premises to pretty much any inductive argument will make it deductive, so its odd to award points for a rather pointless maneuver. What do you think?
cl
says...Hey guys. Above all else, though we should strive for rigor, let’s take a flexible approach. We might not end up with a fully fleshed out scoring system after just one debate. Rather, I suggest the judges make their best attempts at implementing scoring according to the accepted criteria, then let the chips fall where they may. In addition to offering a scorecard, perhaps each judge can give a non-scored evaluation of who won, and why? This way, each judge offers two opinions: one that is scored, and one that’s more of a summary.
More importantly, I’m rethinking whether or not the judges should post their writeups between rounds. It seems to me that doing so could easily spill over towards interaction or advise. After all, they don’t announce each round in a boxing match; they wait until the end.
So, I’m officially in favor of blind judging, all the way until the end, but y’all are free to decide for yourselves. Hell, I’m even thinking of closing comments on each round, for the same reasons…
Your thoughts?
Andres
says...CL, that sounds awesome. I really don’t know how to assign points to this, though I know I can write a kickass essay showing why I agree with something and why I someone more successfully argues their point. A score card and a response essay at the end would be perfect.
Peter Hurford
says...Cl,
The thing I don’t think is fair is judges telling us what they want to see, because then we all lose our objectivity as we seek to placate the judge and the judge gets to see what he wants.
I also don’t think it is fair for us to respond to things and clarify our views outside our word limit. That basically makes word limits useless. However, all open threads can be resolved after the debate.
I don’t, however, at all mind people commenting or objecting to what we write. I think it’s inevitable, and I like seeing people get involved. I don’t mind if it gives you ideas, and I’m surprised that you want to avoid the help. (Not that I think you’re terrible without help, but nearly every idea can be improved through peer review.)
Daniel
says...For what it’s worth,
I am committed to not reading anything but the posts. I am ignoring comments, and outside blog-posts on the debate for the time being and I think it would be prudent for the other judges to do so as well, until we have finished our own assessments.
Peter Hurford
says...I have a slight preference for judges releasing commentary and committing to scores on each essay individually, before the next piece is written. Otherwise, you will be grading my essay based on full knowledge of Cl’s rebuttal, which (if Cl does as well as he usually does) should make you think less of my argument than you ordinarily would. You would be seeing his rebuttal without me getting a chance to “defend myself”.
Thoughts?
cl
says...Peter,
Did you get the impression they were telling us what they want to see? I didn’t. Only thing I recall them saying they want to see is good arguments founded on epistemic warrant. Did I miss something?
I agree. I’m staying out of threads with the exception of discussing format issues. Speaking of which, honestly I think we’re doing just fine even though we might not have worked out all the kinks yet. We’ll all get better as we go along.
That’s actually a great point. I never made an assumption either way but now that you mention it, yeah… I think they should score each piece before reading the next one. However, I’m still in favor of being blind to the whole thing until the end. I like suspense. But that’s just my preference. I’ll leave it up to the judges.
Daniel
says...I’m almost there with Peter’s opening and hope to have the final evaluation sometime tomorrow afternoon. So I do not plan on evaluating Peter in light of cl. That wouldn’t be fair. But should I just hold on to it until after cl posts?
I agree that the judgments and comments should be made before moving on to read the rebuttal. However, I don’t want to sway the debate by submitting my comments and critiques early.
So I think judges should not send or post their comments until the round is complete.
cl
says...I intend to complete the round sometime between 9pm-12pm PST, tomorrow, Sunday the 19th. I’ll send an email to all involved as soon as I post my piece.
Peter Hurford
says...Oh, no. Judges have done a good job so far of refraining from saying things like “I think Peter’s argument could be improved by changing his definition of needless suffering to […]” and instead just saying “I think Peter’s definition of needless suffering could be improved”. And I was commenting to say I want it kept that way.
~
That’s fine. I personally prefer the feedback, but I’m happy to leave that up to personal preference as long as all reviews are written in light of the original piece like we agreed. Do you suppose you’d be able to not read the judges pieces to maintain the suspense for yourself? (And I don’t mean that question in any sort of snarky manner, but genuinely wondering what you’re looking for.)
~
I suppose that would be the best of both worlds, as long as judges promise to evaluate each piece before reading the next piece.
~
Looking forward to it!
~
Has anyone heard from Matt?
cl
says...Yeah, I’m pretty sure I’ll end up reading the reviews if they post between rounds. No big deal. The temptation will just be too much. In fact, I read Adamoriens’ write up late last night. I haven’t heard from Matt but I trust he’s in the mix.
I sure hope so, it’s a doozy.
Daniel
says...We don’t have a rule about missing the time deadline. If we were to make one, it would not apply to the first round. I’m of the mind that points would be taken off by a more significant time than a few minutes. If one were to miss the deadline by, say, an hour, I think it would merit a point deduction. I would also be of the mind that time deadlines are relative to one’s time-zone, no?
cl
says...Yeah, they’re relative. That’s why I stipulated that I was in PST. Sound good to me!
Peter Hurford
says...My proposal/draft for the lateness policy:
1.) The due date for posting the entry for the round is at 11:59PM of the day the round is due in a pre-specified timezone, unless otherwise stated or agreed upon by both participants and the judges.
2.) Participants may have a six hour grace period to be late for any reason with no penalty.
3.) For every six hours late beyond the grace period, one point is to be deducted from the debate score for lateness, unless a justifying reason is given and agreed to by a majority of the judges.
4.) If the score of the debate would be reduced to 0 (total 3 days late) and no reason is given and accepted, the debate ends in a forfeit.
Daniel
says...Peter,
I accept the lateness policy.