• Testing, Trolling, and Type-I erring

    Try to spot the troll here

    Up late prepping for an epi/stats lecture that I have to give tomorrow morning at the OU Health Sciences Center, and it occurs to me that I can put that off a bit by explaining the concept of a binary classification test to my readers. In statistics, in medicine, and in life we often find ourselves trying to simplify everything down to two distinct categories, e.g. is this result statistically significant or not, is that patient suffering from condition X or not, am I getting a good deal here or not?

    Sometimes, nature provides us with fairly clear-cut conditions which make binary classification tests appropriate, such as pregnancy or HIV infection. At other times, nature provides us with a spectrum of possibilities, such as the continuous range from healthy weight to overweight to obese. At these latter times, we usually draw arbitrarily precise lines so as to create bins for classification.

    One of the most important things to know when dealing with binary classification tests, is how likely we are to encounter a false positive or a false negative. A highly sensitive test has a low rate of false negatives, allowing a clinician to rule out disease when the test is negative, whereas a highly specific test has a low rate of false positives, allowing a clinician to rule in a specific condition when the test comes back positive. Generally speaking, there will be a trade-off between sensitivity and specificity, for any given condition being tested.

    Can we apply these statistical concepts to less scientific settings? Seems to me that we can, at least insofar as we find ourselves creating binary classifications in everyday life. As humans, we are incredibly prone to doing this without even thinking about it, especially when it comes to evaluating other humans. We unconsciously lump people into in and out-groups: us vs. them, familiar vs. other, agreeable commenters vs. trolls.

    Seems to me that the skeptical blogosphere generally errs in favor high sensitivity and low specificity when it comes to the binary test for troll/non-troll. That is to say, skeptical bloggers often err against false negatives and in favor of false positives. I understand why this happens. Genuine trolls suck up time and effort that could have been directed at persuading and educating those who prove amenable to education and persuasion, or directed at real world concerns such as putting together a lecture for tomorrow morning. 

    I understand it, but I don’t think it is the right approach. “Troll” is a dehumanizing term, and applying it to someone who turns out to be new to the movement and unclear on the concepts involved will create enemies out of potential allies. This is a good way to divide any movement, and that goes double for any movement that primarily organizes online.

    I hereby commit not to call anyone a “troll” on my blog. If you troll me, I will tell you specifically what I think you are doing wrong, without dehumanizing you or putting labels on you. If I have to ban you from the blog, I will do so firmly but politely, after giving you a chance to explain. This is my commitment to the skeptical community. Please hold me to it.

    Category: Blog RulesDamned Lies and StatisticsSecularism

    Article by: Damion Reinhardt

    Former fundie finds freethought fairly fab.