The Real Story Behind SmartFilter Blocking BoingBoing

Last week, authors at the popular super-blog found out that they were being blocked by the blocking program SmartFilter as a “nudity” site, because two posts last month (out of a total of 692) contained nudity (and non-sexy photos of nudity, at that). SmartFilter told them they could re-organize their site so that all nudity would be under one directory that they could block. Instead, BoingBoing’s editors decided they resented being told how to organize their site to make it easier to censor, and responded by posting a list of methods to get around SmartFilter (including Peacefire’s Circumventor software).

The whole story was summarized in a New York Times article. Many of you probably heard about this already, and some contacted us about it.

Now to explain my heading about “the real story,” let me first say that all the facts in the NYT story and in BoingBoing’s post are correct as far as I know. But I think there’s a bigger issue here. (No, not just “blocking pictures of boobs is idiotic,” although that, too!)

In truth, people have been complaining about wrong blocks by SmartFilter for as long as SmartFilter has existed. The most thorough recent study was by Ben Edelman in 2002 as part of his testimony in the ACLU’s challenge to the Children’s Internet Protection Act. If you search that page for “SmartFilter,” many examples come up that are not even borderline cases, just completely ridiculous, e.g.:

and many others that are perhaps understandable errors, but still mistakes:

Our own page about SmartFilter is creakily old, from 1997. We hadn’t looked at it since then, since it was aimed mainly at the workplace market, which was outside our purview. But – The Stop AIDS Project? The Feminist Majority Foundation? Community United Against Violence? No company that took pride in its accuracy would have let these sites get blocked even once; the fact that these sites were ever blocked suggests that its process was seriously broken.

We also did a study in 2000 of percentage error rates for different programs (although we didn’t include SmartFilter) by taking a random sample of domain names, finding which domains were blocked by each program, and finding what percentage were clearly mistakes. The percentage of wrongly blocked sites ranged from 20% for AOL, to 80% for Cyber Patrol and SurfWatch – that’s not a typo, eighty percent! (Later attributed to one particularly overbroad blocking technique that they used, which is that if one site on a server was blocked, all sites on that server would be blocked – on the theory that ISPs would group all their porn sites together on one server, which is in fact often not the case.)

Now, the block of BoingBoing is stupid – 2 out of 692 posts containing nudity constitutes only 0.28%, and probably more than 0.28% of sites from the entire .com domain contain nudity, so by the same logic it should just block all of .com while it’s at it. But it’s not completely insane. On the other hand there have been many examples of sites blocked by SmartFilter, like blocking a home-schooling site as a sex site, that are just completely out of left field and made no sense at all. (Peacefire is of course blocked, but I would hardly call that a “mistake” since the first link on our site is “How to disable your blocking software.”)

I think the real lesson here is that, despite all the hype, blogs have not solved the “gatekeeper” problem. It’s no longer true that a handful of media giants control what people get to find out about, but it’s still true that an issue doesn’t become an “issue,” and has a hard time entering the public consciousness, unless it gets noticed by one of the new “gatekeepers of information”. People have been complaining about wrong blocks by SmartFilter for nine freaking years, and suddenly BoingBoing gets blocked and it’s in the New York Times!

But it’s a good thing that the issue did get noticed, since, thanks to BB’s efforts, maybe more citizens in countries like Qatar, UAE, Iran, and other countries that use SmartFilter, will learn how to get around the blocking software. SmartFilter gets paid the same whether people can get around its software or not, but it’s probably still wishing they’d never heard of BoingBoing.

[Bennett Haselton of]

[tags]censorship,boingboing,smartfilter,gatekeeper,blocking pictures of boobs,[/tags]