How Often our Anti-spam Search Toolbar Blocks Sites
by Gabriel Weinberg, June 2009
Our Firefox toolbar blocks over 45 million parked domains and provides easy access to our search engine.
Over the past few days, our toolbar users have visited 45,564 distinct domains that we cover.[1] Of those, we blocked 1,071, or 2.4%.
The total frequency with which these parked domains come up is a bit harder for us to quantify given our strict toolbar privacy policy and the way we cache on the client for speed.[2] However, we can still make a decent estimate.
There were 187,384 domains sent to us to check over the same past few days. This number is significantly higher than the number of distinct domains because many users visit the same sites, e.g. YouTube and Facebook.
Of the 187,384 checked domains, 2,338 were parked domains, or 1.3%. This lower percentage makes sense given that good domains, e.g. twitter.com, will have a higher incidence than bad ones, e.g. twittr.com.
Of course, these stats are constrained by our sample of Firefox toolbar users, who are probably more tech-savvy than average. I can't say for sure, but I suspect the percentages would be higher if our toolbar users were representative of the general population.[3]
In any case, I think the statistics are conservative estimates of the real numbers because we have some false negatives (parked domains not flagged as such) and we have hardly any false positives (non-parked domains flagged as parked).
Notes
[1] We only cover .aero, .biz, .com, .coop, .info, .mobi, .net, .org, and .us top level domains, which works out to about 61% (~110M/180M). We also don't cover local domains, IP addresses, subdomains other than www., or domains users ask us not to cover (via preferences).
[2] Our toolbar uses an in-memory cache. So for each user, we only see each domain they visit once per session. Also, domains we don't cover (see [1]) are prevented from being sent to us at all.
[3] On the one hand, I suspect tech-savvyness would lead people to less parked domains. On the other hand, however, such users may be more likely to type in Web addresses into their address bars (and get some wrong) as well as explore parked domains to actually buy them. I have no way to weigh these factors, but I guess that the increased likelihood of non-tech savvy users to click on random links that lead to parked domains overwhelms the other factors.