Blocking MPAA and RIAA spiders

[mike808@users.sourceforge.net]

While reviewing my webserver logs, I noticed a *lot* of accesses coming from a
certain 'crawler1.crawler918.com', specifically, 12.148.209.196. Well, it seems
they've disappeared from DNS.

A little Googling, and it turns out that this is a spider from a company that is
an ISP for people who want to protect their identity when they archive and
gather "evidence" from _your website_. People like the RIAA and the MPAA or
corporate types that object to your content and like to sue them in federal
court. People like your competitors that don't want you to know they're sucking
down your website and even dynamic database-driven content to analyse with a
fine tooth comb, looking for product/service/research information leaks.

If you feel like you don't want to "authorize" any of nameprotect.com's servers
(or their customers who run them) to access _your_ computers or webservers and
their content, see:

http://www.archlug.org/kwiki/FirewallKwikis

In addition, their webtroll does not obey 'robots.txt' access authorizations,
but more importantly, sucks down *every single graphic and multimedia file* on
your website. This tends to *dramatically impact* your bandwidth usage.

And they don't care. They are interested in your content only for the purpose 
of determining whether or not they want to sue you, and they most certainly are
not looking out for *your* interests as they consume your resources without
regard to your stated authorization limits. Especially when they're not
interested in telling you who they are, nor why they want your content. Even
worse, they keep coming back (just in case you might have put up content they
consider to be barratry-worthy).

Dropping any packets from this network at your firewall is a GoodThing(tm), as
implementing the deny rules protects any other services and webservers and such
that are on other ports that these folks may be trolling soon. It's a lot 
easier doing it once than in .htaccess files sprinkled all over your site(s).

If you routinely block competitor accesses to your resources, these folks are
even worse. At least you probably know who your competitors are. Unless, of
course, they're a nameprotect.com customer...

Mike/

---------------------------------------------
http://www.valuenet.net



-
To unsubscribe, send email to majordomo@luci.org with
"unsubscribe luci-discuss" in the body.