Talk Arcades: Forum for Arcade Webmasters 
 |
 |
 |
|
|||||||
 |
 |
|
Welcome to Talk Arcades, the premier forum for arcade webmasters. You are currently viewing our boards as a guest. By joining our community you will be able to make posts, communicate privately with other arcade webmasters and participate in our Live Marketplace. Registration is easy, so please join us today! |
 |
|
|
LinkBack | Thread Tools | Display Modes |
|
|
10-17-2007, 06:46 PM
|
#1 (permalink) |
|
Preferred Member
Join Date: Jun 2007
Posts: 160
|
Bad robots
 Has anyone had problems with the user agent BD Brand protect?BD-Brandprotect | The leader in online threat protection Their robot crawls one of my sites (not an arcade) all the time trolling for copyright material. I have blocked all the ip addresses associated with them in .htaccess (hopefully) It is really irritating when robots don't obey the robots.txt file The crawl rate is fast and lengthy..eating my bandwidth. |
|
 
|
|
|
|
|
|
10-17-2007, 09:18 PM
|
#2 (permalink) |
|
Gamer FX Staff
Senior Member
Join Date: Oct 2006
Posts: 681
|
Don't think I've come across them myself, what do they give for a user agent?
__________________
http://gamersgo.com A list of arcade related sites, add yours. http://auntiefondy.com Lists, add your site. http://kbsplace.com |
|
|
|
|
|
|
|
|
10-18-2007, 03:01 AM
|
#3 (permalink) |
|
Moderator
Senior Member
Join Date: Dec 2006
Location: Under Your Mouse!
Posts: 1,208
|
Try bannin in WHM/Cpanel
__________________
 Arcade Banner Exchange, game packs, top site, arcade script and forum for all of your arcade needs http://www.mibbi.com This site stinks!! www.farting.us Coming soon MibbiJr! |
|
|
|
|
|
|
|
10-18-2007, 06:24 PM
|
#5 (permalink) |
|
Moderator
Senior Member
Join Date: Dec 2006
Location: Under Your Mouse!
Posts: 1,208
|
You can wild card it
172.23.4.*
__________________
Arcade Banner Exchange, game packs, top site, arcade script and forum for all of your arcade needs http://www.mibbi.com This site stinks!! www.farting.us Coming soon MibbiJr! |
|
|
|
|
|
|
|
|
10-18-2007, 07:49 PM
|
#6 (permalink) |
|
Preferred Member
Join Date: Jun 2007
Posts: 160
|
Hi Ricky,
That is a good idea. I did a network search on their bot ip addresses and blocked the 7 I found. They seemed to have stopped crawling. After some more research I found that they are searching for copyright material for clients and submit a DMCA on violators if they don't comply. The bastards can manually crawl my site if they want, their bots (3 or 4 at a time) were hitting my site pretty hard. Now error log shows them being denied. If they don't obey the robots file the get banned!  |
|
|
|
|
|
|
|
|
10-18-2007, 07:52 PM
|
#7 (permalink) |
|
Moderator
Senior Member
Join Date: Dec 2006
Location: Under Your Mouse!
Posts: 1,208
|
Try owning mssupport.net !
I still own it but took down the online community and just have 1 page wikipedi page. When I first had it many years ago, MS was all over it. Every page/folder for weeks. They didnt find anything and moved on. RIckyG
__________________
Arcade Banner Exchange, game packs, top site, arcade script and forum for all of your arcade needs http://www.mibbi.com This site stinks!! www.farting.us Coming soon MibbiJr! |
|
|
|
|
|
Linear Mode
