From time to time I’ve wanted to collect information from readers. I have adopted and discarded various polling plugins. But, they keep changing and being a pain. So, seeing this come through my mail I thought I better take a look.
Let me know if you start having problems with this site. I have been upgrading the security. About 2 or 3 PM each day I see the site slammed by remote sites sucking gigabytes of content out of my site. I’m not sure why they would want it. Nor do I mind that they take it. I do mind that they overload the web server I use and make the site way slow. So, I come in each day and disconnect those that are behaving like bots… continuous dozens-of-requests per second.
On The Sunny Side of the Street
I’ve been changing robots.txt and adding ‘slow down’ instructions for search engines. That helps with the big well mannered engines. But, not the problem bots that have been hitting.
I’ve been blocking the networks the most abusive bots use. But, doing that without restricting legitimate users is complicated. So, it isn’t a good solution. Neither is blocking individual IP addresses. The bad guys just change IP.
So, now I am trying some new software. It is in learning mode. So, I’m ‘completely’ sure what all it will decide to do.
Last month was my rez-day, March. I joined in 2008 some time after Myst Online: Uru Live closed for the second time. Several of my friends from Uru were in Second Life, so I tried it. I liked it. I stayed.
I’m in the process of trying to stop some content scrappers that that regularly hit this site and suck gigabytes of data as fast as they can. It drives the server CPU to 100%, eats bandwidth, and makes this site extremely slow.
In an effort to discourage scraping, I have disallowed ‘hot linking’ of images. If your blog has used images from this site via a link rather than making a copy of the image, let me know. I can allow your site access. Whether I will depends on how you used the images.
Most of the site is cleaned. I am running on all new plugins. Removing others. Everything is only 3 months old or newer. I still have a few to re-install. I’m still trying to figure out which search engines to block. I’m not familiar with European Arabic, and Asian engines.
The software I am using doesn’t organize information as well as it might. Plus timing is fuzzy. But, it is free and has some other features I like, so I’ll keep it.
Today you’ll notice the ads are back. They pay for my hosting. So, deal with them and know I appreciate your patience with them.
I’ve changed over to Google’s ReCAPTCHA. The other company providing Recaptcha was an advertising company and may have been part of the problem with my site. I think it is easier to use. Let me know.
This site went down Tuesday morning. The cause: several people are hitting the site causing problems.
You may have seen the site changing and going off-line Tuesday.
I was in the process of changing things and hardening the site. I thought I had it well protected, but I guess not. It should be harder now. Instead of using several different protective apps I’ve consolidated to one.
I thought I might have to block blocks of IP addresses. So, I was worried I might block some readers so I advised, “If you get blocked from the site, contact me in world.” But, I haven’t has to. I do block some single IP addresses.
There are a couple of search engines I’ve never heard of that abuse the site. I’m tweaking settings to slow them down.
I am slowly replacing plug-ins. I have to remove all the plug-in files and replace them with factory versions. That is tedious. I was surprised to see some of my apps hadn’t updated in 6 years. There were no new versions. I suspect this is where my problem got started.