Saturday, February 18, 2006

Good news

First, we've moved to the new server.

Second, we've recovered from recent problems caused by kind of DOS attack when one of our feeds has been requested every single second by some weird spider. This IP is banned forever, and the owner of this IP should have been already warned by it's hoster.

After some investigation, I have realized that there are some user agents/spiders that do not obey "ttl" parameter of the feeds completely and pull data every 5 minutes or so. This is not a 1-second interval, but nevertheless. To avoid our service lockouts in the future, and to give more air to fellow users that behave nice, I'm planning to create some slashdot-alike automated service to temporarily ban IPs that pollute our service with requests.

Third, I've created a 'CPU protection' feature that tests search patterns against their CPU usage, and recovers from difficult situations gracefully, asking user to refactor search pattern to make it faster (I've added some help on that into context help).

I would like to thank everyone for patience during this incident and want to specially mention Eliot and Blake who donated some money for a faster server. Thank you guys!

No comments: