Core Features

Whenever a user is attempting to read way more data than expected in a really short period of time, one is quickly marked as a hacker and if the behavior lasts a little too long (i.e. the hacker's robot doesn't follow our requests to slow down its requests) then we block one's IP address using snapfirewall.

Our Snap! Websites implementation counts HTML pages and all the various attachments (images, JS, CSS, etc.) as two separate groups. Each access is logged using Cassandra using the HTTP status, date, type, and URI of each hit. This way we can use the count() functions of Cassandra to know how many hits we received.

The information is saved in Cassandra so a backend could do more work on the data, but the blocking happens in realtime (as the user hits the website, the counts are checked right then.) Note, however, that the data is made very ephemerous (10 minutes) using the Cassandra TTL feature. So without any hits in 10 minutes, the table goes back to being empty, which should ensure that the count() function remains fast.

The number of hits you need to get blocked can be changed in your antihammering settings. It also defines various periods to know how to count things and how long a hacker gets blocked when we detect too many hits.

Implementation

This feature makes use of a separate table named "antihammering".

The rows are the IP address of the user accessing Snap! (the 128 bits of an IPv6) This way, each time that specific user accesses the server, we can know who they are. That allows us to very quickly track what is happening and as a result gives us the means to block access to that specific IP address.

The data saved in that row will represent what the user has been accessing in the last little bit (i.e. here we can make all the data temporary since after a few minutes, we do not need to track users anymore, although such data could also be useful to generate graphs of where people go, etc. but that's done by the tracker plugin instead.) If we find too many entries in that table, we can block the user.

The key of columns in the antihammering table are defined with the following parameters:

code--the HTTP code such as 200, 404, and 501 (we do not add 3XX codes, not useful)
type--representing the type of page: "html-page" or "attachment"
date--the date in microseconds composed of 19 digits

The value saved in the column is the URL, although at this point we ignore that point. Whether the hacker hits the same page or various pages over and over, he gets blocked.

Since we save the HTTP code, we can count the number of successful hits and the number of errors and calculate a ratio (not implemented yet) and if that ratio is too large, we can block the user.

At this time we count and block users who hit too many HTML pages. We want to extend the system to do the same with various types of errors and calculate a ratio of valid hits and erroneous hits. Finally work on attachments to properly count the number of hits to specific attachments. So far I have not see as many problems with attachments, but we certainly want to look into this problem too.

Anti-Hammering with Debug

In debug mode, the system locks the database on each access. This has the side effect of serializing each hit. In regard to anti-hammering, this means the system will not be counting quick hits to the system that quickly. So in order to detect "quick" hits in debug mode, you have to substantially increase the amount of time for which hits are counted in order to know whether the anti-hammering is triggered. Once the anti-hammering is triggered, though, the debug mode has much less influence and further hits are counted very quickly as it should.