CAUTION! This is experimental. Please report problems to us!

Summary

This is a free service available to all DShield log contributors.

DShield.org in collaboration with SRI International has established a new experimental custom source address blocklist generation service available to all DShield.org contributors. This new service utilizes a radically different approach to blocklist formulation called Highly Predictive Blocklisting. Each DShield contributor can now access a unique HPB (instructions below) that reflects the most probable set of source addresses that will connect to that contributor's network over a prediction window that may last several days into the future.

Highly predictive blocklists employ a link analysis algorithm similar to Google's PageRank scheme used to find the most relevant web pages given a user's query. Similar to a web query, DShield contributor's firewall logs are cross-compared in search of overlaps among the attackers they report. Each attacker address that is included in an HPB is selected by favoring those ad-dresses that are encountered by other contributors that share degrees of overlap with the HPB owner.

How does it work (for non math geeks ;-) ): We compare your firewall logs to firewall logs submitted by others. If you and other submitters are hit on similar ports, then your are more likely to be attacked by the same IPs. Your personal "HPB" is created from the IP addresses that target submitters with similar reports as you.

Why does it work: Let take port 1434 as an example. Port 1434 is attacked "a lot". Many of our worst offenders attack this port. However, maybe your ISP is already blocking port 1434. So blocking a pure 1434 scanner wouldn't do you any good. We know that your ISP blocks port 1434, because you never submit any reports for port 1434. So if we create a blocklist from all users that never report hits on 1434, we are likely to create a better blocklist for you.

Top of page

How to Access Your HPB

First of all, you have to be a DShield submitter. We can not compute the blocklist if you do not submit data or if you only submit anonymous data.

For more details about the algorithms involved, see Cyber-TA!

  1. Log In
  2. Click on "My Information"
  3. At the bottom of this page, you will find the URL for your HPB.

The URL will look like:

http://www.dshield.org/hpb.html?key=oiUTq74ue5KvKQXfZYxsXw==
(this particular 'key' is our demo key and should only be used for testing)

You can use this particular link to test. But the code at the end will change for your account. The format is identical to our regular "top 10 worst offender" blocklist. You can use whatever script you use to pull the old blocklist. Just replace the URL accordingly.

The Blocklist is recalculated once a day.

Format

  • All lines starting with a '#' are comments and should be ignored.
  • The list starts with a header line (Which is not a comment)
  • The list is tab delimited
  • Each row lists one /24 network
  • All IP addresses are '0' padded for 3-digits per byte. So 100.10.1.0 becomes 100.010.001.000.
  • The following columns are used
    1. First IP Address in netblock (always ends with '000')
    2. Last IP Address in netblock (always ends in '255')
    3. netblock (right now always '24')
    4. How many hosts reported attacks from this netblock
    5. Name of the network (optional)
    6. Country of the Network (optional)
    7. Abuse contact email address (optional)
Top of page