IP blocklists help prevent attack traffic, but can sometimes block regular users from accessing sites
There’s no dearth of information on how to block people from social and mobile accounts. But what about when you get blocked by your own internet service provider (ISP)?
Internet users sometimes find themselves blocked from certain websites – even though they haven’t done anything wrong, they learn that their IP address has been put on a “blocklist,” denying them access to the site.
An IP blocklist is made up of IP addresses that are known to be malicious, and network operators use these lists to help defend their network by blocking attack traffic, which can spam servers and flood the network with data.
“There are so many users on the internet, and some of them are malicious,” said Sivaramakrishnan Ramanathan, PhD student at USC’s Information Sciences Institute (ISI). “Finding the bad guys is hard, and finding the traffic generated by these bad guys is even harder.”
Ramanathan is the lead author of “Quantifying the Impact of Blocklisting in the Age of Address Reuse.” His team researched methods to measure IP address reuse among blocklisted addresses so they could quantify the impacts that blocklisting has on regular users who are sharing their IP address with an attacker.
The research project is sponsored by the DHS’s Science and Technology Directorate, Homeland Security Advanced Research Projects Agency, and Cyber Security Division, as well as the UK government. Along with USC researchers, the project also includes contributions from Anushah Hossain (UC Berkeley), Minlan Yu (Harvard University), and Sadia Afroz (International Computer Science Institute/Avast Software).
To Block or Not to Block
While there are numerous security techniques that can help identify bad traffic, they require a lot of resources – this is why blocklists are one of the simplest ways to deter attack traffic. Through blocklists, attackers can be identified and blocked by their IP address, serving as an effective first layer of defense.
While IP blocklists are key in protecting networks, however, they can sometimes unfairly block regular users that aren’t bad actors. This is because IP addresses are usually reused.
An IP address is like a golden ticket that enables you to use the internet. But, just like golden tickets, they’re limited – there are only 4.2 billion IP addresses available, and many more people and devices that require IP addresses in the world. Because of this, IP address tend to run out.
“To overcome this shortage, ISPs use a technique of NAT (Network Address Translator) to map many users to one IP address,” Ramanathan explained. “Essentially, all these users will be represented by the same IP address.”
The NAT technique to reuse IP addresses can pose a problem, however. With the same IP address representing a group of users, if even one user is a bad actor, that’ll reflect onto the whole group. This can cause multiple users to be blocklisted, even if they weren’t bad actors.
Another method of IP address reuse is called dynamic addressing, where one IP address can be allocated to many users over time. But this can lead to issues as well – if a bad user was allocated an IP address on Monday and then the same IP address is allocated to a good user on Saturday, the good user may potentially be identified as a bad actor because of the address reuse.
New Techniques to Analyze Blocklisting Impacts
So how can network operators avoid unfair blocking? Currently, it can be quite a challenge.
“There’s no established method today to detect IP address reuse,” said Jelena Mirkovic, PI of the project, research associate professor of computer science at USC Viterbi, and project lead at ISI.
While most published research into blocklisting focuses on the issue of detecting groups of IP addresses that may be reused, the ISI team looked into detecting individual addresses and quantifying IP address reuse, which can also be challenging.
“Quantifying address reuse is difficult because most published works use auxiliary, private data sources [to do so], such as popular web server logs or ISP-level monitoring,” Mirkovic added. “Our methods are reproducible by other researchers.”
To tackle issues with NAT-based address reuse, the ISI researchers used BitTorrent, the popular peer-to-peer app that enables users to share content, to help quantify IP address reuse.
“We make use of a BitTorrent crawler,” said Ramanathan. “The crawler goes through the set of BitTorrent users and sees which users have been allocated the same IP address. It also verifies if the users behind the same IP address are active to ensure there’s no stale information with BitTorrent.”
When it comes to the problems with dynamic addressing, Ramanathan’s team utilized RIPE Atlas measurement logs. RIPE Atlas is like the thermometer of the internet, serving as a global network of probes that collect data to measure the health of the internet in real time.
“RIPE provides an infrastructure for researchers to observe the internet from several vantage points,” Ramanathan elaborated. “The infrastructure is spread over several volunteers who have dedicated hardware in their premises, which frequently sends measurement reports to RIPE along with the IP address of the hardware.”
The team observes the reports over a six-month period and analyzes all the IP addresses that have been allocated to the hardware. This enables the team to identify users who may be impacted when a previously blocklisted IP address is allocated to them.
In order to help blocklist maintainers reduce unfair blocking, the team has made their discovered reused addresses public.
If you ever find yourself blocklisted, contact your ISP to request your IP address get unblocked.
—-
The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the Department of Homeland Security, the U.S. Government, or the Government of United Kingdom of Great Britain and Northern Ireland.
Published on April 15th, 2021
Last updated on May 7th, 2021
