firehol-mirror: A Git-Based Cache for Firehol IP Lists Without Private Networks

What Problem It Solves

Firehol maintains a large collection of IP blocklists at iplists.firehol.org. These lists are updated on a schedule, and fetching them directly can be inefficient. The mirror solves a specific annoyance: the original lists include private network addresses (RFC1918), which are useless for public-facing firewall rules. It also addresses two data-quality issues. The .netset files contain subnets, while the .ipset files contain single IPs, and the two can drift out of sync. Old or deprecated lists clutter the collection, and there is no easy way to see which lists are falling behind. The mirror provides a git repository that caches the lists, strips private IPs, archives old lists, and makes the data easier to analyze.

The Mechanism: Cron, Git, and TTL

The project works by running on a cron schedule. It fetches the blocklists from iplists.firehol.org, but it does not fetch every time. The README states that it sets a proper TTL to avoid spamming the upstream site. That means the mirror respects the freshness of the data and only pulls when the lists are due for an update. The fetched content is committed to a Git repository. Git becomes the transport protocol, which is more efficient than repeatedly downloading plain text files. Each commit captures a snapshot of the lists, so you can diff between versions. The README also mentions that it auto-archives old or deprecated blocklists that are older than one month. This keeps the active set clean without deleting history.

Removing Private Networks and Fixing Inconsistencies

A key transformation is the removal of all private IPs. The README says it removes all private IPs, which means RFC1918 ranges like 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16 are excluded from the cached lists. This is useful for firewall rules that only care about public internet traffic. The project also fixes inconsistencies between .netset and .ipset files. The .netset format uses CIDR subnets, while .ipset format uses individual IP addresses. If a list says a subnet is blocked but the corresponding .ipset does not include all the IPs in that subnet, the mirror corrects that. The exact method is not documented in the README, but the goal is to make the two representations agree.

Getting It Running

The README does not include installation or configuration commands. That is a gap. The repository is a cron-based script, but the exact invocation is not shown. You would need to clone the repository and inspect the code to find the cron entry. The project likely expects you to set up a cron job that runs the mirror script on a schedule. The TTL is a configuration value, but the README does not name the file or key. You will have to read the source to determine how to set the TTL and where the output goes. This is a limitation for users who want a plug-and-play solution.

Limitations and Failure Modes

The mirror depends entirely on the upstream Firehol site. If iplists.firehol.org is down or changes its format, the mirror will fail. The cron-based approach means updates are not real-time. There is always a delay between an upstream update and the next cron run. The removal of private IPs is a feature, but it is also a limitation if you need the original lists for any reason. The consistency fixes between .netset and .ipset could introduce changes that are not present upstream. The README does not explain how it resolves conflicts, so you must trust the implementation. Also, the project is a personal effort, not officially maintained by Firehol. The license is unknown, which may be a concern for some users.

Alternatives: Direct Fetching or Other Mirrors

The obvious alternative is to fetch the lists directly from iplists.firehol.org using curl or wget on your own schedule. That approach gives you full control over the TTL and the exact content, but it does not give you git history or automatic consistency fixes. Another alternative is to use Firehol's own tools, such as the firehol scripts that generate these lists, but that is a heavier dependency. The key difference is that firehol-mirror adds a git layer, which enables diffing and statistical analysis. If you do not need those features, a simple cron fetch is simpler. If you need a public mirror, other projects exist, but they may not strip private IPs or fix .netset/.ipset mismatches.

Maintenance and Upgrade Cost

The maintenance cost is tied to the cron job. You must keep the mirror script running, and you must monitor the git repository for failed fetches. The TTL setting reduces the load on the upstream, but you still need to ensure the cron environment has network access and git installed. Upgrades are manual: you pull the latest changes from the repository. Since the project has no releases, you cannot rely on tagged versions. The unknown license is a risk for commercial use. You should contact the author or check the repository files for a license file before adopting it. The README says the project is fun, which suggests it is a hobby project, so support may be limited.

Editorial conclusion

Adopt firehol-mirror if you need a stable, git-versioned copy of Firehol blocklists with private networks removed, and if you can tolerate a cron-based workflow that depends on the upstream Firehol site. Do not use it if you require real-time updates or if you need the original private-IP addresses. Before adopting, verify that the mirror's TTL and archiving policy match your update frequency, and inspect the git history to confirm the consistency fixes behave as expected for the specific lists you consume.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository