python

This script extracts the ip addresses from web access logs for a specific user agent and then via who is ip services checks the ip addresses to compile a list of CIDRs. These can then be used to block a bot via firewall.

A service which offers an endpoint to submit changed to external URLs. The service then updates the database accordingly.

This service generates a dump once a day.