I run a busy website on a Debian server that utilises Shorewall for most of the traffic management.
I need to work out a way to shape excess traffic so that all users can burst up to a maximum speed of 1MB/s for say 60 seconds. But any source IP that continues to use that speed past 60 seconds should then be shaped back to 256kb/s, for example.
This is to slow down (but not block!) downloader bots that many of my users like to use to grab huge sections of our asset dirs, but maintain full speed for regular users.
I do not want to have to keep adding useragents to a blacklist as new bot useragents appear often, and some even lie. I just want to shape anything that is using too much bandwidth past a certain amount of time per source IP.
I figure that this can probably be done with Shorewall. However I don't understand how to implement my specific requirement based on their TC docs. I'm one of those people that understands things fast when working backwards from a good example, but detailed technical docs with few relevant examples are not much help for tools with very complex setup.
I'm happy to use any other software that I can install as a package from Debian stable, if there's something better to use than Shorewall for this task.