In this example which is a bit more complex, we are using multiple domains listed in a text file domains.txt to build a whitelist of IPs, and we prevent IPs from being deleted from UFW whitelist until 300 seconds after they were last observed in a DNS result.
The syntax of my ufw command is somewhat different as I have slightly different use case of allowing the traffic to route across specific interfaces.
I set this script to run at boot time in rc.local and it does a good job of maintaining the UFW with the whitelist, with minimal CPU consumption.
I should note that this solution is not really scalable to more than 100-200 domains because of CPU load of ufw command and the latency required for each ufw command, and for more than this number of domains (and probably less too) we should probably write the iptables rules into /etc/ufw/user.rules manually, and then "ufw reload".... which would likely be a much more scalable approach.
It generates alot of DNS traffic, so best to use an internal caching DNS server (e.g. Pihole, etc.) and point the host towards it.
#!/bin/bash
declare -A ip_whitelist_lastseen_times
while :
do
echo "Reading domains..."
DOMAINS=$(cat domains.txt)
rm ip_whitelist.txt
# Generate whitelist based upon DNS query
echo "Generating whitelist..."
for DOMAIN in ${DOMAINS[@]}
do
host $DOMAIN | grep 'has address' | cut -f4 -d ' ' >> ip_whitelist.txt
done
echo "Whitelist generated, now we will sort it and remove non-unique entries. "
cat ip_whitelist.txt | sort | uniq > ip_whitelist_unique.txt
echo "Indexing whitelist into array"
# Read the whitelist into array.
unset new_ip_list
IFS=$'\n' read -r -d '' -a new_ip_list < <( cat ip_whitelist_unique.txt && printf '\0' )
echo "Reading list of currently loaded rules."
# Get the UFW current list of IPs with WHITELIST description
# Read the existing entries, into an array.
unset old_ip_list
IFS=$'\n' read -r -d '' -a old_ip_list < <( /usr/sbin/ufw status | grep "WHITELIST" | tr -s ' ' | cut -f1 -d ' ' && printf '\0' )
echo "resetting and Flipping array of whitelist"
# Flipping array, so it can be searched by index
# -searching array by index is faster than looping through the
# same array in bash repeatedly for every new/old entry
# and then making same comparisons repeatedly
unset new_ip_list_index
declare -A new_ip_list_index
for new_ip in "${new_ip_list[@]}"
do
new_ip_list_index[$new_ip]=1
done
echo "resetting and flipping array of currently loaded rules"
unset old_ip_list_index
declare -A old_ip_list_index
for new_ip in "${old_ip_list[@]}"
do
old_ip_list_index[$new_ip]=1
done
#: for each new IP, check if it is in the existing list of IPs
#: If it is not, then we need to add a rule
echo "iterating to add rules"
for new_ip in "${new_ip_list[@]}"
do
if [ ! -v "old_ip_list_index[$new_ip]" ] ; then
/usr/sbin/ufw route allow in on enp5s0f0 out on enp5s0f1 from 192.168.0.0/24 to $new_ip port 80 proto tcp comment WHITELIST
/usr/sbin/ufw route allow in on enp5s0f0 out on enp5s0f1 from 192.168.0.0/24 to $new_ip port 443 proto tcp comment WHITELIST
sleep 0.5
fi
# Here we will make a array entry indicating the last time the host was seen in the whitelist
ip_whitelist_lastseen_times[$new_ip]=$(date '+%s')
done
echo "iterating to remove rules"
#: for each existing IP in the ufw status list, if it is not in the new host list, it is due for removal
current_time=$(date '+%s')
for old_ip in "${old_ip_list[@]}"
do
if [ ! -v "new_ip_list_index[$old_ip]" ] ; then
# marking eligible for deletion by calculating delta
delta=$current_time
if [ -v "ip_whitelist_lastseen_times[$old_ip]" ] ; then
let "delta = $current_time - ${ip_whitelist_lastseen_times[$old_ip]}"
fi
if [ "$delta" -gt "300" ] ; then
/usr/sbin/ufw route delete allow in on enp5s0f0 out on enp5s0f1 from 192.168.0.0/24 to $old_ip port 443 proto tcp
/usr/sbin/ufw route delete allow in on enp5s0f0 out on enp5s0f1 from 192.168.0.0/24 to $old_ip port 80 proto tcp
sleep 0.5
unset ip_whitelist_lastseen_times[$old_ip]
fi
fi
done
sleep 30
done
