How to import multiple ip's to Ipset?

Question

I am using iptables with ipset on an Ubuntu server firewall. I am wondering if there is a command for importing a file containg a list of ip's to ipset. To populate an ipset, right now, I am adding each ip with this command:

ipset add manual-blacklist x.x.x.x

It would be very helpfull if I can add multiple ip's with a single command, like importing a file or so.

At command

 for ip in `cat /home/paul/ips.txt`; do ipset add manual-blacklist $ip;done

I get this response

resolving to IPv4 address failed to parse 46.225.38.155

for each ip in ips.txt

I do not know how to apply it.

Answer 1

You can use ipset save/restore commands.

ipset save manual-blacklist

You can run above command and see how you need to create your save file.

Example output:

create manual-blacklist hash:net family inet hashsize 1024 maxelem 65536 
add manual-blacklist 10.0.0.1 
add manual-blacklist 10.0.0.2

And restore it with below command.

ipset restore -! < ips.txt

Here we use -! to ignore errors mostly because of duplication.

Answer 2

If your list is geting really realy big for example 200 000 lines or even more and you have enough memory in server to support this, it is nice to run this whole oneliner cycle with nice:

nice -n 5 bash -c "for IP in \$(cat textfile.txt); do ipset add <setname> \$IP -exist timeout <seconds>; done"

Then your other services can step up from bash and you do not get any networking or mysql or apache or other services lag because bash is using all the resources.

Answer 3

I know this is a few years old but why not just make a simple shell script with all the IP addreses and then run that shell script. I add hundreds of IP addresses this way.

Example: (done in Centos 8 using vi as the editor)

vi manual-blacklist.sh

Then in the manual-blacklist.sh file add all your IP addresses as listed below: (Press the i key to insert your lines)

ipset add manual-blacklist 123.45.67.111
ipset add manual-blacklist 123.45.67.112
ipset add manual-blacklist 123.45.67.113

And so on until all your IP addresses are added. To speed things up I use Excel to make my lists of IP addresses and then export it as a text file and then do a copy and paste into my shell script file.

Save the file. Press ESC key and then :wq! and then press the Enter key to save the file.

Then to run it use the following command:

./manual-blacklist.sh

All your IP addresses will be added to your manual-blacklist ipset file. And if it notices any duplicates it will ignore them as they were already added prior.

Then save this file: (I save it to my home directory and the etc directory)

ipset save > /etc/ipset.conf
ipset save > /home/username/ipset.conf

And to restore:

ipset restore -f /etc/ipset.conf

The -f switch specifies a filename to print into instead of stdout (list or save commands) or read from instead of stdin (restore command).

Hope this helps.

Answer 4

Try this command:

for ip in $(cat </file.txt>); do ipset -A <set-name> $ip;done

If you still get the a error then check your text file for spaces and unreadable\weird characters in a advanced text editor (Notepad++, SublimeText).Delete the spaces and unreadable\weird characters and try again.