1

I am using iptables with ipset on an Ubuntu server firewall. I am wondering if there is a command for importing a file containg a list of ip's to ipset. To populate an ipset, right now, I am adding each ip with this command:

ipset add manual-blacklist x.x.x.x

It would be very helpfull if I can add multiple ip's with a single command, like importing a file or so.

At command

 for ip in `cat /home/paul/ips.txt`; do ipset add manual-blacklist $ip;done

I get this response

resolving to IPv4 address failed to parse 46.225.38.155

for each ip in ips.txt

I do not know how to apply it.

  • Well, why are you using two different commands?  ipset add badipipset add manual-blacklist. – G-Man Says 'Reinstate Monica' Nov 2 '17 at 5:29
  • Sorry, I copied the wrong command, the good one is ipset add manual-blacklist x.x.x.x – byte00 Nov 2 '17 at 5:30
  • I'll bet that your ips.txt file was created under DOS/Windows and has CR/LF on the line ends, so tyour $ip variable has an IP address ending in the bad character CR. – Ian D. Allen Sep 10 '18 at 21:26
6

You can use ipset save/restore commands.

ipset save manual-blacklist

You can run above command and see how you need to create your save file.

Example output:

create manual-blacklist hash:net family inet hashsize 1024 maxelem 65536 
add manual-blacklist 10.0.0.1 
add manual-blacklist 10.0.0.2

And restore it with below command.

ipset restore -! < ips.txt

Here we use -! to ignore errors mostly because of duplication.

  • 1
    It must be noted that restore works blazingly fast compared to add when adding lots of records. On my PC it took 17 sec to add 10000 lines, and only 0.05 sec to restore – AlexM Dec 5 '18 at 5:53
0

Try this command:

for ip in $(cat </file.txt>); do ipset -A <set-name> $ip;done

If you still get the a error then check your text file for spaces and unreadable\weird characters in a advanced text editor (Notepad++, SublimeText).Delete the spaces and unreadable\weird characters and try again.

  • Note that it is much quicker to use restore than the ipset -A command. – Kohjah Breese Feb 11 at 0:44
0

If your list is geting really realy big for example 200 000 lines or even more and you have enough memory in server to support this, it is nice to run this whole oneliner cycle with nice:

nice -n 5 bash -c "for IP in \$(cat textfile.txt); do ipset add <setname> \$IP -exist timeout <seconds>; done"

Then your other services can step up from bash and you do not get any networking or mysql or apache or other services lag because bash is using all the resources.

  • If you have to add many lines, it's better to use restore command, which works very fast as opposed to adding the lines one by one – AlexM Dec 5 '18 at 5:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.