Run > 1024 docker containers

Question

Normal Docker is limited to 1023 containers due to the bridge being limited to 1023 MAC addresses.

But what if you have several bridges? Can you then have more than 1023 containers?

And can you connect those bridges by routing or by bridges of bridges?

If yes: Please show a config that does this - preferably one that makes the containers available to the external ethernet.

Answer 1

It is possible. With this I have managed to run 6000 containers

    servers_min=$1
    servers_max=$2
    # We will start 10x this
    servers_min=$(( servers_min / 10 ))
    servers_max=$(( servers_max / 10 ))
    testssh() {
        ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/tmp/known root@"$1" echo "'$1'" '`uptime`'
    }
    export -f testssh
    docker network create --driver bridge net0
    docker network create --driver bridge net1
    docker network create --driver bridge net2
    docker network create --driver bridge net3
    docker network create --driver bridge net4
    docker network create --driver bridge net5
    docker network create --driver bridge net6
    docker network create --driver bridge net7
    docker network create --driver bridge net8
    docker network create --driver bridge net9
    # Start the virtual servers
    startone() {
        id=$1
        net=$2
        docker run -d --rm --name ubuntu-$id-$net --network $net my-ubuntu
        docker inspect ubuntu-$id-$net
    }
    export -f startone

    seq $servers_min $servers_max | parallel startone :::: - ::: net{0..9} |
        # After this it is possible to do:
        #   ssh 10.0.0.99
        # from another physical server
        perl -nE '/"IPAddress": "(\S+)"/ and not $seen{$1}++ and say $1' |
        tee /tmp/ipaddr |
        parallel testssh
    docker ps

The servers can access each other within the same bridge and the internet, but they are not accessible from the ethernet.