FedoraForum.org - Fedora Support Forums and Community
Results 1 to 7 of 7
  1. #1
    Join Date
    Jan 2017
    Location
    USA
    Posts
    2

    Bridge broken after docker install

    I have a very basic setup on a fresh install of Fedora 25 Workstation for some virtual machines through virt-manager. I have a bridge (br0) setup for the VMs to use with the ethernet as its slave device. I've tried a number of different things but no matter what I do when I install docker, the bridge breaks for the VMs. The host will still be connected and reachable but then the VMs will time out trying to raise their interfaces. Remove docker and the problem goes away.

    I've done a bit of diffing between with docker and without (through a LVM snapshot) and have not come up with anything obvious.

    I've added the bridge after docker, I've removed docker's bridge and it seems no matter what I do docker breaks my bridge for my VMs. I would love to have them both working on this one machine and know it has to be possible (I mean the world wouldn't make sense if it wasn't possible) but I've been unable to figure out what the magic bullet is in quite a few hours of trial and error/reading.

    I'm using Network Manager's cli to setup the bridge, similar to what can be found here (https://www.server-world.info/en/not...edora_23&p=kvm). But I do the ipv4.method as auto instead of static.

    Does anyone have any ideas? TIA....



    tim

  2. #2
    Join Date
    Aug 2016
    Location
    Dallas
    Posts
    49

    Re: Bridge broken after docker install

    have you looked for a difference in the bridges network scripts before an after? I would use cockpit http://cockpit-project.org/ you can configure just about anything inside of it, including docker and bridges, all right out of the box.

    Code:
    sudo dnf -y install cockpit*
    sudo systemctl enable cockpit.socket
    sudo systemctl start cockpit
    sudo firewall-cmd --permanent --add-port=9090/tcp
    firewall-cmd --reload
    Another thing you can do is follow the logs during the bridge failure. Run this when the bridge is working properly, all the way through failure. Then open it and get a cup of coffee and dig in

    Code:
    journalctl -f >> ~/Documents/BridgeLogs.txt

  3. #3
    Join Date
    Jan 2017
    Location
    USA
    Posts
    2

    Re: Bridge broken after docker install

    Thanks for the suggestions. I have done / am doing that and the one thing that immediately stands out is that when I install docker I get a lot of messages of this variety (about failure):

    Jan 02 09:46:26 localhost.localdomain firewalld[927]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE' failed:
    Jan 02 09:46:26 localhost.localdomain audit: NETFILTER_CFG table=nat family=2 entries=63
    Jan 02 09:46:26 localhost.localdomain firewalld[927]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t nat -C DOCKER -i docker0 -j RETURN' failed:
    Jan 02 09:46:26 localhost.localdomain audit: NETFILTER_CFG table=nat family=2 entries=64
    Jan 02 09:46:26 localhost.localdomain firewalld[927]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -D FORWARD -i docker0 -o docker0 -j DROP' failed:
    Jan 02 09:46:26 localhost.localdomain firewalld[927]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t filter -C FORWARD -i docker0 -o docker0 -j ACCEPT' failed:
    Jan 02 09:46:26 localhost.localdomain audit: NETFILTER_CFG table=filter family=2 entries=105
    Jan 02 09:46:26 localhost.localdomain firewalld[927]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t filter -C FORWARD -i docker0 ! -o docker0 -j ACCEPT' failed:
    Jan 02 09:46:26 localhost.localdomain audit: NETFILTER_CFG table=filter family=2 entries=106
    Jan 02 09:46:26 localhost.localdomain firewalld[927]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t filter -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT' failed:
    Jan 02 09:46:26 localhost.localdomain audit: NETFILTER_CFG table=filter family=2 entries=107
    Jan 02 09:46:26 localhost.localdomain firewalld[927]: WARNING: COMMAND_FAILED: '/usr/sbin/iptables -w2 -t filter -C FORWARD -o docker0 -j DOCKER' failed:


    Not sure if this is the source of my pain or not. When I remove docker the bridge for the VMs does not work right away, I have to reboot. But then it goes back to working perfectly....



    Thanks,


    Tim

    ---------- Post added at 10:50 AM ---------- Previous post was at 10:46 AM ----------

    And also, the VM doesn't have trouble raising its interface after I simply install docker, it's only after I start docker that there's an issue (all without reboot).

  4. #4
    Join Date
    Aug 2016
    Location
    Dallas
    Posts
    49

    Re: Bridge broken after docker install

    I know very little about docker, but my guess is that docker0 (the interface it automatically creates when you install it) is itself a bridge. I'm assuming you only have a single NIC, and if both of your bridges (docker0 and br0), are trying to control that interface, it may be causing you problems. But again, I know very little about docker.

    I'd refer to dockers documentation on customizing networks:

    https://docs.docker.com/engine/userg...ustom-docker0/

  5. #5
    Join Date
    Jan 2010
    Posts
    19

    Re: Bridge broken after docker install

    Did you manage to solve this problem? I have exactly the same problem so would love to know the solution

  6. #6
    Join Date
    Apr 2017
    Location
    Russia, Moscow
    Posts
    1

    Re: Bridge broken after docker install

    I'm fighting this issue too, but on Debian 8. KVM + Docker = my broken heart.

    Seems that this issue happens because Docker loads
    Code:
    br_netfilter
    module and sets
    Code:
    net.bridge.bridge-nf-call-arptables
    net.bridge.bridge-nf-call-iptables
    net.bridge.bridge-nf-call-ip6tables
    kernel parameters to 1 (or it is default module config) so all bridged traffic is now subject to iptables filtering.

    See following links about br_netfilter:


    I haven't yet figured out how to fix this issue, because simply blacklisting br_netfilter won't go - Docker needs this module to filter bridged connections between containers that are created with 'link' option. Filtering bridged connections is necessary to Docker to properly handle '--icc' flag, which enables or disables inter-container communication.

    I suppose that we need to figure out some Iptables rule that will ACCEPT all bridged traffic except for traffic going through Docker bridges. One way is to add such rules for each of your custom bridges:
    Code:
    iptables -A FORWARD -i <your-bridge> -o <your-bridge> -j ACCEPT
    But I think it is not flexible solution.

    Could someone propose a better approach?

  7. #7
    Join Date
    Aug 2017
    Location
    St. Louis, MO
    Posts
    1

    Re: Bridge broken after docker install

    I just ran into this problem as well, on Ubuntu 16.04. It occurred on a random update, not a fresh install, so something about the docker package must have changed how it was working.

    Anyway, I just set the following in my /etc/sysctl.conf file and it seemed to work.

    Code:
    net.bridge.bridge-nf-call-arptables = 0
    net.bridge.bridge-nf-call-ip6tables = 0
    net.brdige.bridge-nf-call-iptables = 0
    Did I miss something, is there a reason this isn't good to do? My bridged VMs are back to working correctly and docker seems to be working fine as well.

Similar Threads

  1. docker-io/docker-registry
    By stevea in forum Using Fedora
    Replies: 0
    Last Post: 13th May 2014, 08:05 PM
  2. HOW TO Install Fedora 14 on sundy bridge laptop
    By megaloman in forum Installation, Upgrades and Live Media
    Replies: 0
    Last Post: 8th August 2011, 11:19 PM
  3. Replies: 1
    Last Post: 6th April 2009, 10:35 AM
  4. network bridge setting up bridge-utils
    By brw02005 in forum Servers & Networking
    Replies: 0
    Last Post: 28th March 2007, 11:08 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •