[kwlug-disc] Multiple NICs on Ubuntu Server 14.04

[Chris Irwin]

I'm posting from work, so I'm using gmail. Sorry for any weird formatting.

On Fri, Aug 15, 2014 at 3:56 PM, unsolicited <unsolicited at swiz.ca> wrote:

> Thanks for the message.
>
>
> > - The second NIC is an 'UP', but an unconfigured member of a bridge
> > (brvm), which is also 'UP' but unconfigured. My virtual machines all
> > connect to the brvm bridge for direct network access.
>
> Huh? 'UP'? As in catches DHCP from house router / not specifically
> configured?
>

No, no IP address, but the interface isn't 'DOWN'. Here's a snipped portion
of my IP information. (Also, I use br_lan instead of brvm, which is easier
to understand)

- p4p1 is my only ethernet device with an IP address (Just noticed that I
still have ipv6 enabled on the other interfaces, but I don't use it).

- p6p1 is my "Public" interface, which is a member of br_wan

- p12p1 is my "LAN" interface, which is a member of br_lan.

[root at zeus ~]# ip a
2: p6p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master
*br_wan* state UP group default qlen 1000
    link/ether 00:1f:c6:36:98:cc brd ff:ff:ff:ff:ff:ff
    inet6 fe80::21f:c6ff:fe36:98cc/64 scope link
       valid_lft forever preferred_lft forever
3: p4p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state
UP group default qlen 1000
    link/ether 00:1b:21:05:27:7f brd ff:ff:ff:ff:ff:ff
*    inet 10.10.10.21/16 <http://10.10.10.21/16> brd 10.10.255.255 scope
global p4p1*
       valid_lft forever preferred_lft forever
    inet6 fe80::21b:21ff:fe05:277f/64 scope link
       valid_lft forever preferred_lft forever
4: p12p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast
master *br_lan* state UP group default qlen 1000
    link/ether 00:0e:0c:cf:80:4d brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20e:cff:fecf:804d/64 scope link
       valid_lft forever preferred_lft forever

When you create a VM, creates a virtual network interface on the host,
which are joined to the appropriate bridge interfaces (All VMs are on
br_lan, but my routing machine is also on br_wan).

libvirt itself creates the vibr* interfaces, for host-only and NAT routing.
I don't use them, but haven't had a need to clean them up.

[root at zeus ~]# brctl show
bridge name    bridge id        STP enabled    interfaces
br_lan        8000.000e0ccf804d    no        p12p1
                            vnet0
                            vnet2
                            vnet3
                            vnet4
                            vnet5
br_wan        8000.001fc63698cc    no        p6p1
                            vnet1
virbr0        8000.525400e39548    yes        virbr0-nic
virbr1        8000.525400e7c266    yes        virbr1-nic

I can make a tar.gz of my network configs if wanted, but here's the
important bits from /etc/sysconfig/network-scripts/ (Note that this is all
on a F20 machine):

*ifcfg-p4p1* is a normal network config, for the normal host network.

TYPE="Ethernet"
BOOTPROTO="none"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6_FAILURE_FATAL="no"
NAME="p4p1"
UUID="3eb7febd-bd65-4758-a6ef-9e44d3426c2d"
ONBOOT="yes"
HWADDR="00:1B:21:05:27:7F"
IPADDR0=10.10.10.21
PREFIX0=16
GATEWAY=10.10.10.1
DNS1=10.10.10.1
DOMAIN="chrisirwin.ca"
NM_CONTROLLED=no


*ifcfg-br_lan* and *ifcfg-br_wan* are identical, except for the DEVICE line.

DEVICE=br_lan
TYPE=Bridge
BOOTPROTO=none
NM_CONTROLLED=no
IPV6INIT="no"
IPV6_AUTOCONF="no"
IPV6_DEFROUTE="no"

*ifcfg-p12p1* and *ifcfg-p6p1* are identical, except for HWADDR and BRIDGE
lines.

TYPE="Ethernet"
ONBOOT="yes"
NM_CONTROLLED=no
BRIDGE=br_lan
HWADDR="00:0e:0c:cf:80:4d"
IPV6INIT="no"




>
> So you must have a route on the host to direct traffic to the vm from it
> via this interface. [I get that all traffic outside this host would have vm
> traffic arrive via this interface. The arp'ing should keep the lines
> straight, as you don't have two ways externally to get to the host itself.]
>

I didn't make any special routing rules, just whatever the system sets up
from the above configs. The VMs get connectivity via the bridge -- think of
it like a software network switch. (Ignore the annoying link-local lines,
and the vibr0 lines, which are not used).

[root at zeus network-scripts]# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use
Iface

*default         ipfire.chrisirw 0.0.0.0         UG    0      0        0
p4p110.10.0.0       0.0.0.0         255.255.0.0     U     0      0        0
p4p1*
link-local      0.0.0.0         255.255.0.0     U     1003   0        0 p4p1
link-local      0.0.0.0         255.255.0.0     U     1005   0        0
br_wan
link-local      0.0.0.0         255.255.0.0     U     1006   0        0
br_lan
192.168.100.0   0.0.0.0         255.255.255.0   U     0      0        0
virbr1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0
virbr0


> (Do you have the vm's on a separate subnet?)
>

Nope, it's just like they are physically connected to my network. They are
on the same network, can see broadcasts, can get DHCP, etc.



> I'll bet Cranky would love you even more if you posted config examples or
> specifically pertinent links. (-:


Apparently Network Manager is capable of handling bridges now, so this
might be easier to configure. I haven't revisited in a few versions.


-- 
Chris Irwin
<chris at chrisirwin.ca>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://kwlug.org/pipermail/kwlug-disc_kwlug.org/attachments/20140815/c9398a52/attachment.htm>