[kwlug-disc] Multiple NICs on Ubuntu Server 14.04
Chris Irwin
chris at chrisirwin.ca
Fri Aug 15 15:42:34 EDT 2014
On Fri, Aug 15, 2014 at 2:54 PM, unsolicited <unsolicited at swiz.ca> wrote:
> vm solutions create software virtual layer 2 switches as a software link
> for vm networking, and as the hardware interface from the whole.
>
> My guess is your 2nd nic is not being picked up, or you would see an eth1
> (unless you have 'ifconfig eth1 down'ed it sort of thing.)
>
> You do not want two hardware interfaces on the same subnet unless you are
> bonding them into a single link, which requires hardware on the other end
> of the cable that understands such bonding. Which is highly unlikely that
> you have, or have set up. Such is not typically found in residential
> equipment.
>
> Elsewise, your network will get confused. Two paths to the same
> destination. Packets going out one nic but coming back the other will get
> rejected, and vice versa. Essentially your network will go schizophrenic.
>
> That is not to say a fair portion will continue to work, likely in a round
> robin, not load balanced, fashion, but you will find strange and bizarre
> things happen regularly. Like unsuccessful or dropped connections.
>
Only certain bonding modes require special equipment. I've used bonding at
home in the past (though don't bother anymore), and we've used it
extensively at work for years without any special considerations (but don't
bother anymore since we've moved to a VM model over the last year).
See the mode section of the RHEL bonding docs for a description of the
bonding modes.
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Deployment_Guide/sec-Using_Channel_Bonding.html
We've actually found that it works better with dumber equipment. The only
place we had issues with bonding was with some highly configurable
layer-3-capable routing switches which were confused due to their ability
to see multiple MAC addresses claiming the same IP. All hosts on the
network didn't notice, and regular layer-2 switches were blissfully
ignorant.
That said, I agree with the rest of your message. For home use, trying to
bond multiple NICs together is a bit overkill for home use, and why I don't
bother anymore.
I do still have three NICs in my VM server:
- The first is configured with an IP address and has all host traffic
(including the NFS backing storage) run through this interface.
- The second NIC is an 'UP', but an unconfigured member of a bridge (brvm),
which is also 'UP' but unconfigured. My virtual machines all connect to the
brvm bridge for direct network access.
- My third NIC is configured similarly to the second, but is considered
public (connected to my modem) and is connected only to my router VM :)
--
Chris Irwin
<chris at chrisirwin.ca>
On Fri, Aug 15, 2014 at 2:54 PM, unsolicited <unsolicited at swiz.ca> wrote:
> vm solutions create software virtual layer 2 switches as a software link
> for vm networking, and as the hardware interface from the whole.
>
> My guess is your 2nd nic is not being picked up, or you would see an eth1
> (unless you have 'ifconfig eth1 down'ed it sort of thing.)
>
> You do not want two hardware interfaces on the same subnet unless you are
> bonding them into a single link, which requires hardware on the other end
> of the cable that understands such bonding. Which is highly unlikely that
> you have, or have set up. Such is not typically found in residential
> equipment.
>
> Elsewise, your network will get confused. Two paths to the same
> destination. Packets going out one nic but coming back the other will get
> rejected, and vice versa. Essentially your network will go schizophrenic.
>
> That is not to say a fair portion will continue to work, likely in a round
> robin, not load balanced, fashion, but you will find strange and bizarre
> things happen regularly. Like unsuccessful or dropped connections.
>
> So the first question is: What is it you are trying to do? What do you
> mean by tame?
>
> I do see http://www.linux-kvm.org/page/Networking
>
> Some of the best advice I've seen, and follow, is until you have
> everything up and happy, don't have a 2nd nic present. It just complicates
> your life getting to where you're trying to go. You don't need networking
> issues getting their fingers in any issues, or appearing to, as you sort
> through things.
>
> Internally, a 2nd nic is not going to buy you anything significant - your
> internal gigabit network will likely sufficient satisfy your speed
> requirements within the house, and out through your slower provider link,
> if that's your interest.
>
> That's the physical. Security, if you're offering access to vm's from the
> outside world is a different thing. Be it a standalone LAMP server, or
> virtual variations thereof.
>
> (And you're likely better off for the moment making sure it all works
> happily on a single nic, before introducing a second.)
>
> {It's going to get even messier if you have a mix of vm's, some for
> external consumption, and some for internal consumption. All doable, but
> lots of little pieces to grok in between. e.g. Accept ssh/vnc/rdp for
> maintenance from a foreign, but internal, network address, but ignore for
> the other real world - when to the vm it's all foreign.}
>
> In the end, some combination of the following, and it is not a complete or
> exhaustive list, will be involved:
>
> - this internal kvm switch handing out dhcp to your vm's, on a SEPARATE
> subnet from your internal network.
>
> - changing your router so that designated incoming ports or network
> requests are directed to this 2nd STATICALLY assigned second nic.
>
> - ensuring that the virtual switch is SOLELY associated with this 2nd nic,
> and no inter-nic inter-networking bridging or cross-pollination occurs. You
> are likely ok as it stands as bridging between nics only happens when you
> deliberately set it up, not by itself. (But see schizo, above.)
>
> - you will want to review your iptables or perhaps even load a gui /
> firewall package, so that what you want and expect is what you're getting.
> And make sure that any changes load at system startup (something I still
> haven't mastered, myself).
>
> Which is all only to say, a number of disparate pieces enter the flow,
> each with their own learning curve. Easier to get things as you want then
> add this complexity as a standalone separate topic, rather than all
> complexities all at once.
>
> My bet is that you will find any performance gain (vs security assurance)
> is not worth the additional complexity and maintenance oversight.
>
> But, the first question to answer is: What is it you are looking to
> accomplish? Draw a diagram, and forget whether they're vm's or not. Whether
> you are configuring a virtual or a physical switch, it's all just an
> application interface, to you. It's just easier to get your head straight
> on what you want to do if you ignore whether they're physical or virtual,
> for the moment.
>
>
>
>
> On 14-08-15 11:30 AM, Khalid Baheyeldin wrote:
>
>> On Fri, Aug 15, 2014 at 10:40 AM, CrankyOldBugger
>> <crankyoldbugger at gmail.com <mailto:crankyoldbugger at gmail.com>> wrote:
>>
>>
>> virbr0 Link encap:Ethernet HWaddr ca:e3:65:49:dc:e4
>> inet addr:192.168.122.1 Bcast:192.168.122.255
>> Mask:255.255.255.0
>> UP BROADCAST MULTICAST MTU:1500 Metric:1
>> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:0
>> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>>
>>
>> Don't ask me why virbr0 is there. (Google tells me it's there
>> because I started tinkering with KVM at some point but I never
>> finished the task.)
>>
>>
>> Yes, I have that too, and yes, it is caused by libvirt/kvm. And it is
>> not from configuration in /etc/network/interfaces.
>>
>> I want to start building virtual machines on this box, but before I
>> do I would like to see the server using both NIC cards appropriately.
>>
>>
>> You can make both cards work by putting something like this in your
>> /etc/network/interfaces
>>
>>
>> auto eth0
>> iface eth0 inet dhcp
>>
>> auto eth1
>> iface eth1 inet dhcp
>>
>> The above will make both interfaces come up at boot, and configure
>> themselves via DHCP, so the router assigns them an IP address, ...etc
>>
>> Or you can assign that manually using something like:
>>
>> auto eth0
>> iface eth0 inet static
>> address 192.168.0.2
>> netmask 255.255.255.0
>> network 192.168.0.0
>> broadcast 192.168.0.255
>> gateway 192.168.0.1
>>
>> auto eth1
>> iface eth1 inet static
>> address 192.168.0.3
>> netmask 255.255.255.0
>> network 192.168.0.0
>> broadcast 192.168.0.255
>> gateway 192.168.0.1
>>
>> This means the first NIC will get 192.168.0.2 and the second will be .3,
>> assuming that the router is at .1.
>>
>> By that I mean it will share the load between any future VMs between
>> the two NICs,
>>
>>
>> I am not sure what share the load means, or if there is a real answer to
>> it as asked. So I will leave that to others who know more than me.
>>
>> or at least reserve one NIC for the host server and one NIC for the
>> VMs.
>>
>>
>> This can be done in /etc/network/interfaces as well. Assuming you want
>> the VMs to use only the 2nd network card, then you add:
>>
>> auto br0
>> iface br0 inet dhcp
>> bridge_ports eth1
>>
>> And when creating a VM, via libvirt, you say:
>>
>> sudo virt-install --name vm_name ... --network bridge=br0 ...
>>
>> This tells libvirt to use the bridge br0, which uses eth1, which you
>> have configured separately.
>>
>> I have not tried this specifically, but it "should work" (yeah, famous
>> words!)
>>
>> (I'm ashamed to say this is much easier in MS Hyper-V, but I would
>> much rather learn how to do this in Linux CLI!)
>>
>>
>> Tsk tsk tsk ... traitors and infiltrators in our midst ... ;-)
>>
>> So I guess my question is two-fold:
>>
>> a) how do I tame my NICs in preparation for setting up VMs, and
>>
>>
>> See above.
>>
>> b) what's the preferred method for setting up VMs in Ubuntu
>> Server? Does anyone know of a good how-to link?
>>
>>
>> If you have kvm capable server, then use libvirt. You create VMs using
>> virt-install on the command line, and use virt-viewer to see the BIOS
>> and console output, and manage them using virsh.
>>
>> Or wait till November, when I will cover all this in a presentation for
>> KWLUG.
>> --
>> Khalid M. Baheyeldin
>> 2bits.com <http://2bits.com>, Inc.
>>
>> Fast Reliable Drupal
>> Drupal optimization, development, customization and consulting.
>> Simplicity is prerequisite for reliability. -- Edsger W.Dijkstra
>> Simplicity is the ultimate sophistication. -- Leonardo da Vinci
>> For every complex problem, there is an answer that is clear, simple, and
>> wrong." -- H.L. Mencken
>>
>>
>> _______________________________________________
>> kwlug-disc mailing list
>> kwlug-disc at kwlug.org
>> http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
>>
>>
>
> _______________________________________________
> kwlug-disc mailing list
> kwlug-disc at kwlug.org
> http://kwlug.org/mailman/listinfo/kwlug-disc_kwlug.org
>
--
Chris Irwin
<chris at chrisirwin.ca>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://kwlug.org/pipermail/kwlug-disc_kwlug.org/attachments/20140815/15c9b96f/attachment.htm>
More information about the kwlug-disc
mailing list