[kwlug-disc] Bandwidth aggregation
Cedric Puddy
cedric at thinkers.org
Thu Feb 12 00:32:19 EST 2009
On 11-Feb-09, at 11:43 AM, L.D. Paniak wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Thanks for putting the time into a detailed explanation. I understand
> now how my simple-minded picture of the issue was lacking.
>
> Do you know how 'multi-link PPP' which appears to be available from
> the
> ISP TekSavvvy compares to a "multihomed IP space"? Are these
> operationally the same thing?
No, they aren't. "multi-link PPP" is a channel bonding feature. It's
for scenarios were you have two parallel channels that you want to
load balance a single connection over.
The classic example is ISDN-U interface, which is the basic ISDN phone
line that they (planned) on delivering to people homes and such. It
had 2 data channels ("B" channels, each 64 kbit/sec) and 1 low-speed
signalling channel (stuff like "your phone is ringing, call from
#1234567890").
You could have both channels connected to your service provider at a
time (via multi-link PPP), and you could even have it drop one of the
channels if you had incoming phone call, etc... (thanks to the
dedicated Out Of Band signalling channel).
When Andrew talks about "multihomed IP space", he is talking about
something known as "Autonomous Systems", or "AS". In BGP4 (the
routing protocol that keeps the internet backbone working), you have
this thing called an "Autonomous System".
(I'm no BGP expert, and may get some of this slightly wrong. Andrew
can correct me if I stray too far! :)
The Core Concept is this:
- Each independent network is identified by a AS number. (this would
be any ISP, backbone provider, whatever that wanted to have multi-
homed address space)
- the ASN is a 16 bit number that uniquely identifies every
independent network in the entire internet (just recently, eg: 2007,
apparently extensions to BGP have happened that extend this to 32 bits
-- see: http://en.wikipedia.org/wiki/Autonomous_system_(Internet) )
- BGP4 is essentially all about routing large blocks of IP addresses
between different Autonomous Networks.
From that, you can draw some conclusions about the typical set up
that a multi-homed ISP has do go through:
1) Since the ASN was until recently a 16 bit number, that means there
was a limit of 65,535 networks. Now that they've made it a 32 bit
number, there would approximately as many possible ASN numbers as IP
address.
2) this address space expansion not withstanding, this has typically
been a scarce resource, and one only handed out to organizations that
pass through the appropriate hazing ritual.
3) even if you do manage to get assigned an ASN for yourself, the
only way you could use it is by getting you hands on some routable
address space. For an individual, this would require you to be able
to justify having a /20 block (8 class-C blocks, or 2048 IP address
assigned to just you!)
4) then you'd need a router capable of participating in Big-Internet
BGP routing. You can get Cisco 7200 series routers used on eBay these
days, of course....
5) you'd need at least two big-internet type internet pipes installed
-- probably a minimum cost of $1000/mo each.
As you can see, Andrews tunnelling to a multi-homed address makes a
lot more sense than trying to build all that for ones self, in a great
many situations.
-Cedric
> Andrew Kohlsmith (lists) wrote:
>> On February 10, 2009 03:52:44 pm L.D. Paniak wrote:
>>> I think it would be a great topic. WAN connections are easily the
>>> most
>>> unreliable part of a modern computer system. For things like VOIP,
>>> failover capability is essentially a requirement.
>>>
>>> Is it really that esoteric? Don't most homes/businesses have
>>> access to
>>> cable and DSL or two channels of DSL?
>>
>> Yes it is. I said "multihomed IP space" -- that is not the same
>> thing as two
>> DSL or cable connections at all.
>>
>> Multihomed IP space means an IP address which is reachable through
>> two or more
>> providers. I'm not aware of any DSL, Cable, Cell or Wifi providers
>> which give
>> you this kind of address.
>>
>> If you get 55.1.2.3 from your DSL provider and 99.1.2.3 through
>> your cable
>> provider, both of those addresses are single-homed from your
>> perspective. If
>> someone on the internet sends a packet to 55.1.2.3, it will ONLY
>> come from
>> your DSL line, and never from your cable. If you send a request
>> out your DSL
>> connection, the response will come back through your DSL
>> connection. It
>> won't balance between the DSL and cable connections. You can try
>> to fake
>> your source address to have a request go out one and back the
>> other, but any
>> ISP worth their salt will be dropping packets not sourced from
>> their own IP
>> space.
>>
>> What I am doing is "extruding" multihomed IP space through a number
>> of
>> single-homed connections. Through SCS Internet, I have access to
>> multihomed
>> IP space. (e.g. 77.1.2.0/24.) Traffic to any of these addresses
>> can come
>> from one of (I think) three or four different providers into his
>> rack.
>>
>> Using a server in that rack (with IP 77.1.2.2, for example), I
>> "extrude" an
>> IP, say 77.1.2.3. I use proxy-arp to as a means to capture traffic
>> for that
>> IP. Basically that means that whenever someone asks for the
>> ethernet address
>> of 77.1.2.3, the server responds with its own MAC.
>>
>> Your router at the office has a DSL and cable connection, with IP
>> 55.1.2.3 and
>> 99.1.2.3 from your respective providers. What I do now is to set
>> up two GRE
>> tunnels, one from 55.1.2.3 to 77.1.2.2 and one from 99.1.2.3 to
>> 77.1.2.2. So
>> now your home router actually has four routes: DSL-to-world, cable-
>> to-world,
>> DSL-to-server-via-tunnel0, cable-to-server-via-tunnel1.
>>
>> What I do now is to use iproute2 and iptables to set up a packet
>> marking and
>> load balancing system. I say that traffic for the world (i.e.
>> default route)
>> is via DSL-to-server-via-tunnel1 *AND* to cable-to-server-via-
>> tunnel1, with
>> equal weighting (neither route is preferred). I also to disable route
>> caching, which works against us in this scenario. Now any traffic
>> that the
>> router wants to send out to the world will go through either tunnel
>> and "pop
>> out" as traffic from 77.1.2.3.
>>
>> It may have taken the path through the cable or DSL IP, but it was
>> encapsulated in the tunnel at that point. The traffic exits from
>> 151 Front
>> with a source IP of 77.1.2.3 and heads out through any of the
>> providers that
>> he has available. It reaches the destination, and a response comes
>> back to
>> 77.1.2.3. The server receives that traffic, sends it back through
>> one of the
>> two tunnels, and your router gets it and does what it would
>> normally do with
>> it. All the tunnels are brought up and torn down with your
>> standard ip-up
>> style scripts that PPPoE gives, but you could also do it with the
>> normal
>> networking post-route scripts (for cable or wifi connections, for
>> example).
>>
>> If a tunnel fails, the other tunnel's there, and traffic still
>> flows. If you
>> add more tunnels, the bandwidth is load-balanced over them. Since
>> the
>> 77.1.2.3 address is multihomed, you don't lose connectivity when
>> one of the
>> providers that SCS Internet uses has troubles, since that IP is
>> reachable
>> through several other paths. You can play with route metrics to
>> favour one
>> tunnel over the other, and of course you can play with the routing
>> table
>> selections on the router so that you do NOT use the multihomed
>> space for
>> regular web traffic, windows updates or whatever other nonessential
>> traffic
>> you may have.
>>
>> You can *also* use IPSec to encrypt everything over the tunnel.
>>
>> The only real downsides are
>> 1) bandwidth considerations at the colo
>> 2) slightly increased latency due to tunnelling and encryption
>>
>> I've used this method to provide highly-available bandwidth for the
>> auto
>> industry, where they were willing to pay for the colocated server
>> at 151
>> Front -- multihomed IP space was simply not available at some of
>> their
>> plants, and simply too expensive at others. They decided to get
>> more bang for
>> their buck by using the colocated server as an IPSec endpoint, FTP
>> server for
>> *huge* CAD drawings and a conferencing bridge instead of bringing
>> absolutely
>> all that traffic back to their home network.
>>
>> Anyway -- that's why I feel it's an esoteric topic. You can bring
>> two DSL or
>> cable connections together for home/office use, but that's not the
>> same thing
>> as what I described.
>>
>> -A.
>>
>> _______________________________________________
>> kwlug-disc_kwlug.org mailing list
>> kwlug-disc_kwlug.org at kwlug.org
>> http://astoria.ccjclearline.com/mailman/listinfo/kwlug-disc_kwlug.org
>>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD8DBQFJkwA38h2PnOHbiQcRAmc9AJ9/EVBv+v1jdo/cBDPjY/7Gna2aIwCcDA5F
> sF0TLRJynor/J4SDWVyPf2Y=
> =dGT2
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> kwlug-disc_kwlug.org mailing list
> kwlug-disc_kwlug.org at kwlug.org
> http://astoria.ccjclearline.com/mailman/listinfo/kwlug-disc_kwlug.org
| CCj/ClearLine - Unix/NT Administration and TCP/IP Network Services
| 118 Louisa Street, Kitchener, Ontario, N2H 5M3, 519-489-0478
\________________________________________________________
Cedric Puddy, IS Director cedric at thinkers.org
PGP Key Available at: http://www.thinkers.org/cedric
More information about the kwlug-disc
mailing list