[kwlug-disc] tc and traffic shaping

Fernando Duran liberosec at yahoo.ca
Wed Sep 8 21:49:34 EDT 2010


(since my name was invoked he)

Hi Andrew,

It's been 10 years since I touched tc (and at that time the code was not 
being maintained for several years, the htb algorithm by a different developer 
was the only new thing) so I'm sure you know more about the scripts but I'll try 
to make a couple of not-totally-useless comments.


- The general policy (traffic order) is sound: VoIP first, interactive traffic 
and then the rest, mail etc.
- It never occurred to me to prioritize particular packets, like ACKs. If you 
found some reliable documentation or tests about this then it may make sense but 
my first instinct is that prioritizing just packets is not going to work or make 
much difference.
- Regarding UDP probably I'm not understanding your situation: most of your UDP 
traffic is going to be VoIP, why would you want to throttle that?
- You may want to optimize DNS traffic too (as part of "interactive")
- As you said it doesn't make any sense to prioritize incoming traffic
- Does it work? If you're not sure I suggest you use a bandwidth logging tool 
(or at worst logging iptables) to find out if the rules are doing what they 
should. There are traffic generators you can use and set up a few scenarios.
- An option instead of tc is to use the Tomato 
firmware http://www.polarcloud.com/tomato which has traffic shaping / QoS 
capabilities with nice GUI and graphics, I haven't tried it myself but I keep 
hearing good things about it.

Cheers,
---------------------
Fernando Duran
http://www.fduran.com



----- Original Message ----
> From: Andrew Kohlsmith (mailing lists account) <aklists at mixdown.ca>
> To: KWLUG discussion <kwlug-disc at kwlug.org>
> Sent: Wed, September 8, 2010 5:28:09 PM
> Subject: [kwlug-disc] tc and traffic shaping
> 
> Given that at least two people have openly commented that they're experienced 
> traffic shapers and one in particular (Frernando) has even admitted to using  
>tc, 
>
> I thought I'd share my traffic shaping script and ask some  questions.
> 
> I'm on DSL and have an internal modem. Linux has direct  control over the 
> outgoing message queue and thus I don't have to worry as  much about saturating 
>
> an 800kbit upstream with a 100mbit ethernet port  sourcing it.
> 
> I also run VOIP, and do not have a "real" telephone line.  Haven't for years. 
> In order to keep the Wife Acceptance Factor (WAF) high,  I've implemented 
>traffic 
>
> shaping and policing on the ADSL line.
> 
> I've  tried wondershaper and the various other ones out there (in fact mine is 

> based largely on wondershaper and the lartc manual online) but they were all 
> lacking in some way.
> 
> What I've pasted below is what I'm using. It's  almost perfect, and I'm hoping 

> that those with a little more tc  understanding than me can help me perfect 
it.
> 
> What the script does:
> 5  classes for outgoing traffic, in order of priority:
> - VOIP
> - ICMP ECHO,  TCP ACK, interactive traffic
> - default (bulk) traffic
> - mail
> - lowest  priority traffic
> 
> I set up the system so that VOIP traffic is sent via  FIFO, and the streams in 

> the other classes are handled in a fair queuing  manner. I guarantee bandwidth 

> for individual classes, and the total  guaranteed (576kbit) isn't close to my 
> maximum (800kbit).
> 
> >From an  downstream (data to me) perspective, I tried playing with priority 
> maps and  policing some of the bulk traffic in an effort to have the 
>application 
>
> layer  throttle back the UDP traffic.
> 
> I set up upstream traffic shaping/policing  on the DSL network interface, and 
>the 
>
> downstream traffic shaping/policing on  the LAN network interface; it doesn't 
> make any sense to try to do things  with the data coming IN to the firewall in 

> either direction.
> 
> So,  given all of this, and given the script below... would anyone care to 
> comment, give some tips or guidance?  What do you use to monitor  traffic 
>shaping 
>
> performance?
> 
> -A.
> 
> 
> # cat  /etc/rc.d/rc.tc
> #!/bin/bash
> 
> DSLDEV=ppp0
> LANDEV=eth0
> UPRATE=700
> DOWNRATE=4400
> 
> 
> tc_upstatus()  {
>   tc -s qdisc ls dev $DSLDEV
>   echo
>   tc -s class ls  dev $DSLDEV
> }
> 
> 
> tc_downstatus() {
>   tc -s qdisc ls dev  $LANDEV
>   echo
>   tc -s class ls dev  $LANDEV
> }
> 
> 
> tc_stop() {
> # clean existing down- and uplink qdiscs,  hide errors
>   tc qdisc del dev $DSLDEV root    2> /dev/null  > /dev/null
>   tc qdisc del dev $DSLDEV ingress 2> /dev/null >  /dev/null
> 
>   tc qdisc del dev $LANDEV root    2>  /dev/null > /dev/null
>   tc qdisc del dev $LANDEV ingress 2>  /dev/null > /dev/null
> 
> #  iptables -D FORWARD -p tcp --tcp-flags  SYN,RST SYN -j TCPMSS 
>--clamp-mss-to-
> pmtu 2> /dev/null
> #   iptables -t mangle -D PREROUTING -m p2p -j CONNMARK --set-mark 1 2> 
> /dev/null 2> /dev/null
> #  iptables -t mangle -D PREROUTING -m  connmark --mark 1 -j CONNMARK --
> restore-mark 2> /dev/null >  /dev/null
> }
> 
> 
> tc_start() {
> # *** UPSTREAM (SENDING) CONFIG  ***
> 
>   CEIL=$[90*$UPRATE/100]
>    MISCRATE=$[90*$CEIL/100]
> 
> # set packet queue much smaller than default  (100):
>   ip link set dev $DSLDEV qlen 100
> 
> # install root HTB,  point default traffic to 1:30:
>   tc qdisc add dev $DSLDEV root handle 1:  htb r2q 1 default 30
>   tc class add dev $DSLDEV parent 1: classid 1:1  htb rate ${MISCRATE}kbit ceil 
>
> ${CEIL}kbit
> 
> # 1:10 - VOIP
> # 1:20 -  ICMP ECHO, TCP ACK, interactive traffic
> # 1:30 - default (bulk) traffic
> #  1:40 - mail
> # 1:50 - lowest priority traffic
> 
>   tc class add dev  $DSLDEV parent 1:1 classid 1:10 htb rate 256kbit ceil 
> ${MISCRATE}kbit prio  1
>   tc class add dev $DSLDEV parent 1:1 classid 1:20 htb rate 64kbit  ceil 
> ${MISCRATE}kbit prio 2
>   tc class add dev $DSLDEV parent 1:1  classid 1:30 htb rate 128kbit ceil 
> ${MISCRATE}kbit prio 3
>   tc class  add dev $DSLDEV parent 1:1 classid 1:40 htb rate 64kbit ceil 
> ${MISCRATE}kbit  prio 4
>   tc class add dev $DSLDEV parent 1:1 classid 1:50 htb rate  64kbit ceil 
> ${MISCRATE}kbit prio 5
> 
> # VOIP gets FIFO, the rest get  Stochastic Fairness:
>   tc qdisc add dev $DSLDEV parent 1:10 handle 10:  pfifo limit 50
>   tc qdisc add dev $DSLDEV parent 1:20 handle 20: sfq  perturb 10
>   tc qdisc add dev $DSLDEV parent 1:30 handle 30: sfq perturb  10
>   tc qdisc add dev $DSLDEV parent 1:40 handle 40: sfq perturb  10
>   tc qdisc add dev $DSLDEV parent 1:50 handle 50: sfq perturb  10
> 
> # now set up filters to toss packets into the appropriate  class:
> #a) VOIP
>   tc filter add dev $DSLDEV protocol ip prio 1 u32  match ip dst 208.68.91.47 
> match ip protocol 17 0xff flowid 1:10
>   tc  filter add dev $DSLDEV protocol ip prio 2 u32 match ip dst 208.68.89.134 

> match ip protocol 17 0xff flowid 1:10
> 
> #b) Interactive stuff, ICMP  ECHO and TCP ACKs
>   tc filter add dev $DSLDEV protocol ip prio 10 u32  match ip tos 0x10 0xff 
>flowid 
>
> 1:20
>   tc filter add dev $DSLDEV  protocol ip prio 11 u32 match ip protocol 1 0xff 
> flowid 1:20
>   tc  filter add dev $DSLDEV protocol ip prio 12 u32 match ip protocol 47 0xff 

> flowid 1:20
>   tc filter add dev $DSLDEV protocol ip prio 13 u32  match ip protocol 50 0xff 

> flowid 1:20
>   tc filter add dev $DSLDEV  protocol ip prio 14 u32 match ip sport 53 0xffff 
> flowid 1:20
>   tc  filter add dev $DSLDEV protocol ip prio 15 u32 match ip dport 53 0xffff 
> flowid 1:20
>   tc filter add dev $DSLDEV protocol ip prio 16 u32  \
>      match ip protocol 6 0xff \
>      match u8 0x05  0x0f at 0 \
>      match u16 0x0000 0xffc0 at 2 \
>       match u8 0x10 0xff at 33 \
>      flowid 1:20
> 
> #c)  mail:
>   tc filter add dev $DSLDEV protocol ip prio 40 u32 match ip dport  25 0xffff 
> flowid 1:40
>   tc filter add dev $DSLDEV protocol ip prio  41 u32 match ip sport 25 0xffff 
> flowid 1:40
>   tc filter add dev  $DSLDEV protocol ip prio 42 u32 match ip sport 110 0xffff 

> flowid  1:40
>   tc filter add dev $DSLDEV protocol ip prio 43 u32 match ip sport  143 0xffff 

> flowid 1:40
> 
>   tc filter add dev $DSLDEV protocol ip  prio 50 u32 match ip dst 208.68.91.47 

> flowid 1:50
> 
> # any traffic that  the p2p match module for iptables finds (it marks with 
>--set-
> mark  1):
> #  tc filter add dev $DSLDEV protocol ip prio 59 handle 1 fw flowid  1:50
> 
> # LAN ingress handler; drop any NON-VOIP traffic >  rate
> #  tc qdisc add dev $DSLDEV handle ffff: ingress
> #  tc  filter add dev $DSLDEV parent ffff: protocol ip prio 90 u32 match ip src 
>
> 208.68.91.47 match ip protocol 17 0xff flowid :1
> #  tc filter add  dev $DSLDEV parent ffff: protocol ip prio 91 u32 match ip src 
>
> 208.68.89.134  match ip protocol 17 0xff flowid :1
> #  tc filter add dev $DSLDEV parent  ffff: protocol ip prio 99 u32 match ip src 
>
> 0.0.0.0/0 police rate  $[90*$DOWNRATE/100]kbit buffer 8k drop flowid :1
> 
> # *** DOWNSTREAM  (RECEIVING) CONFIG ***
> 
>   CEIL=$DOWNRATE
>    MISCRATE=$[98*$CEIL/100]
> 
> # set packet queue much smaller than default  (100):
>   ip link set dev $LANDEV qlen 100
> 
> # default priomap  -------------------------------------------> 1 2 1 1 2 2 2 2 
>
> 0 0 0 0 1 1  1 1
> #  tc qdisc add dev $LANDEV root handle 1: prio bands 5 priomap 2 2  2 2 2 2 2 

> 2 1 1 1 1 2 2 2 2 
> 
> # install root HTB, point default  traffic to 1:30:
>   tc qdisc add dev $LANDEV root handle 1: htb r2q 5  default 30
>   tc class add dev $LANDEV parent 1: classid 1:1 htb rate  ${MISCRATE}kbit ceil 
>
> ${CEIL}kbit
> 
> # 1:10 - VOIP
> # 1:20 - ICMP ECHO,  TCP ACK, interactive traffic
> # 1:30 - default (bulk) traffic
> # 1:40 -  mail
> # 1:50 - lowest priority traffic
> 
>   tc class add dev $LANDEV  parent 1:1 classid 1:10 htb rate 256kbit ceil 
> ${MISCRATE}kbit prio  1
>   tc class add dev $LANDEV parent 1:1 classid 1:20 htb rate 64kbit  ceil 
> ${MISCRATE}kbit prio 2
>   tc class add dev $LANDEV parent 1:1  classid 1:30 htb rate 128kbit ceil 
> ${MISCRATE}kbit prio 3
>   tc class  add dev $LANDEV parent 1:1 classid 1:40 htb rate 64kbit ceil 
> ${MISCRATE}kbit  prio 4
>   tc class add dev $LANDEV parent 1:1 classid 1:50 htb rate  64kbit ceil 
> ${MISCRATE}kbit prio 5
> 
> # VOIP gets FIFO, the rest get  Stochastic Fairness:
>   tc qdisc add dev $LANDEV parent 1:10 handle 10:  pfifo limit 50
>   tc qdisc add dev $LANDEV parent 1:20 handle 20: sfq  perturb 10
>   tc qdisc add dev $LANDEV parent 1:30 handle 30: sfq perturb  10
>   tc qdisc add dev $LANDEV parent 1:40 handle 40: sfq perturb  10
>   tc qdisc add dev $LANDEV parent 1:50 handle 50: sfq perturb  10
> 
> # now set up filters to toss packets into the appropriate  class:
> #a) VOIP
>   tc filter add dev $LANDEV protocol ip prio 1 u32  match ip src 208.68.91.47 
> match ip protocol 17 0xff flowid 1:10
>   tc  filter add dev $LANDEV protocol ip prio 2 u32 match ip src 208.68.89.134 

> match ip protocol 17 0xff flowid 1:10
> 
> #b) Interactive stuff, ICMP  ECHO and TCP ACKs
>   tc filter add dev $LANDEV protocol ip prio 10 u32  match ip tos 0x10 0xff 
>flowid 
>
> 1:20
>   tc filter add dev $LANDEV  protocol ip prio 11 u32 match ip protocol 1 0xff 
> flowid 1:20
>   tc  filter add dev $LANDEV protocol ip prio 12 u32 match ip protocol 47 0xff 

> flowid 1:20
>   tc filter add dev $LANDEV protocol ip prio 13 u32  match ip protocol 50 0xff 

> flowid 1:20
>   tc filter add dev $LANDEV  protocol ip prio 14 u32 match ip sport 53 0xffff 
> flowid 1:20
>   tc  filter add dev $LANDEV protocol ip prio 15 u32 match ip dport 53 0xffff 
> flowid 1:20
>   tc filter add dev $LANDEV protocol ip prio 16 u32  \
>      match ip protocol 6 0xff \
>      match u8 0x05  0x0f at 0 \
>      match u16 0x0000 0xffc0 at 2 \
>       match u8 0x10 0xff at 33 \
>      flowid 1:20
> 
> #c)  mail:
>   tc filter add dev $LANDEV protocol ip prio 40 u32 match ip dport  25 0xffff 
> flowid 1:40
>   tc filter add dev $LANDEV protocol ip prio  41 u32 match ip sport 25 0xffff 
> flowid 1:40
>   tc filter add dev  $LANDEV protocol ip prio 42 u32 match ip sport 110 0xffff 

> flowid  1:40
>   tc filter add dev $LANDEV protocol ip prio 43 u32 match ip sport  143 0xffff 

> flowid 1:40
> 
>   tc filter add dev $LANDEV protocol ip  prio 50 u32 match ip src 208.68.91.47 

> flowid 1:50
> 
> # any traffic that  the p2p match module for iptables finds (it marks with 
>--set-
> mark  1):
> #  tc filter add dev $LANDEV protocol ip prio 59 handle 1 fw flowid  2:50
> 
> # p2p detection
> #  iptables -t mangle -A PREROUTING -m p2p  -j CONNMARK --set-mark 1
> #  iptables -t mangle -A PREROUTING -m connmark  --mark 1 -j CONNMARK --
> restore-mark
> }
> 
> case "$1"  in
> 'start')
>   tc_start
>   ;;
> 'stop')
>    tc_stop
>   ;;
> 'restart')
>   tc_stop
>   sleep 1
>    tc_start
>   ;;
> 'upstatus')
>   tc_upstatus
>    ;;
> 'downstatus')
>   tc_downstatus
>   ;;
> *)
>   echo  "usage $0  start|stop|restart"
> esac
> 
> _______________________________________________
> kwlug-disc_kwlug.org  mailing list
> kwlug-disc_kwlug.org at kwlug.org
> http://astoria.ccjclearline.com/mailman/listinfo/kwlug-disc_kwlug.org
> 






More information about the kwlug-disc_kwlug.org mailing list