[kwlug-disc] tc and traffic shaping

Andrew Kohlsmith (mailing lists account) aklists at mixdown.ca
Wed Sep 8 17:28:09 EDT 2010


Given that at least two people have openly commented that they're experienced 
traffic shapers and one in particular (Frernando) has even admitted to using tc, 
I thought I'd share my traffic shaping script and ask some questions.

I'm on DSL and have an internal modem. Linux has direct control over the 
outgoing message queue and thus I don't have to worry as much about saturating 
an 800kbit upstream with a 100mbit ethernet port sourcing it.

I also run VOIP, and do not have a "real" telephone line. Haven't for years. 
In order to keep the Wife Acceptance Factor (WAF) high, I've implemented traffic 
shaping and policing on the ADSL line.

I've tried wondershaper and the various other ones out there (in fact mine is 
based largely on wondershaper and the lartc manual online) but they were all 
lacking in some way.

What I've pasted below is what I'm using. It's almost perfect, and I'm hoping 
that those with a little more tc understanding than me can help me perfect it.

What the script does:
5 classes for outgoing traffic, in order of priority:
- VOIP
- ICMP ECHO, TCP ACK, interactive traffic
- default (bulk) traffic
- mail
- lowest priority traffic

I set up the system so that VOIP traffic is sent via FIFO, and the streams in 
the other classes are handled in a fair queuing manner. I guarantee bandwidth 
for individual classes, and the total guaranteed (576kbit) isn't close to my 
maximum (800kbit).

From an downstream (data to me) perspective, I tried playing with priority 
maps and policing some of the bulk traffic in an effort to have the application 
layer throttle back the UDP traffic.

I set up upstream traffic shaping/policing on the DSL network interface, and the 
downstream traffic shaping/policing on the LAN network interface; it doesn't 
make any sense to try to do things with the data coming IN to the firewall in 
either direction.

So, given all of this, and given the script below... would anyone care to 
comment, give some tips or guidance?  What do you use to monitor traffic shaping 
performance?

-A.


# cat /etc/rc.d/rc.tc
#!/bin/bash

DSLDEV=ppp0
LANDEV=eth0
UPRATE=700
DOWNRATE=4400


tc_upstatus() {
  tc -s qdisc ls dev $DSLDEV
  echo
  tc -s class ls dev $DSLDEV
}


tc_downstatus() {
  tc -s qdisc ls dev $LANDEV
  echo
  tc -s class ls dev $LANDEV
}


tc_stop() {
# clean existing down- and uplink qdiscs, hide errors
  tc qdisc del dev $DSLDEV root    2> /dev/null > /dev/null
  tc qdisc del dev $DSLDEV ingress 2> /dev/null > /dev/null

  tc qdisc del dev $LANDEV root    2> /dev/null > /dev/null
  tc qdisc del dev $LANDEV ingress 2> /dev/null > /dev/null

#  iptables -D FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-
pmtu 2> /dev/null
#  iptables -t mangle -D PREROUTING -m p2p -j CONNMARK --set-mark 1 2> 
/dev/null 2> /dev/null
#  iptables -t mangle -D PREROUTING -m connmark --mark 1 -j CONNMARK --
restore-mark 2> /dev/null > /dev/null
}


tc_start() {
# *** UPSTREAM (SENDING) CONFIG ***

  CEIL=$[90*$UPRATE/100]
  MISCRATE=$[90*$CEIL/100]

# set packet queue much smaller than default (100):
  ip link set dev $DSLDEV qlen 100

# install root HTB, point default traffic to 1:30:
  tc qdisc add dev $DSLDEV root handle 1: htb r2q 1 default 30
  tc class add dev $DSLDEV parent 1: classid 1:1 htb rate ${MISCRATE}kbit ceil 
${CEIL}kbit

# 1:10 - VOIP
# 1:20 - ICMP ECHO, TCP ACK, interactive traffic
# 1:30 - default (bulk) traffic
# 1:40 - mail
# 1:50 - lowest priority traffic

  tc class add dev $DSLDEV parent 1:1 classid 1:10 htb rate 256kbit ceil 
${MISCRATE}kbit prio 1
  tc class add dev $DSLDEV parent 1:1 classid 1:20 htb rate 64kbit ceil 
${MISCRATE}kbit prio 2
  tc class add dev $DSLDEV parent 1:1 classid 1:30 htb rate 128kbit ceil 
${MISCRATE}kbit prio 3
  tc class add dev $DSLDEV parent 1:1 classid 1:40 htb rate 64kbit ceil 
${MISCRATE}kbit prio 4
  tc class add dev $DSLDEV parent 1:1 classid 1:50 htb rate 64kbit ceil 
${MISCRATE}kbit prio 5

# VOIP gets FIFO, the rest get Stochastic Fairness:
  tc qdisc add dev $DSLDEV parent 1:10 handle 10: pfifo limit 50
  tc qdisc add dev $DSLDEV parent 1:20 handle 20: sfq perturb 10
  tc qdisc add dev $DSLDEV parent 1:30 handle 30: sfq perturb 10
  tc qdisc add dev $DSLDEV parent 1:40 handle 40: sfq perturb 10
  tc qdisc add dev $DSLDEV parent 1:50 handle 50: sfq perturb 10

# now set up filters to toss packets into the appropriate class:
#a) VOIP
  tc filter add dev $DSLDEV protocol ip prio 1 u32 match ip dst 208.68.91.47 
match ip protocol 17 0xff flowid 1:10
  tc filter add dev $DSLDEV protocol ip prio 2 u32 match ip dst 208.68.89.134 
match ip protocol 17 0xff flowid 1:10

#b) Interactive stuff, ICMP ECHO and TCP ACKs
  tc filter add dev $DSLDEV protocol ip prio 10 u32 match ip tos 0x10 0xff flowid 
1:20
  tc filter add dev $DSLDEV protocol ip prio 11 u32 match ip protocol 1 0xff 
flowid 1:20
  tc filter add dev $DSLDEV protocol ip prio 12 u32 match ip protocol 47 0xff 
flowid 1:20
  tc filter add dev $DSLDEV protocol ip prio 13 u32 match ip protocol 50 0xff 
flowid 1:20
  tc filter add dev $DSLDEV protocol ip prio 14 u32 match ip sport 53 0xffff 
flowid 1:20
  tc filter add dev $DSLDEV protocol ip prio 15 u32 match ip dport 53 0xffff 
flowid 1:20
  tc filter add dev $DSLDEV protocol ip prio 16 u32 \
     match ip protocol 6 0xff \
     match u8 0x05 0x0f at 0 \
     match u16 0x0000 0xffc0 at 2 \
     match u8 0x10 0xff at 33 \
     flowid 1:20

#c) mail:
  tc filter add dev $DSLDEV protocol ip prio 40 u32 match ip dport 25 0xffff 
flowid 1:40
  tc filter add dev $DSLDEV protocol ip prio 41 u32 match ip sport 25 0xffff 
flowid 1:40
  tc filter add dev $DSLDEV protocol ip prio 42 u32 match ip sport 110 0xffff 
flowid 1:40
  tc filter add dev $DSLDEV protocol ip prio 43 u32 match ip sport 143 0xffff 
flowid 1:40

  tc filter add dev $DSLDEV protocol ip prio 50 u32 match ip dst 208.68.91.47 
flowid 1:50

# any traffic that the p2p match module for iptables finds (it marks with --set-
mark 1):
#  tc filter add dev $DSLDEV protocol ip prio 59 handle 1 fw flowid 1:50

# LAN ingress handler; drop any NON-VOIP traffic > rate
#  tc qdisc add dev $DSLDEV handle ffff: ingress
#  tc filter add dev $DSLDEV parent ffff: protocol ip prio 90 u32 match ip src 
208.68.91.47 match ip protocol 17 0xff flowid :1
#  tc filter add dev $DSLDEV parent ffff: protocol ip prio 91 u32 match ip src 
208.68.89.134 match ip protocol 17 0xff flowid :1
#  tc filter add dev $DSLDEV parent ffff: protocol ip prio 99 u32 match ip src 
0.0.0.0/0 police rate $[90*$DOWNRATE/100]kbit buffer 8k drop flowid :1

# *** DOWNSTREAM (RECEIVING) CONFIG ***

  CEIL=$DOWNRATE
  MISCRATE=$[98*$CEIL/100]

# set packet queue much smaller than default (100):
  ip link set dev $LANDEV qlen 100

# default priomap -------------------------------------------> 1 2 1 1 2 2 2 2 
0 0 0 0 1 1 1 1
#  tc qdisc add dev $LANDEV root handle 1: prio bands 5 priomap 2 2 2 2 2 2 2 
2 1 1 1 1 2 2 2 2 

# install root HTB, point default traffic to 1:30:
  tc qdisc add dev $LANDEV root handle 1: htb r2q 5 default 30
  tc class add dev $LANDEV parent 1: classid 1:1 htb rate ${MISCRATE}kbit ceil 
${CEIL}kbit

# 1:10 - VOIP
# 1:20 - ICMP ECHO, TCP ACK, interactive traffic
# 1:30 - default (bulk) traffic
# 1:40 - mail
# 1:50 - lowest priority traffic

  tc class add dev $LANDEV parent 1:1 classid 1:10 htb rate 256kbit ceil 
${MISCRATE}kbit prio 1
  tc class add dev $LANDEV parent 1:1 classid 1:20 htb rate 64kbit ceil 
${MISCRATE}kbit prio 2
  tc class add dev $LANDEV parent 1:1 classid 1:30 htb rate 128kbit ceil 
${MISCRATE}kbit prio 3
  tc class add dev $LANDEV parent 1:1 classid 1:40 htb rate 64kbit ceil 
${MISCRATE}kbit prio 4
  tc class add dev $LANDEV parent 1:1 classid 1:50 htb rate 64kbit ceil 
${MISCRATE}kbit prio 5

# VOIP gets FIFO, the rest get Stochastic Fairness:
  tc qdisc add dev $LANDEV parent 1:10 handle 10: pfifo limit 50
  tc qdisc add dev $LANDEV parent 1:20 handle 20: sfq perturb 10
  tc qdisc add dev $LANDEV parent 1:30 handle 30: sfq perturb 10
  tc qdisc add dev $LANDEV parent 1:40 handle 40: sfq perturb 10
  tc qdisc add dev $LANDEV parent 1:50 handle 50: sfq perturb 10

# now set up filters to toss packets into the appropriate class:
#a) VOIP
  tc filter add dev $LANDEV protocol ip prio 1 u32 match ip src 208.68.91.47 
match ip protocol 17 0xff flowid 1:10
  tc filter add dev $LANDEV protocol ip prio 2 u32 match ip src 208.68.89.134 
match ip protocol 17 0xff flowid 1:10

#b) Interactive stuff, ICMP ECHO and TCP ACKs
  tc filter add dev $LANDEV protocol ip prio 10 u32 match ip tos 0x10 0xff flowid 
1:20
  tc filter add dev $LANDEV protocol ip prio 11 u32 match ip protocol 1 0xff 
flowid 1:20
  tc filter add dev $LANDEV protocol ip prio 12 u32 match ip protocol 47 0xff 
flowid 1:20
  tc filter add dev $LANDEV protocol ip prio 13 u32 match ip protocol 50 0xff 
flowid 1:20
  tc filter add dev $LANDEV protocol ip prio 14 u32 match ip sport 53 0xffff 
flowid 1:20
  tc filter add dev $LANDEV protocol ip prio 15 u32 match ip dport 53 0xffff 
flowid 1:20
  tc filter add dev $LANDEV protocol ip prio 16 u32 \
     match ip protocol 6 0xff \
     match u8 0x05 0x0f at 0 \
     match u16 0x0000 0xffc0 at 2 \
     match u8 0x10 0xff at 33 \
     flowid 1:20

#c) mail:
  tc filter add dev $LANDEV protocol ip prio 40 u32 match ip dport 25 0xffff 
flowid 1:40
  tc filter add dev $LANDEV protocol ip prio 41 u32 match ip sport 25 0xffff 
flowid 1:40
  tc filter add dev $LANDEV protocol ip prio 42 u32 match ip sport 110 0xffff 
flowid 1:40
  tc filter add dev $LANDEV protocol ip prio 43 u32 match ip sport 143 0xffff 
flowid 1:40

  tc filter add dev $LANDEV protocol ip prio 50 u32 match ip src 208.68.91.47 
flowid 1:50

# any traffic that the p2p match module for iptables finds (it marks with --set-
mark 1):
#  tc filter add dev $LANDEV protocol ip prio 59 handle 1 fw flowid 2:50

# p2p detection
#  iptables -t mangle -A PREROUTING -m p2p -j CONNMARK --set-mark 1
#  iptables -t mangle -A PREROUTING -m connmark --mark 1 -j CONNMARK --
restore-mark
}

case "$1" in
'start')
  tc_start
  ;;
'stop')
  tc_stop
  ;;
'restart')
  tc_stop
  sleep 1
  tc_start
  ;;
'upstatus')
  tc_upstatus
  ;;
'downstatus')
  tc_downstatus
  ;;
*)
  echo "usage $0 start|stop|restart"
esac




More information about the kwlug-disc mailing list