In well-built networks with lots of capacity on all links between source and destination, even fully loaded links behave nicely and packet delay remains acceptable. In some scenarios, however, particularly when the bandwidth on the last leg is low, and packet buffers on some routers in the network are overdimensioned, packet delay becomes a real issue. When transferring larger files, bufferbloat quickly sets in, and applications such as interactive shell sessions, web browsing and voice calls become unusable. When I was recently faced with such a situation, I found an interesting traffic shaping tool that fixed the problem: Wondershaper.
The scenario I recently encountered was supporting a device behind what I would consider a slow Internet connection these days: 11 Mbit/s in the downlink direction and around 3.5 Mbit/s in the uplink direction. With good routers and packet buffer sizes, such a link is slow but still quite usable. Unfortunately, when saturating the uplink with a large cloud backup, ping response times would quickly rise beyond 10 seconds and make interactive ssh sessions impossible. Also, web browsing on the local device became pretty much impossible, and voice calls over the link became impossible. A typical case of buffer bloat.
My first reaction was to throttle the cloud backup itself. I use rsync for the purpose, which can be limited to a certain speed with the ‘–bwlimit’ option. After checking out the maximum transmission speed of the link with iperf in both directions, I set the bandwidth limit a bit lower. This stabilized the situation. But I was not entirely happy with the situation, so I kept looking for other ways to improve the situation.
This is how I came across Wondershaper, which is basically a shell script for Linux that can be run on an a client. The script makes heavy use of the ‘traffic control’ tc command to create traffic shaping queues and rules for the link. The basic idea behind the script is the same as limiting the bandwidth in rsync, but on an interface level: Limiting uplink and downlink throughput below what the access link to the Internet supports to avoid buffers in the network filling up. This is done by throwing away packets if the transmission rate becomes too high instead of buffering packets, a standard procedure to throttle TCP connections.
Note: If you have the luxury, you can also run Wondershaper on a Linux based router just behind the access gateway to shape the traffic of all devices in the network. This was not an option in my case but also not a problem, because there was just one device in the network that created a heavy load.
Wondershaper was first created in 2002, and is available in the repository of all major Linux distributions. Over time, the project was forked and enhanced significantly and the command line interface has changed. I therefore recommend to clone the latest version from Github. After cloning the directory, no installation is necessary, the script can be used directly from the cloned directory. So here’s my command to rate limit the wireless interface of the remote device to 10 Mbit/s in the downlink direction and 3 Mbit/s in the uplink direction:
sudo ./wondershaper -a wlp3s0 -u 3000 -d 10000
You might have noticed that these values are slightly lower than the actual line speed I talked about above. This is intentional to avoid the buffers filling up. In practice, this worked like a charm! I could fully load uplink and downlink, even simultaneously, and my interactive shell sessions would still work acceptably. Also, running voice calls over the same backhaul link from another device worked well.
There’s also an installation script to run it as a service on power-up, but it didn’t work on my notebook. That’s probably because the Wifi interface is only initialized after systemd has run the script. Also, the interface is reset every time the device was suspended, so it needs to be run every time the device returns from sleep. Something to keep in mind!
Also, I noticed that IPv6 traffic completely bypasses the ‘tc’ queues and filter rules. That’s because they only apply to IPv4. Wondershaper could probably be enhanced to apply the same rules for IPv6. However, this is not quite straight forward, because in addition to throttle the link, the script also configures rules, e.g. to prefer the transmission of ICMP messages and small TCP packets, to keep ssh and other text based sessions as interactive as possible. Preferring small packets is also important to return TCP ACK packets as fast as possible, which is important to keep other services as responsive as possible. Furthermore, Wondeshaper has the option to prefer or discriminate traffic to certain hosts or subnets. All of this is done by looking at the corresponding bytes in the IP header. In IPv6, IP addresses are of course different and the Type of Service and length indications are in other places. As I needed a quick solution, I decided to skip this for now and just temporarily turn-off IPv6 on the device:
sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1 sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1 sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
Obviously, this is not pretty. However, like the traffic shaping queues and rules, this is just temporary and a reboot or even only a suspend/resume cycle of the device removes these changes again. So no permanent harm done.
And one more important option: To remove the traffic shaping again, or to change the bandwidth settings, the following command is required to clear the configuration:
sudo ./wondershaper -c -a wlp3s0
In practice, I’ve been using this for a couple of days now and am quite happy with the performance. A nifty tool for special circumstances!