Martin – Page 28 – WirelessMoves

TCP Tracing – Part 6 – Segmentation Offloading

With multi gigabit per second Ethernet and wireless interfaces, CPUs are quite challenged by the sheer number of packets that need to be handled. Let’s say a transmission on a (meager) 1 Gbps Ethernet link is well utilized by large file transfers (let’s say 100 MBbytes per second) and a typical maximum segment size (MSS) of 1460 bytes is used. That’s 68.493 packets per second in one direction, not even counting the TCP ACKs in the other direction! Also, Wireshark starts smoking when it has to look at a 60 seconds trace with 4 million packets inside. But there’s a fix for that with almost no downsides that: Segmentation Offloading.

Kubernetes Intro – Part 12 – Jaeger

It’s been a while since part 11 of my Kubernetes Intro, but new things keep coming. When you have a nice little Kubernetes cluster with dozens of interacting microservices running, you quickly come to the point where you need to debug the system. Today, and particularly in the telecoms networking world, tracing on network interfaces and analyzing the result with Wireshark is one of the most important debugging tools. But that’s a bit of a challenge when your signaling path flows through a Kubernetes cluster.

TCP Tracing – Part 5 – High Speed Wi-Fi and the Receive Window

In part 4 of this series, I’ve compared the TCP behavior of high speed LTE and Wi-Fi to a virtual machine in a dater center over links with round trip delay times of around 25 ms. The results were very insightful but also raised new questions that I will follow up in this post. So here’s a quick recap of what happened so far to set things into context:

For fast Wi-Fi, I went to the local university in Cologne that has an excellent Eduroam Wi-Fi deployed in many areas. There, I was able to get an iperf uplink throughput of around 500 Mbps to a virtual machine in Nürnberg. When analyzing the TCP behavior in more detail, I noticed that my notebook quite frequently hit the default maximum TCP receive window size of 3 MB of the VM in the remote data center. In other words: More would perhaps have been possible with a bigger TCP receive buffer. So I gave this another try to see what would happen.

Get Notified of SSH Logins

Here’s a quick tip of the day that I came across when I needed a way to get a notification on my mobile device when particular users log into a server. Turns out that the Linux PAM (Pluggable Authentication Modules) offers a convenient way to do this:

TCP Tracing – Part 4 – Comparing LTE and Wi-Fi

In the first part of episode 3 on this topic, I have shown how throughput looks like in a good LTE network in which more than one user was activate at a time. I can’t be sure, but based on typical cellular scenarios, there were probably more than 30 concurrent active users in the cell. So I wondered how the throughput graph would look like in a 5 GHz Wi-Fi network with many users. A place with heavy Wi-Fi use are universities, so I decided to run a few throughput tests in an Eduroam Wi-Fi network. And here’s how the throughput of an iperf3 download from one of my servers to my notebook looked like:

I took this throughput graph on a 5 GHz Wi-Fi channel on which the beacon frames announced that more than 50 devices were currently associated. As you can see, the throughput varied mostly between 150 and 300 Mbit/s, with a (single) short downwards dip to 50 Mbit/s. Overall, I’d say this is stellar performance in a relatively busy environment. Also, if you compare basic shapes and behavior of the transmission, this looks strikingly similar to the LTE throughput graph in the previous blog post on the topic.

But there is more…

TCP Tracing – Part 3 – Unexpected Fluctuations in Belgium

In part 1 of this series, I compared how the TCP throughput graphs look like for a VDSL vs. an LTE link in stationary and mobility scenarios. Recently, I’ve been to Belgium for a few days and I was expecting to see pretty much the same LTE throughput behavior as in my home network. But it turned out that the graph looks entirely different there.

While I was in Belgium, my stationary scenario was in a location with relatively week signal strength, around -102 dBm. As it was pretty much in the countryside, I was limited to two LTE carriers, one on band 20 and one on band 3. As a reference, here’s a graph in similar network conditions in my home network in Cologne:

To me, this looks like a nice curve. I had 4 LTE carriers with low signal strength, and most of the time, throughput was well beyond 100 Mbit/s with only a few dips down to 75 Mbit/s.

And here’s the graph in the Belgian network I used:

TCP Tracing – Part 2 – Keep PCAP Sizes Reasonable

I’ve been doing quite some tracing on the TCP/IP layer recently and especially at higher speeds over links that support hundreds- to thousands of megabits per second, pcap dump files are getting rather large. Fortunately, both Wireshark and tcpdump offer an option to cut packets after a configurable length. That’s perfect for throughput tracing, as I’m only interested in what is happening on the TCP layer and not in the content of the packets. So here are the commands I usually use to dump large data transfers on an interface for later analysis:

Debugging Non-Interactive Bash Scripts – Pipe Everything to Syslog

Here’s a cool hack I’ve recently come across when I wanted to modify a Bash script that runs in the background without a shell:

As usual when I touch Bash with its strange syntax, my modifications didn’t work at first, and only sending debug messages with logger() to syslog wasn’t doing me much good in this case. So I was thinking that it would nice if I could hook into stdout and stderr of the script to see Bash’s error message when the script falls over? So I googled a bit and found a solution that pipes all output to syslog. All that is required is adding the following line at the beginning of the script:

exec 1> >(logger -s -t $(basename $0)) 2>&1

Looks short and innocent and very Bash syntax like. And it works like a charm, all bash script output goes to the syslog. And if you would like to understand just what this line does, have a look here.

TCP Tracing – Part 1 – How TCP Reacts to Changing Network Conditions

After the interesting experience of chasing packet loss issues of my fiber router in Paris and learning about different TCP congestion control algorithms in the process, I wondered how TCP handles changing network conditions in wireless networks. After all, especially when on the move, air interface conditions and general load of the cell changes all the time. So how does that look like on the TCP/IP level? To find out, I made a number of throughput tests and generated graphs from the data dumps that show different scenarios. As a baseline, I first made an iperf3 downlink throughput test over a Wi-Fi link and a fixed line VDSL backhaul. Here’s how the throughput looks like:

The graph is pretty much flat at around 100 Mbps with a few minor wiggles, which is what I expected to see from a fixed line and completely uncongested link.

Now let’s have a look how the same download looks like over an LTE connection, again with Wi-Fi as the last hop in a stationary, i.e. non-moving setup:

On-Train Internet Experience between Cologne and Hamburg

I’ve been traveling quite a bit by train in Germany lately and I take from the press that German network operators have made some effort to improve the on-train network experience. And indeed, the experience on a recent trip from Cologne to Hamburg was a lot better compared to what I remember from the past.