In the previous post I've described the problems frequently encountered with Internet access provided at international meetings with more than just a few participants. One way to ensure good connectivity as a host is obviously talking to the ISP or conference venue manager and requesting feedback to questions along the lines of my previous post. The other alternative is of course to do it yourself, provided you have the necessary expertise, time for preparation and time during the meeting to keep things going and of course the necessary backhaul capacity available at the meeting location. As I had all these things available for a recent meeting of 40 people I hosted, I decided to do it myself. And here are the details:
The Wi-Fi
To connect the participants notebook, smartphones, pads, etc. I used two Wi-Fi access points, one occupying a standard 20 MHz 802.11b/g channel in the 2.4 GHz band and, for additional capacity, a 40 MHz 802.11n channel in the 5 GHz band. Forget about using off-the self Wi-Fi access points for home use. Many of those will only allow 20 or so devices on the network before refusing further service. Also, their user interface gives little insight how well it is running, if there are performance issues and if so because of what (see previous post). With 40 people on your back you don't want to fly this blind.
For ultimate configurability, manageability and reliability I decided to use a Linksys WRT-54GL 802.11b/g router. This device has been on the market for many years and has, compared to recent devices, a quite low powered CPU and little memory. Also the Wi-Fi chip features only few enhanced features. For this application, this is rather and advantage as less supported extra features on the air interface means fewer interoperability issues.
Also, the WRT-54GL's default OS can be replaced with an open source project such as Open-WRT and DD-WRT which offers the required flexibility and insight into how well the network is running. In the past, I've used Open-WRT for various experiments and this time chose to go for DD-WRT. No particular reason, I just wanted to try something else.
In addition to the Wi-Fi interface, I used the WRT-54GL as an IP router and for network address translation (NAT). The WAN interface to the router was connected to the backhaul as described in a minute, with a single configured IP address. On the LAN and Wi-Fi part of the network, the router acted as a DHCP server and supplied IP addresses to devices requesting it over Wi-Fi or Ethernet. That makes one somewhat independent from the DHCP capabilities provided by the backhaul part of the network.
The DHCP Sever
For 5 GHz Wi-Fi connectivity I used a second Wi-Fi access point on which the DHCP server was deactivated, thus acting only as a Wi-Fi to Ethernet bridge to the WRT-54GL, which did the rest of the work. One limit of DD-WRT is that the user interface only allows to configure a /24 subnet for the DHCP server, limiting the number of theoretical simultaneous users to around 250. Not an issue for my meeting but it is not an order of magnitude away from my requirements and thus has to be kept in mind in case the setup is to be scaled up for larger meetings in the future.
The Backhaul and Amount of Data Transferred
As said in the previous post I was a bit daring and used a cellular broadband network as a backhaul. In terms of raw downlink and uplink speed, enough capacity was available, with speeds of beyond double digit MBit/s reached at any time. Your mileage may vary, however, depending on what kind of network you use, how fast the backhaul to the base station you are using is, how good reception conditions are at your meeting, etc. Often, meetings are deep indoors or often underground. Under such circumstances, forget about using cellular. In some cases you are lucky and have dedicated indoor coverage with repeaters, something not uncommon in hotels or meeting venues in Europe. There is no way around a site survey before the meeting to evaluate signal conditions and data transfer rates available at that location. If you don't do it, you are likely in for a very bad surprise. Another thing that can be a bit tricky to find out is if the network operator can handle the number of simultaneous VPN connectiosn required by the meeting participants over a single cellular connection. For me it was the case and I had about 40 VPNs running in parallel from my network into the Internet over a single cellular connection. There is no guarantee, however, that this will work with all network operators (fixed or mobile).
The next thing that needs to be considered is the amount of data that is transferred. The 40 participants in my meeting transferred around 4 GByte of data per day. That's about 100 MB per day per participant. Considering that most VPNs already generate several megabytes worth of data per hour just for keep-alive signaling that value is not very high. In other meetings, depending on the kind of applications use, much more or much less data might be transferred, it's difficult to predict. In other words, don't plan to run the meeting with a subscription that starts to throttle the connection after a gigabyte or two.
Windows Internet Connection Sharing
As my Linksys router is quite old it doesn't have a USB port for the Internet dongle. My second choice for cellular connectivity was therefore a notebook running Windows Visa and Internet Connection Sharing (ICS) between the dongle and the Ethernet port. (Windows ICS has come in handy many times since I first wrote about it back in 2006)
The WRT-54 was then connected via an Ethernet cable to the Windows notebook which routed the packets, via another network address translation, over the USB dongle to the Internet. Yes, Windows and double NATing, far from being the ideal solution. The reason for the double-NATing was that I had some issues with the Windows ICS DHCP implementation assingning IP addresses reliably to client devices. Therefore I decided to use NATing on the WRT-54 as well to shield client devices from Microsoft's strange implementation.
The Windows NAT was also my single blind spot, as I am not aware of the maximum number of simultaneous NAT translations the OS will do. Therefore having a NAT with statistics on the WRT-54 at least would have provided me with numbers as to how far it went in case it failed. Fortunately, it didn't.
Maximum number of NAT translations and Processor Power
On the Linksys router the number of simultaneous NAT translations can be configured and the highest value, due to the low amount of RAM is 4096. In practice around 400 – 1200 translations were required simultaneously at any one time, well below the theoretical limit. However, again not an order of a magnitude away from real life use so something to be kept in mind. Performing network address translation requires the router to look into every IP packet and changing the header. In other words, the processor and RAM get quite utilized. The Linksys WRT-54GL with its 200 MHz processor was doing the job perfectly with packet round trip times never going up no matter what the current data rate and number of translations in use where. A good indication that things were going well. I could notice, however, that the processor was pretty much running at full capacity so I am not sure how far I was away from processing power having become a bottleneck. Perhaps not much. So for meetings with more participants I would go for a more powerful device in the future. More about that below.
Bottlenecks Elsewhere
The picture on the left (click on the picture to enlarge) shows around 30 minutes of network traffic during the meeting. As you can be seen, the maximum throughput peak was at around 13 MBit/s, by far below of what the backhaul link would have been able to provide. The bottleneck was somewhere else however. My private VPN tunnel is limited to around 8 MBit/s and that of my employer to about 2 MBit/s. From that bandwidth usage graph I take it that other companies have similar restrictions. Also, most web servers and file servers won't send files and web pages down the pipe with more than a few MBit/s per user. As the graph shows a duration of around 30 minutes individual web page downloads only register as narrow peaks. The big red areas are large file downloads, several tens of megabytes at a time. All of the blue area in the graph is unused backhaul capacity.
Summary
Altogether, the setup described above provided ample capacity and very high instantaneous data rates to every user in the network throughout the meeting days. From an application usage point of view the ultimate stress test was a webex screen sharing session of around 20 computers in the room via a webex server on the Internet. Backhaul capacity required for this wasn't very high compared to what was available but the sheer number of packets flowing when the screen owner modified text and pictures was probably at its peak during that time.
And, most important, connectivity to the cellular network didn't drop a single time during the three meeting days. Quite an achievement of the cellular network and devices used locally. Even with the best preparations, there are some unknowns that could make things pretty rough during the meeting. Therefore my last advice in this post is to always have a backup plan available. Fortunately, I didn't need mine.
very interesting. Thanks for sharing