My First IPv6 “First Call” To My Own Services At Home

A date to remember for me: On 15th January 2016 I contacted my web server at home running Owncloud and Selfoss for the first time over IPv6. From an end user's point of view no difference is visible at all but from a technical point of view it's a great "first time" for me, made even sweeter by the fact that my PC was not connected to another IPv6-enabled fixed line but connected via tethering to a smartphone with dual-stack IPv4v6 cellular connectivity.

The Thing With Dynamic IPv6 Addresses for Servers

And it's been a bit of a struggle to put together, this IPv6 stuff is not as straight forward as I hoped it would be. For a crash course I wrote back in 2009 have a look, here, here, here and here. The major challenge that I had to overcome for this to happen is to find a dynamic DNS service that can handle not only dynamic IPv4 addresses but also dynamic IPv6. Noip.com, where I host my domain and where I use the dynamic DNS service can handle IPv6 addresses for my domain but only static entries. The response to a support question how to do dynamic IPv6 addresses with them resulted in the little informative answer that they are working on it but no date has been announced by when this will be available. Hm, looking at their record, they seem to be working on IPv6 already since 2011 so I won't get my hopes up that this will happen soon. Is it really that difficult? Shame on you!

O.k., another dynamic DNS service I use is afraid.org and they do offer dynamic DNS with IPv6. Unfortunately, they have a DNS entry time to live (TTL) for IPv6 of 3600 seconds, i.e. 1 hour. This is much too long for my purposes as my IPv6 prefix changes once a day and any change must be propagated as quickly as possible and not only after an hour in the worst case. They offer a lower TTL with a paid account, but their idea and my idea of how much this may cost are too far apart. I've found a couple of other dynamic IPv6 services but they were also not suitable for me because they also had TTLs that were too long for my purpose.

One option I found that didn't have this restriction is dynv6.com. Their service is free and they do offer IPv4 and IPv6 dynamic DNS with a TTL of 1 minute but only for their own domain. Not an option for me either, I want to be reachable via my own domain. Kind of a deadlock situation…

But here's how I finally go it to work: The Domain Name System has a forwarding mechanism, the "Canonical Name Record" (CNAME). By using this mechanism, I can forward DNS queries for my domain that is hosted at noip.com (let's say it's called www.martin.com) to my subdomain at dynv6.com (let's say my domain there is called martin.dynv6.com). So instead of updating the DNS entry for www.martin.com when my IPv6 address changes once a day I can now update martin.dynv6.com which has a TTL of 1 minute while the CNAME forwarding at noip.com from www.martin.com to martin.dynv6.com in the DNS system is static and remains unchanged.

As a result the web page name in the browser remains "www.martin.com" but I can use my dynamic IPv6 record at dynv6.com where customer specific domains are not offered. Not ideal but it will do until NO-IP.com gets their act together.

LTE-A Pro for Public Safety Services – Part 2 – Advantages over PMR in 2G

LTE for Public Safety Services, also referred to as Private Mobile Radio (PMR) is making progress in the standards and in the first part of this series I’ve taken a first general look. Since then I thought a bit about which advantages a PMR implementation might offer over current 2G Tetra and GSM PMR implementations and came up with the following list:

Voice and Data On The Same Network: A major feature 2G PMR networks are missing today is broadband data transfer capabilities. LTE can fix this issue easily as even bandwidth intensive applications safety organizations have today can be served. Video backhauling is perhaps the most demanding broadband feature but there are countless other applications for PMR users that will benefit from having an IP based data channel such as for example number plate checking and identity validation of persons, access to police databases, maps, confidential building layouts, etc. etc.

Clear Split into Network and Services: To a certain extent, PMR functionality is independent of the underlying infrastructure. E.g. the group call and push to talk (PTT) functionality is handled by the IP Multimedia Subsystem (IMS) that is mostly independent from the radio and core transport network.

Separation of Services for Commercial Customers and PMR Users: One option to deploy a public safety network is to share resources with an already existing commercial LTE network and upgrade the software in the access and core network for public safety use. More about those upgrades in a future post. The specific point I want to make here is that the IP Multimedia Subsystem (IMS) infrastructure for commercial customers and their VoLTE voice service can be completely independent from the IMS infrastructure used for the Public Safety Services. This way, the two parts can evolve independently from each other which is important as Public Safety networks typically evolve much slower or and in fewer steps compared to commercial services as there is no competitive pressure to evolve things quickly.

Apps vs. Deep Integration on Mobile Devices: On mobile devices, PMR functionality could be delivered as apps rather than built into the operating system of the devices. This allows to update the operating system and apps independently and even to use the PMR apps on new devices.

Separation of Mobile Hardware and Software Manufacturers: By having over-the-top PMR apps it’s possible to separate the hardware manufacturer from the provider of the PMR functionality except for a few interfaces which are required such as setting up QoS for a bearer (already used for VoLTE today, so that’s already taken care of) or the use of eMBMS for a group call multicast downlink data flow. In contrast, current 2G group call implementations for GSM-R require deep integration into the radio chipset as pressing the talk button required DTAP messages to be exchanged between the mobile device and the Mobile Switching Center (MSC) which are sent in a control channel for which certain timeslots in the up- and downlink of a speech channel were reserved. Requesting the uplink in LTE PMR requires interaction with the PMR Application Server but this would be over an IP channel which is completely independent from the radio stack, it’s just a message contained in an IP packet.

Device to Device Communication Standardized: The LTE-A Pro specification contains mechanisms to extend the network beyond the existing infrastructure for direct D2D communication, even in groups. This was lacking in the 2G GSM-R PMR specification. There were attempts by at least one company to add such a “direct” mode to the GSM-R specifications at the time but there were too many hurdles to overcome at time time, including questions around which spectrum to use for such a direct mode. As a consequence these attempts were not leading to commercial products in the end.

PMR not left behind in 5G: LTE as we know it today is not likely to be replaced anytime soon by a new technology. This is a big difference to PMR in 2G (GSM-R) which was built on a technology that was already set to be superseded by UMTS. Due to the long timeframes involved, nobody seriously considered upgrading UMTS with the functionalities required for PMR as by the time UMTS was up and running, GSM-R was still struggling to be accepted by its users. Even though 5G is discussed today, it seems clear that LTE will remain a cornerstone for 5G as well in a cellular context.

PMR On The IP Layer and Not Part of The Radio Stack (for the most part): PMR services are based on the IP protocol with a few interfaces to the network for multicast and quality of services. While LTE might gradually be exchanged for something faster or new radio transmission technologies might be put alongside it in 5G that are also interesting for PMR, the PMR application layer can remain the same. This is again unlike in 2G (GSM-R) where the network and the applications such as group calls were a monolithic block and thus no evolution was possible as the air interface and even the core network did not evolve but were replaced by something entirely new.

Only Limited Radio Knowledge Required By Software Developers: No deep and specific radio layer knowledge is required anymore to implement PMR services such as group calling and push to talk on mobile devices. This allows software development to be done outside the realm of classic device manufacturer companies and the select few software developers that know how things work in the radio protocol stack.

Upgradeable Devices In The Field: Software upgrades of devices has become a lot easier. 2G GSM-R devices and perhaps also Tetra devices can’t be upgraded over the air which makes it very difficult to add new functionality or to fix security issues in these devices. Current devices which would be the basis for LTE-A Pro PMR devices can be easily upgraded over the air as they are much more powerful and because there is a broadband network that can be used for pushing the software updates.

Distribution of Encryption Keys for Group Calls: This could be done over an encrypted channel to the Group Call Server. I haven’t dug into the specification details yet to find out if or how this is done but it is certainly possible without too much additional work. That was not possible in GSM-R, group calls were (and still are) unencrypted. Sure, keys could be distributed over GPRS to individual participants but the service for such a distribution was never specified.

Network Coverage In Remote Places: PMR users might want to have LTE in places that are not normally covered by network operators because it is not economical. If they pay for the extra coverage and in case the network is shared this could have a positive effect when sharing a network for both consumer and PMR services. However, there are quite a number of problems with network sharing one has to be careful when proposing this. Another option, which has also been specified, is to extend network coverage by using relays, e.g. installed in cars.

I was quite amazed how long this list of pros has become. Unfortunately my list of issues existing in 2G PMR implementations today that a 4G PMR system still won’t be able to fix is equally long. More about this in part 3 of this series.

Owncloud Must Think “WordPress” When It Comes To Updating

There is two extremes in the popular cloud space when it comes to ease of updating: WordPress and Owncloud…

On one side is WordPress which has about the most simple and most reliable update process that is fully automatic and doesn't even have to be triggered by the administrator for small upgrades and a simple click when going from one major release to the next. It hasn't failed me once in the past five years. And then there is Owncloud, which is the exact opposite.

Over the past year it failed me during each and every update with obscure error messages even for small security upgrades, broken installations and last resort actions such as deleting app directories and simply ignoring some warnings and to move ahead despite of them. If you think it can't be that bad, here's my account of one such update session last year. In the meantime I've become so frustrated and cautious as to clone my live Owncloud system and first try the update on a copy I can throw away. Only once I've found out how to run the upgrade process, which unfortunately changes every now and then as well, which things break and how to fix them do I run an upgrade on my production system. But perhaps there is some hope in sight?

My last upgrade a couple of days ago worked flawlessly, apart from the fact that the update process has changed again and it's now mandatory to finalize the upgrade process from a console. But at least it didn't fail. I was about to troll about the topic again but this morning I saw a blog post over at the Owncloud blog in which they finally admit in public that their upgrade process leaves a lot to be desired and that they have started to implement a lot of things to make it more robust and easier to understand. If you have trouble updating Owncloud as well I recommend to read the post, it might make you feel a bit better and give you some hope for the next update process.

And to the Owncloud developers I would recommend to go a bit beyond what they have envisaged so far: Blinking lights, more robustness and more information of what is going on during an update is a nice thing and will certainly improve the situation. In the end, however, I want an update process that is just like WordPress'es: You wake up in the morning and have an email in your inbox from your WordPress installation that tells you that it has just updated itself, that all is well and that you don't have to do anything anymore! That's how it should be!

LTE-A Pro for Public Safety Services – Part 1

In October 2015, 3GPP has decided to refer to LTE Release 13 and beyond as LTE-Advanced Pro to point out that LTE specifications have been enhanced to address new markets with special requirements such as Public Safety Services. This has been quite long in the making because a number of functionalities were required that go beyond just delivery of IP packets from point A to point B. A Nokia paper published at the end of 2014 gives a good introduction to the features required by Public Safety Services such as the police, fire departments and medical emergency services:

  • Group Communication and Push To Talk features (referred to as "Mission Critical Push To Talk" (MCPPT) in the specs, perhaps for the dramatic effect or to perhaps to distinguish them from previous specifications on the topic).
  • Priority and Quality of Service.
  • Device to Device communication and relaying of communication when the network is not available.
  • Local communication when the backhaul link of an LTE base station is not working but the base station itself is still operational.

Group Communication and Mission Critical Push to Talk have been specified as IP Multimedia Subsystem (IMS) services just like Voice over LTE (VoLTE) that is being introduced in commercial LTE networks these days and can use the eMBMS (evolved Mobile Broadcast Multicast Service) extension in case many group participants are present in the same cell to only send a voice stream in the downlink once instead of separately to each individual device.

In a previous job I've worked on the GSM group call and push to talk service and other safety related features for railways for a number of years so all of this sounds very familiar. In fact I haven't come across a single topic that wasn't already discussed at that time for GSM and most of them were implemented and are being used by railway companies across Europe and Asia today. While the services are pretty similar, the GSM implementation is, as you can probably imagine, quite different from what has now been specified for LTE.

There is lots to discover in the LTE-A Pro specifications on these topics and I will go into more details both from a theoretical and practical point of view in a couple of follow up posts.

IPv6 At Home And Away Now

In the second half of last year, my mobile network operator of choice has introduced IPv4/v6 dual-stack functionality and since then I've been enjoying IPv6 on my mobile device while away from home. Not that I would notice as a normal user as all services I use can still be reached over IPv4, but as a tech-geek, you know… For me this was a bit ironic as I always assumed that I would have IPv6 on my DSL connection long before I use it on my mobile devices in the cellular network. And I could have, to be honest, but I just didn't want to update my fixed line connection at home to "all-IP" as it's a critical link and I don't change critical infrastructure just like that if not really necessary. Anyway, back in December 2015 I had to switch my DSL line to "all-IP" because my network operator politely forced me to and apart from a number of other sweet things the new package included native IPv6 connectivity if I wanted to.

As I was traveling a lot in December I decided to keep IPv6 off for the time being and start experimenting with it once back home for more than just a couple of hours. So this week I finally go around to switching IPv6 on and just let it ran for a while without any other modifications to make sure my servers are not negatively impacted. So far, things have run smoothly except for one thing I was expecting. After switching on IPv6, my devices immediately found the public IPv6 prefix and assigned public IPv6 addresses to themselves. The servers did so as well, including my Raspberry Pis, a nice side effect of having upgraded them from Raspbian based on Debian Wheezy to Raspbian based on Debian Jessie last year. That will make things a bit easier to make them reachable not only via IPv4 but also via IPv6 from the Internet later. The one thing I was actually expecting to break is that for some services I use VPN connections to overcome geo-blocking. As my external VPN service provider does not support IPv6 but happily returns IPv6 addresses to DNS queries I had to disable IPv6 on that machine.

Speaking of inbound IPv6 to my servers that's going to be an interesting thing to get working. So far I see two issues that have to be addressed:

  • Today I run several servers behind the same IPv4 address and domain name. With IPv6 they will have different IP addresses so using the same domain name is going to be a challenge.
  • My Dynamic-DNS provider does support IPv6 AAAA records but not updating IPv6 records dynamically other than over the web interface. Quite a shame in 2016…

Two fun things to figure out in 2016…

Wi-Fi WPA-Professional with Certificate Authentication

Wifi-ttls-1Today, most Wi-Fi hotspots at home use the standard WPA/WPA2 authentication and encryption mechanism that uses a shared password between the Wi-Fi hotspot and clients. The downside of this approach is that all users have to share the same password which enables an attacker who is in range of the network and in possession of the password to decode encrypted packets if he has observed the initial EAPOL authentication and ciphering dialog of another client. Another downside is that the password needs to be stored in all access points of a Wi-Fi network. All these things are clearly not acceptable in company environments or during public events that want to offer air interface security. For such environments, the Wi-Fi alliance has specified the WPA-Professional authentication approach that can use various authentication methods with certificates and individual passwords for each user. Let's have a closer look at one option of which I was recently able to take a Wireshark trace:

To address the need of companies for a centralized user management, WPA/WPA2-Professional manages certificates and passwords from a central authentication server, often referred to as a RADIUS server. In practice it's not straight forward to discuss such setups because they are mostly used by companies and hence can't be discussed in public. Fortunately I've now found one network that uses WPA2-Professional with a certificate and passwords that can be discussed in public: The Wi-Fi network that was used during 32C3.

As they've described on their Wiki, a server side certificate was used to authenticate the network towards the user via TTLS. To authenticate clients, a username/password of choice could be used in the network. As the conference network administrators were not interested to authenticate users, any username and password combination was accepted. In practice users could remain anonymous this way while at the same time an individual secret was used to generate cipher keys, i.e. packets can't be deciphered by an attacker even if the authentication packets were captured.

The screenshot of the Wireshark trace on the left (here's the pcap in case you want to have a more detailed look) shows how the TTLS/Certificate authentication works in practice. After associating with the network, the Wi-Fi access point asks for a username, which can be anonymous and then tells the user that it wants to proceed with a TTLS-EAP authentication procedure. The client device then answers with a 'Client Hello' packet that contains all cipher suites it supports. The network then selects a cipher suite and sends it's signed certificate to authenticate itself which contains it's public key.

Wifi-ttls-2In company environments, the certificate used in Wi-Fi networks is usually signed by a private certificate authority. To enable the device to validate the signed certificate that was sent, the public key of the certificate authority that has signed the delivered certificate has to be stored in the device before it connects to the network for the first time.

In case of the 32C3 network a public certification authority was used to sign the certificate delivered to the device. As anyone can get a signature for a certificate from a public certification authority if he is the owner of the domain specified in the certificate an additional client side configuration is required to ensure that only signed certificates with the correct domain name of the Wi-Fi network are accepted. Unfortunately, Ubuntu's graphical network configuration tool doesn't have a field to configure this extra information as shown in the second screenshot.

Fortunately it's possible to modify Ubuntu's configuration file for the network after it has been created in '/etc/NetworkManager/system-connections' by adding the 'altsubject' line in the 802.1x section with the domain name used by the Wi-Fi network's certificate.

[802-1x]
eap=ttls;
identity=x
ca-cert=/etc/ssl/certs/StartCom_Certification_Authority.pem
altsubject-matches=DNS:radius.c3noc.net;
phase2-auth=pap
password-flags=1

Putting in a wrong value in this line makes the connection establishment fail so I could verify that the overall authentication process is secure.

Once the client device has accepted the server certificate (packet 14 in the trace) an encrypted handshake message is exchanged that is client specific. For this dialog, the client uses the public key that was part of the certificate to encrypt the message. Decoding the packets on the network side is only possible with the private key. As the private key is never sent over the air an attacker can't use a copy of the certificate for a rogue access point.

Afterward the standard 4 step EAPOL Wi-Fi messaging is used to activate link level wireless encryption, based on an individual secret exchanged during the TTLS process. Packet 22 shows the first encrypted packet exchanged between the access point and the client device, a DHCP message to get an IP address. As the trace was done on the client device the decoded version of the packet is shown. Once the IP address has been received the connection is fully established and user data packets can be exchanged.

Book Review: Pioneer Programmer

If you have some background in computer science you’ve probably come across the term “von Neumann Architecture” before. The term goes back to the brilliant mathematician John von Neumann who, for the first time in 1945, described the computer architecture we still use today with an arithmetic logic unit, a control unit, registers and combined program and data memory in a seminal paper on the EDVAC. As pointed out in the Wikipedia article there is quite some controversy about this paper as it was only intended as a first internal draft for review and only bears van Neumann’s name but not those of the main inventors of the concepts, John Mauchly and Presper Eckert. While intended as an internal paper it was still distributed to a larger community and thus it had the appearance that van Neumann had come with the ideas all by himself. While attempts were made to set the record straight, the term “von Neumann architecture” stuck and has remained in place up to the present day.

There is a lot of controversy about the reasons, motivation and character of Herman Goldstine to distribute the paper without consent. “Pioneer Programmer” the autobiography of Jean Jennings Bartik edited by Jon T. Kickmann and Kim D. Todd has a lot of background information on this and many other topics of the early days of computing in the United States from her point of view. Jean was a member of the initial team of programmers of the ENIAC, the first fully electronic computer in the mid-1940s and could thus witness this and many other events first hand and decided to set a number of things straight with her autobiography. Pretty much forgotten until many decades later, the first ENIAC programmer team consisted solely of female mathematicians as due to the war there was a shortage of male mathematicians and the boys were more interested in building the computing machines than to program them. Pioneer Programmer intends not to only set the record straight but also to tell the story of how women shaped early computing and to describe the difficulties they had in a male dominated scientific world in the US and Europe during that time and the decades afterward. A fascinating story that starts with her childhood on a farm in rural America and ends with the jobs and positive as well as negative experiences she had in the computing industry as a woman in the decades after leaving the ENIAC behind.

Probably not a very well known book but for those who are interested in the facts behind the stories of early computing a must read that I’ve very much enjoyed reading!

A Flatrate For Calling All EU Fixed Lines And Mobiles From Anywhere In The EU

I travel a lot in Europe and I call lots of people in different countries not only when I'm in my home country but also while traveling. In other words, without a good tariff for international calling and data use, it's no fun. A new mobile EU-Flat tariff introduced by my mobile network operator last year has, for the first time, enabled me to use my monthly mobile data volume anywhere in the EU and to call back home without per minute fees for a modest additional monthly fee of 5 euros. This has helped a lot and apart from one exception I didn't have to use local SIM cards anymore.

One thing that the offer didn't include, however, was making calls from my home country or while traveling in the EU to fixed and mobile phones of other EU countries. At prices of well over a euro a minute this had remained prohibitively expensive. But things have moved on since then and my fixed and mobile network operator of choice has made me a bundle offer to extend my current EU-flat to also include voice calls from anywhere in the EU to anywhere in the EU to both fixed and mobile devices. Yes, that's what I've been waiting for for some many years. The 'everything inclusive anywhere EU-Flat' now costs an extra 10 euros on top of my normal fixed line and mobile contracts instead of the 5 euros I paid before but with my usage pattern that's a deal I was more than willing to take.

No longer waiting to make some calls only until I'm back home or in the office or by using complicated dial-in numbers, no nightmares about costs spiraling out of control because that conference call only offers an international dial-in number, it's definitely my tariff add-on of the year!

P.S.: No, this is not an advertisement for a particular fixed and mobile network operator which is why I haven't named the company in the first place. This blog entry is about documenting a very positive change in the telecommunication market and to encourage other network operators to follow.

32C3 – Congress Infrastructure Review And A Plea for GSM ARFCNs for 2016

Like ever year at the end of the Congress one of the last sessions is the Infrastructure Review. Here, the people who built the congress data network, the DECT network, the GSM network and the Seidenstraße talk about the technology they used this year. It always interesting to hear how much data, calls and packets have been shuffled through the networks and how that compares to last year, how many people used the network, how many wireless devices were used, which networking equipment was used and so on and so on. For anyone interested in networks this talk is a must see and fun to watch not only because of the interesting numbers but also because the presentation contains a lot of hacker fun and sarcasm. This year was certainly no exception. The cut video of the session is not yet on the congress streaming server but the raw-uncut version can be found here in the meantime.

One important message I also want to repeat here: So far, the GSM network of the Congress used the 1800 MHz DECT/GSM guard band. This won't be possible in 2016 anymore as that part of the spectrum has been auctioned in 2015 so one of the network operators in Germany has to be kind enough to loan the Congress GSM network organizers a couple of ARFCNs for the week. So if you are working in spectrum planning at a network operator and think you can spare a couple of channels for a week at the Congress location please think about it and get in touch with the organizers. If you don't know how to do that, let me know, I'll be glad to help!

32C3 – Vehicle2Vehicle Communication with IEEE 802.11p

One feature some proponents are pushing for future 5G networks are ultra short reaction times for ultra critical communication, for example between cars. What I failed to understand so far in this discussion was why for car to car communication a fixed network infrastructure and a backhaul network was necessary!? After all, car to car communication mainly makes sense for cars that are in close proximity to exchange information about potential dangers such as emergency braking, breakdowns and their current status such speed, direction etc., etc. It seems that my skepticism was not unfounded because unknown to me and perhaps also to the 1 ms 5G proponents, decentralized solutions not requiring a network infrastructure already exist.

While Europe and the US seem to be on different paths (once again) on higher layers of the protocol stack, both approaches are based on the IEEE 802.11p extension of the Wi-Fi standard. In this "Wi-Fi" flavor, there are no central access points, no fixed equipment and no backhaul of any kind. On top of this physical layer, event and context information is exchanged. An interesting challenge is how to ensure that messages are sent from "real" vehicles and not from rouge devices that want to disrupt traffic, e.g. by sending messages about emergency breaking etc. while at the same time ensuring privacy, i.e. to send messages anonymously to prevent tracking.

The concept that car companies have come up with is a public key infrastructure and cars equipped with a master certificate by car manufacturer. Based on the master certificate, temporary certificates are signed by a certificate authority which are then included in 802.11p messages sent by cars. Vehicles receiving messages can then validate the message by checking the temporary certificate which does not contain the car's identity and which are changed frequently. Rouge devices that do not have a master certificate can't get temporary certificates, at least in theory, and therefore can't include proper temporary certificates in their messages. That makes me wonder of course how hard it might be in the future to get a valid certificate by extracting it from an on-board computer of a vehicle. SIM cards of mobile devices have provided pretty good security over the past decades so there is at least some hope that the master certificates can be stored safely.

For more details, here's the talk on this topic from 32C3.