WirelessMoves

While most features of Prosody and Conversations for XMPP messaging are straight forward there is one feature that is very popular in other IM systems that requires a bit of manual tweaking to set-up: Pictures in Group Chats.

To be able to post pictures that are shown automatically in a group chat in Conversations, the server needs to support XMPP extension XEP-363, "HTTP File Upload". If you have a Prosdoy server at home a corresponding module can be added without too much difficulty. I haven't seen widespread support on public servers yet, with Jabber.at being a notable exception. If you know of others supporting the feature, please consider leaving a comment below. If you are with a public server you can check in Conversations if it is supported by going to "Manage Accounts", clicking on the account that is active at the moment, go to the "…" in the top right corner menu and select "Server Info". That brings up a list that shows which XMPP features are supported by the server. The screenshot on the left shows the features supported by my server, with XEP-363 being the important one for pictures in group chats.

To install the http_upload module in Prosody, have a look here. Once done and Prosody is restarted, support will be indicated in Prosody in the "Server Info" overview. After that there is one more hurdle to be overcome. As it could be potentially dangerous to automatically load in a message stream, Conversations restricts this to pictures that are sent via HTTP links from known contacts. For pictures received from other contacts, only the URL from which they can be downloaded is shown. A long-press on the URL will then download the picture on demand. By default, chat groups created in Conversations are anonymous so even if you know others communicating in a group, Conversations can't identify them and hence only the link shows up. So to see pictures in a chat group instead of links, the group needs to be set into "non-anonymous" mode before participants are added. In this regard, security is a bit in the way of usability I'm afraid.

One other thing is important to remember: As group chats are not end-to-end encrypted (yet), pictures are uploaded to the XMPP server and are stored there in the clear. TLS still protects the conversation so nobody can eavesdrop in a public and open Wi-Fi hotspot. That's probably enough for most people but one should be aware of the fact and always remember that an unencrypted copy is stored for a while on the server. But from what I hear a solution is already in the works.

Already back in August there have been reports that mobile network consolidation has also reached Italy with the proposed merger of Wind and 3-Italy. This would, like in many other countries in Europe over the last few years, reduce the number of independent networks from four down to three. It is a regrettable trend as there are many signs that three network operators with physical infrastructure competing with each other is too little competition. An interesting example of this is France, where the emergence of Free as a fourth mobile network operator back in 2011/12 has significantly heated up competition and reduced prices that were at a very high base compared to most other countries in Europe. Which makes me muse if Free was the last 4th mobile network operator to start operation in a European country in the wake of a wave of consolidation moves in other parts of Europe!?

A couple of days ago, Skype suffered a global outage which made it to a couple of tech news websites and thus caught the attention of the nerd community for about a millisecond or so. Once service was restored, however, it wasn't even newsworthy to report. Global outage and nobody cares?

I use Skype quite regularly so I also noticed that the service was down and in the process trying to get it working again I managed that the Skype app threw away my super long super secure Skype password and I had to spend at least 5 minutes to get it back up working again. Couldn't have happened with a SIM card approach I thought.

But how come nobody really complained about an almost day-long global outage? That doesn't necessarily speak for Skype's popularity. I think the reaction would have been quite different if Facebook or WhatsApp would have been globally down for the same amount of time. That doesn't bode well for Skype's owner Microsoft…

The other thought I had was that in the 'good old days' the worst that could have happened and actually did happen very seldom were local outages of the communication system when a local exchange or the long distance link to it went down for some reason. But that was pretty much it, everything was distributed and even a failure of a long haul transport exchange wouldn't have brought down the network in a whole country, let alone cause a global outage. But these days are long gone, today systems are built for global service and despite of redundancy and fail-safe mechanisms occasionally fail massively on a global level.

Something is wrong with that approach, the Internet was build with survivability in mind, not for centralized services, control or management that cause global outages. But while a couple of decades ago, a whole building was necessary to house a local exchange, a couple of rows of server racks are now sufficient for central control of a nation's telephony system. Sure, that's a lot cheaper than owning, maintaining and powering buildings and equipment in each town to keep the telephone system running but in terms of reliability it is a nightmare. It's time to find a middle ground!

When the XMPP protocol for instant messaging was first created one and a half decades ago the protocol could rightly assume stable Internet connectivity on the client side. Few if any people at the time thought about mobile networks and users moving around, loosing network coverage every now and then or suddenly changing IP addresses. As a result, XMPP in its original flavor doesn't deal with such scenarios very gracefully and drops messages that are in flight during such events.  Fortunately, XMPP did not stand still and there's a fix for this.

The fix is called "stream management" and has been specified in XMPP extension XEP-198. What stream management basically does is message buffering on the server side and requiring an acknowledgement of the client before the message is discarded. In practice, the stream management implementation on my Prosody server at home can easily correct for client side "coverage holes" of well beyond 5 minutes. Works like a charm. Out of the box, stream management is deactivated in Prosody, but it can easily be switched on by downloading and copying "mod_smacks" to Prosody's module directory and then activating it in the config file. The Prosody website has all the necessary details here. A must have feature for mobile clients, no doubt.

R&D and marketing are usually a couple of steps ahead of what's really deployed in live LTE networks so I thought I'd write a post today and the state of the art I can currently see in live networks and relate that to a number of recent press releases about new features and what I think we will realistically see next in networks.

From what I can see, the state of the art in live LTE networks is a maximum theoretical downlink speed of 300 Mbit/s. In some countries like Germany, this is achieved by bundling two 20 MHz downlink carriers for a combined 40 MHz channel. According to this post over at Gigaohm a network operator in Korea achieves similar speeds by combining 3 carriers, one with 20 MHz and two with 10 MHz. So while that requires more sophisticated devices the end result is the same. Network operators in the US probably do carrier aggregation as well these days but since they mostly use 10 MHz channels, with some exceptions, they are not quite reaching those theoretical speeds yet.

So, what are the next steps? Like in Korea, network operators in other parts of the world also have spectrum in more than two frequency bands so it is likely that 3 carrier aggregation (3CA) will be used in other parts of the world to go beyond the theoretical 300 MBit/s, soon. In Europe for example, some network operators could bundle resources in the 800, 900, 1800, 2100 and 2600 MHz band as recent spectrum auctions have left them with ample resources for now to expand their services in the frequency domain. Bundling 3 carriers with 10, 20 and 20 MHz would result in a theoretical top speed of 375 Mbit/s and should somebody be lucky enough to have three 20 MHz carriers, it would result in 450 Mbit/s.

On the mobile device side, most devices sold today are not carrier aggregation capable. This is still the domain of the high end devices such as the Galaxy S6. This is likely to change, however, and I expect that in 18 months from now, 2CA will be the status quo.

In the meantime, high end devices will have gone to 3CA. Qualcomm has recently announced an LTE Category 12 downlink modem that can bundle 3 carriers in downlink which also supports 256QAM modulation for an additional speed boost to the 64QAM currently used close to the base station. Instead of 450 Mbit/s, a theoretical top speed of 600 Mbit/s is thus possible when combining three 20 MHz carriers. Few network operators, however, are likely to have three carriers with such a bandwidth available. Also, 256QAM is only reachable very very close to the base station so I wouldn't count on it making a lot of a difference in real live. For me the most interesting part of the announcement is that the chipset will also support LTE Carrier Aggregation in the Uplink direction. So far, Uplink speeds even in networks using 20 MHz carriers are limited to 16QAM and a single carrier which results in a theoretical peak of 50 Mbit/s. The new Qualcomm modem will support 64QAM in uplink direction and bundling two 20 MHz carriers for a theoretical uplink speed of 150 Mbit/s, which makes it a LTE category 13 uplink device.

While the numbers are certainly impressive, I don't think I need 600 Mbit/s in the downlink on a mobile device anytime soon. As usage of networks increases, however, the main benefit I see of the various advances in LTE carrier aggregation is that more advanced devices will be able to send and receive data over a broader channel, which is important as networks are filling up. It would of course also be possible to make non-carrier aggregation devices hand-over to other frequency bands if a carrier is already highly loaded. However, measuring the availability of other carriers is a non trivial task as well so going the full mile isn't much of an additional trouble.

I've been running Prosdody, an XMPP messaging server on one of my Raspberry Pi's for ages now, mostly for desktop instant messaging and testing mobile messaging clients every now and then. Yes, I could use a public XMPP server but why should I if I can host it myself, too? It's easy to install from the Debian package archive and easy to configure. Now that I've found 'Conversations' a fantastic mobile messaging app, I wanted to upgrade Prosody to take advantage off a number of new features such as for example handling of temporary absence of clients due to network coverage holes which are not part of the version contained in the Debian archive. It turned out that the upgrade process is rather simple.

Instead of first de-installing the Debian archive version (0.8) and using Prosody's package repository to install the latest Prosody version (currently 0.9.8) from scratch one can go ahead and directly configure to use the external repository and install the latest version over the existing one. It only takes a few commands which are described here generically. For Debian on a Raspberry Pi they look like this:

sudo apt-get install software-properties-common python-software-properties
sudo add-apt-repository "deb http://packages.prosody.im/debian wheezy main"
wget https://prosody.im/files/prosody-debian-packages.key -O- | sudo apt-key add –
sudo apt-get update
sudo apt-get install prosody

Before doing so, I saved the configuration file from /etc/prosody to be able to restore it later. There are only few changes between the old and new configuration file so once the upgrade process was finished I renamed the new configuration file, put my old one back in place, added the new parameters to my own configuration file and restarted the server with 'sudo service prosody restart'. The whole process takes 10 minutes at most so it's not much trouble at all.

If you are installing Prosody for the first time you can of course skip copying the configuration file and instead configure it from scratch. Only a few parameters need to be filled in and the configuration file is well documented. Once done, the only other thing that needs to be done for a new installation is to create user accounts as documented here.

Before everybody in the EU went on summer vacation this year, legislation was finalized to work towards abolishing roaming charges mid-2017. A number of news organizations took bits and pieces out of EU press releases and made a nice story out of it but the details were not widely discussed. I recently came across the detailed proposed legislation that can be found here. From my point of view it says the following:

From the press release:

"Under today's agreement roaming charges will cease to exist in the EU as of 15 June 2017."

That part is pretty clear. The press release goes on and says:

"[…] from April 2016, roaming will become even cheaper: operators will only be able to charge a small additional amount to domestic prices up to €0.05 per minute of call made, €0.02 per SMS sent, and €0.05 per MB of data (excl. VAT)."

The legislative proposal that can be found here goes into the details of what these statements actually mean in detail. Concerning the second statement about the 2016/17 interim-timeframe it is made clear that network operators have to inform their customers that their home tariff also applies while roaming in the EU but that they are levying a surcharge for the time being. In other words they have to spread the message that their standard (home) tariff now also applies abroad (but with a surcharge for the time being). On 15 June 2017 they can then announce to their customers that the surcharge has been lifted.

Concerning the full abolition of roaming charges, things are a bit more difficult. Here are some excerpts that shed some light on how this part is going to be implemented:

Article 6a – Abolition of retail roaming surcharges:

"With effect from 15 June 2017, provided that the legislative act referred to in Article 19(2) is applicable on this date, roaming providers shall not levy any surcharge in comparison to the domestic retail price on roaming customers in any Member State for any regulated roaming call made or received, for any regulated roaming SMS/MMS message sent and for any regulated data roaming services used, nor any general charge to enable the terminal equipment or service to be used abroad, subject to Article 6b and 6b bis."

Article 6b – Fair Usage

"Roaming providers may apply […] a “fair use policy” […] to prevent abusive or anomalous usage of regulated retail roaming services […] for purposes other than periodic travel."

The fair usage clause is the part that is not widely discussed in public but it raises a number of interesting points. First, according to the proposed legislation, fair use policies must be approved by national telecom regulators. Regulators have to come up with a set of rules by 15 December 2016 when a FUP can be applied based on factors such as the imbalance of incoming and outgoing data traffic over roaming interfaces, pricing levels in different member states, traveling patterns in the EU (read – some countries have more people visiting compared to the local population than others).

The way I interpret the text is that the fair usage policy, which has yet to be defined by the telecom regulators will govern payments between network operators if there is an imbalance of incoming and outgoing traffic in some parts of the EU or if there is an over proportional roaming traffic. It does NOT mean, as far as I understand it, that end users have to pay extra in some circumstances if they use more voice minutes, SMS or data volume over a certain threshold.

The proposed legislation also makes it clear that network operators can take measures to ensure the system is only used by customers for occasional travel to other EU countries and not to buy a SIM card with a cheap data subscription from an operator in one country to be permanently used in another country in another operator's network that is more expensive. It's going to be interesting where the line is drawn between occasional use and misuse and which steps can be taken by network operators to prevent misuse.

Note: This is not the typical post on this blog entry as you will see but I couldn't find the information I put together below easily so I hope a search engine will make this easier for others searching how to map characters to keys in Linux in the future.

Two years ago I was faced with the problem of assigning language specific characters to a key on the keyboard from a language other than what was currently used. After a lot of searching I came up with the xmodmap command in Linux that does the job nicely. At the time I only needed to assign basic Latin character set. My hack didn't work however for characters in extended character sets that do not have names assigned to characters but just a number. So I had to go looking if and how such characters can be mapped with xmodmap as well.

Once you know it, the answer is actually quite simple, just use the UTF number in hexadecimal format assigned to the character with the xmodmap command. Here's an example that maps the combined "oe" character used in French to the right-ALT + 'o' key:

xmodmap -e "keycode 32 = o O o O U0153 U0152 U0153 U0152"

Which of course raises the question of how to find the character code in the first place!? The easiest way I could come up with is to use Libreoffice's "Insert – Special Character" functionality as shown in the screenshot on the right. And to get upper and lower case mappings of the character use the corresponding adjacent character numbers from the table.

After my previous glowing review of the 'Conversations' XMPP Client App for Android I thought it would be a good idea to write a couple of follow up posts to talk about some of the technical topics I could only quickly mention in the review. So this first follow up post is about XMPP federation and why that is a good thing.

Let's have a look at the mainstream: Services like WhatsApp and others use a centralized server system. That makes it easy for users to register to the service initially and to find each other later-on. Unfortunately centralization also means that messages can be intercepted and stored in this central place, usage behaviors can be analyzed and unwanted adds and other things can be injected into the message stream. Also, the centralized server is aware of the IP addresses of all clients connected to it so location profiling is also possible.

Conversations on the other hand is an XMPP client. XMPP is a federated system of many independent servers on the Internet that can communicate with each other. Users are registered with individual servers and there is no central instance, which is why it is called a 'federated' system. If a user who has an account on server A wants to send a message to a user with an account on server B, server A and B communicate with each other to exchange the user's message. To make user IDs unique it looks like an email address, i.e. there's a username, an '@' sign and a server name.

In practice there's two ways to use the system. The simpler case is to create an XMPP account, sometimes also called a 'Jabber' account, on one of the community operated and open XMPP servers. And there are quite many of them. Its no problem if one wants to communicate with a user with an account somewhere else as servers communicate with each other. The other way to use the XMPP system is to run an XMPP server at home, e.g. on a Raspberry Pi. That's obviously a bit more work but if one has previously installed other servers at home such as Owncloud it's not too difficult to do. If the server at home should be part of the overall XMPP federation rather than just serving its own users, it has to be reachable via an official domain name. A dynamic DNS address suffices.

Just like email, really, just instant 🙂