WirelessMoves

Before everybody in the EU went on summer vacation this year, legislation was finalized to work towards abolishing roaming charges mid-2017. A number of news organizations took bits and pieces out of EU press releases and made a nice story out of it but the details were not widely discussed. I recently came across the detailed proposed legislation that can be found here. From my point of view it says the following:

From the press release:

"Under today's agreement roaming charges will cease to exist in the EU as of 15 June 2017."

That part is pretty clear. The press release goes on and says:

"[…] from April 2016, roaming will become even cheaper: operators will only be able to charge a small additional amount to domestic prices up to €0.05 per minute of call made, €0.02 per SMS sent, and €0.05 per MB of data (excl. VAT)."

The legislative proposal that can be found here goes into the details of what these statements actually mean in detail. Concerning the second statement about the 2016/17 interim-timeframe it is made clear that network operators have to inform their customers that their home tariff also applies while roaming in the EU but that they are levying a surcharge for the time being. In other words they have to spread the message that their standard (home) tariff now also applies abroad (but with a surcharge for the time being). On 15 June 2017 they can then announce to their customers that the surcharge has been lifted.

Concerning the full abolition of roaming charges, things are a bit more difficult. Here are some excerpts that shed some light on how this part is going to be implemented:

Article 6a – Abolition of retail roaming surcharges:

"With effect from 15 June 2017, provided that the legislative act referred to in Article 19(2) is applicable on this date, roaming providers shall not levy any surcharge in comparison to the domestic retail price on roaming customers in any Member State for any regulated roaming call made or received, for any regulated roaming SMS/MMS message sent and for any regulated data roaming services used, nor any general charge to enable the terminal equipment or service to be used abroad, subject to Article 6b and 6b bis."

Article 6b – Fair Usage

"Roaming providers may apply […] a “fair use policy” […] to prevent abusive or anomalous usage of regulated retail roaming services […] for purposes other than periodic travel."

The fair usage clause is the part that is not widely discussed in public but it raises a number of interesting points. First, according to the proposed legislation, fair use policies must be approved by national telecom regulators. Regulators have to come up with a set of rules by 15 December 2016 when a FUP can be applied based on factors such as the imbalance of incoming and outgoing data traffic over roaming interfaces, pricing levels in different member states, traveling patterns in the EU (read – some countries have more people visiting compared to the local population than others).

The way I interpret the text is that the fair usage policy, which has yet to be defined by the telecom regulators will govern payments between network operators if there is an imbalance of incoming and outgoing traffic in some parts of the EU or if there is an over proportional roaming traffic. It does NOT mean, as far as I understand it, that end users have to pay extra in some circumstances if they use more voice minutes, SMS or data volume over a certain threshold.

The proposed legislation also makes it clear that network operators can take measures to ensure the system is only used by customers for occasional travel to other EU countries and not to buy a SIM card with a cheap data subscription from an operator in one country to be permanently used in another country in another operator's network that is more expensive. It's going to be interesting where the line is drawn between occasional use and misuse and which steps can be taken by network operators to prevent misuse.

Note: This is not the typical post on this blog entry as you will see but I couldn't find the information I put together below easily so I hope a search engine will make this easier for others searching how to map characters to keys in Linux in the future.

Two years ago I was faced with the problem of assigning language specific characters to a key on the keyboard from a language other than what was currently used. After a lot of searching I came up with the xmodmap command in Linux that does the job nicely. At the time I only needed to assign basic Latin character set. My hack didn't work however for characters in extended character sets that do not have names assigned to characters but just a number. So I had to go looking if and how such characters can be mapped with xmodmap as well.

Once you know it, the answer is actually quite simple, just use the UTF number in hexadecimal format assigned to the character with the xmodmap command. Here's an example that maps the combined "oe" character used in French to the right-ALT + 'o' key:

xmodmap -e "keycode 32 = o O o O U0153 U0152 U0153 U0152"

Which of course raises the question of how to find the character code in the first place!? The easiest way I could come up with is to use Libreoffice's "Insert – Special Character" functionality as shown in the screenshot on the right. And to get upper and lower case mappings of the character use the corresponding adjacent character numbers from the table.

After my previous glowing review of the 'Conversations' XMPP Client App for Android I thought it would be a good idea to write a couple of follow up posts to talk about some of the technical topics I could only quickly mention in the review. So this first follow up post is about XMPP federation and why that is a good thing.

Let's have a look at the mainstream: Services like WhatsApp and others use a centralized server system. That makes it easy for users to register to the service initially and to find each other later-on. Unfortunately centralization also means that messages can be intercepted and stored in this central place, usage behaviors can be analyzed and unwanted adds and other things can be injected into the message stream. Also, the centralized server is aware of the IP addresses of all clients connected to it so location profiling is also possible.

Conversations on the other hand is an XMPP client. XMPP is a federated system of many independent servers on the Internet that can communicate with each other. Users are registered with individual servers and there is no central instance, which is why it is called a 'federated' system. If a user who has an account on server A wants to send a message to a user with an account on server B, server A and B communicate with each other to exchange the user's message. To make user IDs unique it looks like an email address, i.e. there's a username, an '@' sign and a server name.

In practice there's two ways to use the system. The simpler case is to create an XMPP account, sometimes also called a 'Jabber' account, on one of the community operated and open XMPP servers. And there are quite many of them. Its no problem if one wants to communicate with a user with an account somewhere else as servers communicate with each other. The other way to use the XMPP system is to run an XMPP server at home, e.g. on a Raspberry Pi. That's obviously a bit more work but if one has previously installed other servers at home such as Owncloud it's not too difficult to do. If the server at home should be part of the overall XMPP federation rather than just serving its own users, it has to be reachable via an official domain name. A dynamic DNS address suffices.

Just like email, really, just instant 🙂

Earlier this year I had a post about how a simulated floppy drive that reads SD cards helped me to experience long gone times on a real Commodore C64 rather than on a simulator. The C64 wasn't the only computer I had during my teenage years, it was followed by an Amiga 500 before PC clones and Microsoft Windows started to dominate home and office computing. Like with the C64, my Amiga vintage adventures are made difficult by software I once had but long lost and aging floppy drives and disks. But like for the C64 there are floppy drive hardware simulators available that plug right into the floppy cable and power supply of the original Amigas.

Instead of 3.5 inch floppy disks, the simulator is fed with USB Flash sticks that can contain an almost infinite number of floppy disk files. Switching from one floppy to another is done by pressing buttons at the front of the drive. The replacement drive I got for me even simulates the sound of the drive head motor of a real floppy drive for a more realistic feeling. The only difficulty I had was that the hardware doesn't fit into the Amgia 500 case as the USB port and the keys are at the wrong location and would be covered by the external case if assembled as intended.

While most people seem to use a longer ribbon cable and extend the power cable to use the floppy simulator outside the case or modify the case to get access to the USB port and the keys I came up with a slightly different approach which is to use a USB extension cable to drag out the USB port through the opening for the drive in the case and put the floppy simulator into the case itself. The disadvantage is that the drive can't be fixed in the case and a pen has to be used to reach the keys to simulate a change of the floppy disks. Not ideal, but to me that's much better than modifying the original case or have an "ugly" ribbon cable and a grey box outside the case.

P.S.: This is a good opportunity to link to my previous post about Amiga 30th anniversary events as the event in Neuss Germany is still outstanding and scheduled for the 10th of October 2015.

As a privacy conscious person it goes without saying that I don't use centralized instant messaging solutions such as WhatsApp. Over the years I played around with a number of XMPP based mobile messaging apps but I found all of them lacking one thing or the other. The usual pain points were a combination of ugly user interface, unreliability, no indication if a message was received or not, painful picture messaging, lack of end-to-end encryption, high power consumption, etc. etc. Then during the recent Froscon conference I attended Daniel Gultsch's talk about XMPP mobile messaging and was completely blown away as he addressed all my issues during his talk and had a solution that sounded to good to be true: The solution is his Android Conversations app and its motto is 'the very last word in instant messaging'. After a week of using it I could not agree more! I'm not easily impressed but this app does it and here's why:

The User Interface

In short, the user interface is beautiful and easy to use but still offers a lot of options in the the advanced configuration settings for people like me. A picture says more than a thousand words so click on the screenshot on the left for a closer look.

Getting a Jabber / XMPP Account For Conversations

Even non-technical people can install and configure Conversations, as only a username and a password of an XMPP account are required. There are lots of open XMPP servers around that interconnect with each other so it's easy to get an account or to create one for another person and to simply text the username and password to them. Servers can have different policies of how to create an account which is the major hurdle in the process. The Chaos Computer Club Jabber server for example asks the user during account creation to go to a website, get the contents of a captcha and type it in. Conversations isn't capable of doing that but Pidgin, my desktop IM messenger, had no problem with that so I first registered and account there and then used the username and password on my mobile device.

Delivery Indications and Transport Reliability

One thing that I find very important to have is an indication if a message has been delivered, if delivery is still pending or if it has been lost somewhere on the way. Conversations does a great job at this because it shows a green check mark next to a message when it has been delivered. In addition it shows up to which message the recipient has read the message stream.

Many mobile messaging apps I've tried so far sooner or later failed when it comes to reliably transport messages. Some apps drop messages when sent to an offline contact, or, even more often, when contact is online but currently out of coverage. Conversations handles both scenarios very nicely. For out of coverage scenarios a bit of help is required from the server side about which I'll write a separate post as that's something quite interesting in its own right. But servers usually support this today and I couldn't get Conversations to drop even a single message in normal out of coverage scenarios. When I tried very hard, and I mean really hard, I could get a message to drop but Conversations then indicates this very clearly. Also, sending messages to offline contacts just works even if the recipient only comes online hours after the message was sent. Once the message is delivered the message is marked with a delivery indication so the user is aware at all times if the message is still pending or if it was delivered. Excellent!

Pictures, Voice Messages and Files

Text messages are so yesterday, today people want to send pictures and voice messages. Conversations can do both and pictures are automatically downsized to a couple of hundred kilobytes unless they are sent as regular files. It's also possible to take a picture from within Conversations and send the result without going through the gallery. Like pictures, voice messages can also be recorded within Conversations and sent directly. If supported by the server, content is sent via a proxy server. If a server doesn't have a proxy, files are cut up into several messages and sent inline. I tried both methods and both transfer files quickly.

Multi-User Chat

In addition to 1:1 messages, Conversations also supports group chats if the server offers the service, which most do. Creating a new conference, i.e. a multi-user group, and adding participants works swiftly in Conversations and the only feature I found lacking so far is the ability to send pictures into a group chat. I guess that is important for some people so let's hope that is added at some point.

Encryption

And finally, let's have a look at encryption. Conversations supports the standard OTR (Off The Record) end-to-end encryption, which I've been using with my desktop IM program for a long time already and also OpenPGP, which I didn't try, however. The major problem of OTR is, however, that it's not very well suited for mobile communication that is at times unreliable due to users suddenly moving out of coverage for a while and it's limited to 1:1 communications, i.e. it can't be used in a group chat. Therefore, Conversations implements an additional and new protocol called OMEMO that addresses both issues. While it works much better than OTR when I was testing it in a number of different out of coverage scenarios, it should be mentioned that it is still brand new and from a security point of view I wouldn't trust it very much yet until it has been reviewed and security tested independently. Also, I'm not sure if pictures and voice messages or OTR, OpenPGP or OMEMO encrypted when they are sent but at least they are protected by the SSL encryption of the transport layer to and from the server. [Update: The author has since confirmed that file and picture transfers are encrypted if one of the encryption settings are used for communicating in a 1:1 chat]. In the advanced options, Conversations has a tick box to not to trust the system certificates. In other words, if activated, Conversations asks the user if he wants to trust even a fully valid and trusted server certificate and then warns the user whenever the certificate changes. Man-in-the-middle attacks by organizations that can generate a valid SSL certificate on the fly and sit in between the client and the server are not only prevented but the user is even warned that this is happening. This is certainly not a feature for the average user, which is why it has to be activated in the advanced options menu but it shows how much thought has gone into Conversations from the GUI right down into the machine room…

Summary

I've been using Conversations for a week now and it didn't crash on me a single time, didn't loose messages, showed me when they were received, it works great for pictures and voice messages, too, has a slick interface and the security features are first class as well. So by all means, give it a try if you like mobile messaging but are concerned about your privacy and storage of all your data on a server far far away.

Every now and then it would be great to have a full operating system on a tablet rather than Android or another mobile optimized and castrated replacement that is just too inflexible for my taste. From some reports I've seen over time I was aware that Ubuntu has some sort of touch screen and on-screen keyboard integration but little is written about it. Recently, I came across the Lenovo Yoga 3 – 11" that has a 360 degrees rotatable keyboard and a touch screen. It can be used as a Laptop, as a screen only with the keyboard as a stand and in a full "tablet" like mode with the screen rotated 360 degrees so it rests on the backside of the keyboard. That sounded interesting and it's price tag of €599 fit my idea of what I would pay for such a device. But would it run Linux in a way that the device is usable in a non-keyboard mode? I tried hard to find more information on the net but came up pretty much empty handed. So I decided to give it a try myself as some of the stuff that I found at least indicated that it could work. After about a week I'm ready to give you an account of how it went from me, so here we go. It's going to be one of those long blog entries so if you only care for the summary, jump right to the end the post. Otherwise just read on for the nitty-gritty details of my adventure.

The Hardware

Let's have a look at the hardware first. Apart from it's 360 degrees rotatable keyboard the Yoga 3 – 11 has a screen resolution of 1900×1080 pixels. On an 11 inch display that almost looks "Retina" like. Further, my device came with 4GB of RAM, 128 GB SSD, 1 USB 3 port and 2 USB 2 ports of which one doubles for charging with a special connector. Other configurations with more RAM and a bigger SSD are available as well. To make it light and fanless (!) Lenovo uses an up to date Intel Broadwell dual core multi threading "Core M-5Y10c" that runs at 0.8 GHz with bursts up to 2+ GHz. For further details have a look at this detailed review over at notebookcheck.com.

Installing Ubuntu 15.04

As the processor and the device in general is pretty new and even the review linked to above is only from April 2015 I decided not to install the current Ubuntu LTS (14.04) but take the latest version of Ubuntu, which currently is 15.04, with the latest supported Kernel of that version, which is Linux 3.19. Before installing, I booted Clonezilla from a USB stick after disabling secure boot in the BIOS to make a backup of the existing and entirely unused and not even booted once Windows installation on the SSD, just in case. The Yoga 3 has a small key on the side of the device that needs to be pressed with a pen to start the BIOS settings when the device is off. It's a little bit unusual but I can see the advantages of the approach over frantically searching for the right function key to press to enter the BIOS settings after having switched-on the device.

While the backup was running I downloaded a copy of Ubuntu 15.04 and put it on a really fast USB 3 Flash memory stick. I then booted Ubuntu 15.04 from the memory stick and as it's a fast stick it booted to the desktop in just a couple of seconds. I had a quick check of the suspend and resume functionality since this is a critical thing for me and it was working. However that's just a first indication that guarantees nothing as we shall see later. One thing that wouldn't work out of the box is the on-board Wi-Fi but I was already aware of this and was reasonably certain I could make that work later on. Consequently I used an Ethernet to USB adapter so 3rd party add-ons and OS updates would be downloaded automatically during the installation process. Absolutely breathtaking, a full Ubuntu 15.04 installation from the fast USB Flash stick including the download of several hundred megabytes of 3rd party add-ons and patches took less than 10 minutes. Yes, 10 minutes!

First Steps in Ubuntu 15.04

Booting Ubuntu on the Yoga 3 with it's SSD takes around 10 seconds. I didn't time it but it's just as fast as on my other SSD based devices. Next I had a closer look at the Wi-Fi card and quickly found a driver that can be downloaded and installed with a couple of commands. DKMS makes sure it's automatically included when a kernel update is done during a normal update process. Thanks to Adam Lee who seems to work for the Ubuntu/Canonical hardware enablement team for the terrific work!

Keyboard and Touchpad

Concerning usability, the keyboard's size is pretty standard so 10 finger typing works well. Due to the limited height of the device, however, the key-lift is quite small and typing is not as comfortable as on a more standard keyboard. The Yoga 3 shares this disadvantages which other devices in its class. As I'm German, my device came with a German keyboard layout and unfortunately, the ' key has been moved away from its normal place to somewhere above the enter key. While for German text this is not a problem, it's a bit of a hassle when typing English text. Also inconvenient is the missing key at the top right to jump to the beginning of the current line, something I use quite often. And yet another inconvenience for me is that by default the overlays of the function keys to adjust sound volume, brightness etc. are reachable without a modification key. This can be changed in the setup but if done, the functions are not usable anymore in Ubuntu. So I have to chose between two evils. I chose to give up the F-keys, which work in combination with the ALT-modification key but I don't think I can get used to pressing ALT on this keyboard while not pressing ALT on others. And while we are on the input side, the touchpad works fine, also for 2-finger multi-touch do drag web pages and other things up and down the screen without the clicking on the slider on the right side of windows. Also, as a Thinkpad user I would prefer the touchpad keys at the top close to the keys rather than at the edge. Again, a personal preference.

Processor Speed and Video Playback

While the Yoga 3 11", and I keep pointing out the 11 inch as there are other variants available as well, especially the Yoga 3 Pro, which is a completely different device, feels very snappy overall the Core M processor is actually quite slow. The SunSpider 1.1 test takes 630 ms to complete, which is three times longer than what my 2 year old i3 based Thinkpad takes to execute the performance test. But I didn't notice this at all in day to day usage at all as other components, especially the GPU help a lot to make web browsing, word processing and other non-processing intensive activities very smooth. Full screen HD content on Youtube plays smoothly but I had some problems with Netflix in the Google Chrome browser which often showed shear when the camera moved sideways. Strangely enough, video quality was absolutely o.k. when the video played in a window, even if the window was extended to the full screen size except for Ubuntu's and Chromes status and window bars.

Touch Screen, Multi-touch and Rotation

One of the main questions I had before running this experiment was if and how well Ubuntu supports touch screens and devices without physical keyboards, i.e. with the Yoga 3 in video or tablet configuration when the hardware keyboard is of little use and even deactivated in full tablet mode to prevent unintended key presses. It turned out that Ubuntu 15.04 supported the touch screen very well and the on screen keyboard functionality is quite intelligent as well as it can be configured only to show the keyboard when text input is expected. Once enter is pressed the keyboard fades away until the user again taps on a window that waits for keyboard input. Also the on screen keyboard recognizes when the hardware keyboard is used again and doesn't pop-up automatically anymore. And finally, the on-screen keyboard can be resized and moved and even moves by itself when input is expected by a window the user clicks on that would be under the keyboard. Well done!

One thing that doesn't work is automatic rotation of the screen when the user flips the Yoga 3 in tablet mode from landscape to portrait mode. Shell commands launched via icons can do the trick, however. It's a bit of a kludge but something I can live with. Ubuntu also recognizes 2 finger multi-touch on the screen but it depends on programs what they do with it. By default Firefox doesn't do anything with it but there's a nice add-on that adds the functionality. After that, scrolling up and down on a web page works like one expects on a tablet.

Built in Camera and Video Telephony

Another important application for me is video telephony. Ubuntu supports the camera out of the box and Skype uses the camera without any further configuration. Video calls are smooth and the image on both sides is good, nothing there to complain about. The audio connector works as well but I only tried with headset without a microphone so I'm not sure if that's supported as well. There are volume hardware keys on the side of the Yoga 3 11" as well which work as they are supposed to and are probably most useful in tablet mode.

Suspend/Resume

So far, so good. Next I gave the suspend and resume functionality a closer look. Unfortunately I noticed that every now and then, a resume would end up with broken fonts being displayed and used thereafter until a reboot. I tried if installing 4.0 and 4.1 kernels would fix the issue but unfortunately it didn't. It seems to happen quite randomly as sometimes I would be able to suspend/resume 20 times before I saw it happening while after some reboots it would appear already after the third or fourth time. Also, the screen remained black sometimes while I could still switch to the text consoles via the function keys or the system froze completely while resuming. That's real deal breaker so I invested a lot of time to see if anything can be done about it. I search forums up and down and even installed Ubuntu 14.04 LTS to see if the issue would go away. It didn't and after a couple of days I had to admit defeat on that front.

Strange LibreOffice Behavior

Another think that kept me puzzled over many hours was LibreOffice's behavior. While I was editing a spreachsheet that had lots of cells with a lot of formatted text inside, Calc would either crash on me entierly after a while or it would suddenly become very slow when moving to a cell with a lot of text with the cursor keys. When looking at memory consumption in 'htop' I noticed that when this happened, Calc started using enormous amounts of memory each time I went into such a cell. At some point when calc became almost unresponsive it had accumulated almost 2 GB of RAM. I save the file and tried on another Ubuntu 15.04 installation with the same LibreOffice version but I could neither replicate the crashes nor the memory consumption there. Again I spent several hours going after this and could finally fix it by installing the brand new LibreOffice 5.0. But I could never find out why Libreoffice 4.4 behaved so strangely on this machine while it was just working fine on another.

Swap Partition always gone in Ubuntu 14.04

And while we are talking about the things that just don't make sense, I couldn't get the encrypted swap partition working when I installed Ubuntu 14.04. After every reboot the swap partition was no longer recognized. I could recreate it manually in 'gparted' and adapt the 'fstab' and 'crypttab' files but after a reboot it was again not recognized. Again something I've never experienced on any other PC I've installed Ubuntu on and it have been quite a few.

Summary

Despite the less then ideal hardware keyboard and touchpad, which are the only negative points for me on the hardware side, I really fell in love with this machine. I found Ubuntu very usable in tablet and video stand mode, the on-screen keyboard and touch-screen support is great. Apart from the strange Libreoffice issues that I could fix by installing version 5.0, all programs I use on a daily basis ran well despite the slow Core M processor and I did not notice a speed difference compared to using them on my i3 based Lenovo E330 laptop. The deal breaker unfortunately were the issues around suspend/resume. I don't like defeat but after dedicating several evenings to the problem I decided it was not worth further effort. As it doesn't happen too often I decided to live with it and hope that things will be fixed in a future kernel version.

After coming back from vacation and having had some time before my next trip I decided it was a good time to upgrade my Samsung Galaxy S4 from CyanogenMod Android 4.4.4 (Kitkat) to 5.1.1 (Lollipop) mostly because of the Stagefright security issues discovered in the weeks before. While that fixed the security issues I was actually positively surprised about a number of new and different things on Lollipop (i.e. CyanogenMod 12.1). Read on for the details.

First of all, the eye candy has gotten slicker again. Things roll in and out of the screen into different directions and animations are much smoother. Despite the added animation complexity there's a noticeable improvement of the calendar entries being presented in a widget on the start screen. Instead of taking 15 second they appear immediately.

Also, program startup times have been much reduced and, what I've been asking for a long time, apps are no longer terminated in the background after some time of non-use or when bigger apps such as Osmand are started in the foreground. Also, switching between running apps with the new task switching functionality that shows running apps like they were in a Rolodex is much better than on Kitkat. Another thing that has gotten attention was the startup time of the camera app. While it usually took several seconds before, it's now almost instantaneous.

Speaking about Osmand, the GUI of the current version that was released independently of my OS upgrade has become much slicker. It took a bit of time to get accustomed to where I find different functions as they look a bit different now but overall I like the user interface changes.

Further on the positive list, a '#' sign is now shown in the status bar if an app is running with root rights. Speaking of the status bar, I like the new icon design and notification handling both in locked and unlocked state and the ability to completely block notifications of apps that I don't want or only ban them from the lock screen.

And last but not least, my network blocking scripts to remove obnoxious advertisement in web pages and the ability to disable GSM for LTE/UMTS only access still work. Excellent, both are must-have features for me.

Unfortunately there are a number of major issues with the CM 12.1 nightly development stream as well: Nightlies distributed in the middle of August for the S4 had a major problem with the GPS chip and my device couldn't get a fix at all or took several minutes. This seems to have been fixed in late August but the current nightlies keep soft resetting the GUI several times a day on my device. I tried an S5 for comparison and had the same experience with soft resets. I guess some more work needs to go into CM 12.1 before it can be declared reasonably stable and usable. A bit of a pity, I'd really like to use it now 🙂

The second issue I have noticed is power consumption. I can't yet put my finger on it but it seems that while power consumption is the same as before when the phone is locked, significantly more power is required with CM 12.1 while the device is used.

Well, I guess you can't have it all, but the broken GPS functionality and regular GUI resets are a showstopper for me at the moment. Fortunately the Galaxy S4 CM maintainer has now issued CM 11 nightlies with the Stagefright fix as well. So let's see, hopefully that's more stable.

Not so easy I'm afraid…

Every now and then I flash CyanogenMod on an Android device and as there are usually weeks or months between two sessions I frequently have to look up the instructions again. To complicate matters, the procedure for flashing an alternative Android on a device also depends on the device manufacturer. Altogether a very confusing procedure if you don't do it all the time. But things get much simpler once one has understood what lies at the core of the different procedures and all the steps required.

The Recovery Partition

No matter how different the procedures are, the core is always the same: When Android starts, the bootloader on any device has several choices it can make. Usually, the bootloader just boots Android. Some manufacturers have included a proprietary flashing software that the bootloader can run instead of starting Android when the user presses a certain key combination during power up. This is proprietary, however, and is not directly related to Android. And finally, the bootloader can also start the software in Android's recovery partition. And this recovery partition is what is used on all devices to install an alternative ROM from a zip file that contains the complete Android installation.

Software For The Recovery Partition

So the first trick is to get an alternative recovery software on the recovery partition. A well known alternative recovery software is Clockworkmod (CWM).  This is the key in all flashing procedures I have come across so far. Exactly how this is can be achieved depends on the device and manufacturer and procedures vary wildly. I'll give a couple of examples below. For now, the key thing to remember is that an alternative recovery software such as CWM has to end up in the recovery partition in some way.

Getting The Alternative Android Software On The Device

Once the recovery software is on the recovery partition it can then be used to flash the alternative Android system into the system partition. Before that is done the device has to be wiped with the recovery software as configuration settings of the 'old' Android system might not be compatible with the new system. The second big task in flashing a new Android system is to get the zip file that contains the new Android system into the device so the recovery software can flash it. The easiest variant is to put it on a removable memory card. Another variant some devices support is to upload the ZIP file to the data partition of the device over USB while it is in recovery mode. One way or another, once the Android system ZIP file is on the device the alternative recovery software (e.g. CWM) is then used to install it. In the case of CyanogenMod I usually also transfer a second ZIP file to the device that contains a part of the Google software to be able to access the Play store. Transferring and flashing the second ZIP file works the same way as for the main Android system ZIP file.

So Why Are The Procedures Different For Different Device Manufacturers?

So far so good. In practice, however, different manufacturers have chosen different ways to configure the different Android storage partitions and startup procedures. Also, some companies such as Sony, for example, lock the bootloader and won't allow an alternative software on the recovery partition before the bootloader is unlocked. This is done via a code that can be obtained from their unlock website. The Samsung devices I have flashed so far do not require this step, their boot loaders are open. However, there seem to be some US network operators that have the bootloader of their device variants locked. On Sony devices the recovery software has to be sideloaded via Android debug tools. On Samsung devices, Samsung specific tools are required. And, not to forget, the key combinations to make the phone boot into the flashing software in Samsung's case or the recovery partition are also different on a device and manufacturer basis. Just like the keys to enter a PC's BIOS settings…

But in the end, all these different procedures have one goal: To get an alternative recovery software on Android's recovery partition. Keeping that in mind and the confusing multistage instructions of how to do that might seem a little bit less confusing.

 

Wow, I'm flabbergasted, did you know that the BIOS can contain Windows executables that are run at every Windows system startup that the user can't delete and that even won't go away when you re-install Windows unless the feature is deactivated in the BIOS before re-installing?

I didn't until I read this article in Ars security about a well known company that misused the 'feature' that is supposed to help with wipe-after-theft and other security features to deploy un-deletable crapware to buyers of their machines. Another one of those 'the road to hell is paved with good intentions' features that was hijacked. Am I glad that I'm running a Linux OS on all my machines whose developer's would never even dream of supporting such a dangerous BIOS behavior.