Category: Uncategorized

This post is part 3 of the series that looks at how Android based devices interact with Google in the background. In part one, I've been analyzing what an Android device does if the user gives the device his Google login credentials and otherwise leaves the settings as they are. Part 2 then looked at what the ordinary user can do to reduce the exchange of data with Google. But even with all options turned off via the user interface there is still some interaction going on through encrypted connections. While encryption is obviously necessary to prevent eavesdropping it also makes it impossible to see from the outside what kind of data is exchanged. So I was wondering if there is a way to stop the device from talking to Google completely.

And indeed there is away. During my research I noticed that like most other Internet based services, Google uses the Domain Name Service (DNS) to resolve domain names such as www.google.com to IP addresses that the applications such as Google Talk, the Android market, maps, calendar and address book synch, etc. use to talk to servers in the cloud. In practice, name resolution comes into play when a program opens a connection to a server with a domain name. Before the server is contacted, the OS first sends a request to the DNS server in the network to get the IP address of the application server. By tapping into this process and giving the application the local loopback address instead of the IP address of the server, communication can be stopped. Obviously this should only happen for certain domain names as otherwise web browsing and other services would stop working as well.

So how can the local loopback address be returned for certain domain names? If you are in control of the DNS server that is used for a connection then it's possible to control it on the external server. In most cases, however, there's no way to control the external DNS server because for cellular connections, Android does not offer the possibility to specify a DNS server manually, i.e. the network operator chooses which DNS server to use.

The second possibility is to interact with the DNS resolver on the device directly. The Android DNS resolver, as it is based on Linux, always queries a file called "hosts" in the /etc directory for local name resolution before it queries an external server. Usually the file only contains one entry:

127.0.0.1           loopback

By adding additional entries for the domain name of Google services, communication can be prevented. Here's an example:

127.0.0.1     android.clients.google.com
127.0.0.1     mtalk.google.com
127.0.0.1     www.google.com

Depending on the manufacturer additional lines are necessary to stop the phone talking to HTC, Samsung, LG as well.

The problem with the hosts file is that it is located in a protected area so the device has to be rooted first. How this is done depends on the model and the maker of the device. Once done and after installing a terminal program such as "Terminal Emulator" to get to a shell, the final obstacle is that the partition the /etc directory is located is mounted as read only. So before the file can be changed the partition has to be remounted as writable. Here's the shell command to do that for a Samsung Galaxy S:

mount -oremount,rw /dev/block/st19 /system

Other devices might have the /etc directory mounted somewhere else which can be found out by using the mount command without any options.

It takes quite some effort to stop the conversation of background services but depending on your privacy needs it's an effort well worth taking. Every now and then, however, even I would like to use a Google service such as maps, I just don't want my device to exchange data with Google all the time. To do that the lines added to the hosts file have to be removed again (after making sure address book and calendar synch is still disabled in the settings). Perhaps that is something that should be automated…

You might have noticed that every now and then I not only discuss mobile related things on the blog but also security related thoughts, which usually have a mobile edge as well. In that regard I very much enjoy listening to the weekly "Security Now" podcast with Steve Gibson and Leo Laporte which recently pointed me to "Zero Day", a novel by Mark Russinovich. So far I was always very disappointed by authors putting computer and security related issues in a novel as they were not deep enough into the technology to describe things realistically. Quite a different story here. I can't remember when I last read a contemporary 300+ page novel in less than a week. This one is a first in many years and thus highly recommended. The link above leads to Amazon where you can find more details. No need to repeat them here.

In the past couple of days I've been discussing the idea of rooting an Android phone with a couple of friends in the industry. At some point, more than one of them said "it's like jailbreaking the…". That's interesting because even many knowledgable people in the industry do not see the difference between rooting an Android phone and a jailbreak.

So here's my opinion in this: With most network operators (exceptions prove the rule…) Android phones are open. You can install the apps you want, via Google's Android market, Amazon's market, GetJar, etc. Also, people can directly install apps they have downloaded to the memory card. Also, there are no restrictions as to which apps can run on the device and which not (again, exceptions prove the rule). In other words, these phones are free (as in free speech) so there is no need to "jailbreak" them.

So what's "rooting" then? Android has a security framework in place to prevent apps from doing malicious things. The first line of defense is during installation when the app has to present the list of required permissions to the user (e.g. for network access, for reading location information, etc.) so he can decide if he wants to install the app or not. Once the user gives the app the required permissions it is installed and can then be started. Android then insures the app does does not go beyond the defined security context.

But no matter how many permissions the user gives to an app there are some things that are never allowed such as for example, writing to a protected part of the filesystem that belongs to the operating system only. This is potentially dangerous as changing certain files in the system breaks the phone. This is the same concept that is also in desktop operating systems such as Windows, Linux and MacOS. Still, for power users there are good reasons to have access to restricted parts of the operating system and this is where "rooting" comes in.

Android is based on Linux and if on a Linux user wants to do something that only the root user can do he requests "root rights" and typing in a password. On Android, an app by default can't acquire root rights for security reasons. "Rooting" an Android phone makes this possible, i.e. the app can ask the operating system to get root rights and thus be able to do things that ordinary apps are restricted from doing.

Jailbreaking on the other hand is a concept required on devices where the user is severly restricted by the concept of the device, e.g. a jailbreak is already necessary to install apps from places other than the official market. In other words, jailbreaking is a method to circumvent a closed garden system while rooting is a concept in open garden systems (such as Android) for apps to get root level access to the system.

Obviously the main feature of my first self written Android App is to do some drive testing to see how many cells are out there and how good the network coverage in general is on a particular stretch. So in the past couple of days I took my apps for drive testing to see how many cells cover my way from and to work.

The distance between my home and work is 38 km. To see if there are differences in the number of 2G and 3G cells I did the exercise a couple of times to get the number for each network technology. And to see if there are differences between networks I ran the exercise with SIM cards of two network operators. As a result I could see around 63 cells on my way to work, both in GSM and UMTS.In other words the network density of 2G and 3G seems to be quite similar, at least in the part of the world I live.

Also there hasn't been a significant difference in the number of cells between the two networks so from a capacity and coverage point of view the networks are very similar. Perhaps I should run the exercise again with the other two networks as their marketing and budget approaches might be different.

The 64 cells and 38 km distance can be translated into 38 km/63 cells = 600m/cell. In other words, each cells covers on average 600m of my way to work. On my trip I go through cities and also a few kilometers through the countryside so it's likely the cells in the cities are a bit smaller while the cells outside are a bit larger.

This does not mean, however, that there is a cell tower every 600 meters as a base station usually has three sectorized cells each covering 180 120 degrees. So the average distance from one cell tower to another is somewhere between this value and and its double. It's somewhere in between as I imagine that while the trip covers two sectors of some base stations, it will only touch a single sector of others.

Ever wondered if your Wi-Fi Access Point at home is already in Google's Wifi location database? If so, have a look at this web page where you can type in the BSSID/MAC address of your access point. If present in the database, the location will then be shown on a map. With most Android devices collecting location information on the go, I wonder how long it takes until a new Wi-Fi access point shows up in the database!? Hm, an interesting experiment…

Note: The MAC address to be typed in here is usually not the MAC address that you see in Wireshark if you communicate with the web front end of your access point but a separate ID that you either have to find in the configuration menu of the router or if by using Wireshark in Wi-Fi sniffing mode. This exposes the real Wi-Fi MAC header that contains the source, destination and the BSSID MAC addresses instead of only source and destination.

In a previous post I have started looking at with whom and Android based device communicates in the background after it has been switched on. Unsurprisingly, it starts talking with a couple of Google services as detailed in part 1. Naturally, I was wondering if and how that can be reduced again using my private Wi-Fi access point trace setup.

Google Talk

Of particular interest in this regards was first of all Google Talk. In the default configuration, my Android phone automatically registers me to Google Talk after I power on the mobile phone and lists me as available. Further, as part of the default configuration, it updates my Google Talk status as "away" whenever the screen saver activates and again back to "available" every time the screen saver is disabled. In other words, an interaction with Google whenever I use the phone so Google could potentially use the data to track when and how often I use my phone. This behavior can be switched off by starting Google Talk and disabling it in the settings. It's also possible to disable the "auto sign-in on startup of the device" and I did that as well because I don't use Google talk at all. Despite this, my device still starts talking to Google Talk when I power it on. Not sure why that is, the communication is encrypted, so it's not possible to tell. But the question remains, why does it contact Google Talk if I don't want an auto log-on?

Android.Clients.Google.Com

Another interaction that can be reduced is the initial interaction with http://android.clients.google.com when the device starts. By changing the password of the account that is used for the Android market, calendar synch, address book synch, Google Talk, etc., the initial interaction will fail and the device remains quite silent afterward. Even Google Talk remains silent and doesn't attempt to establish a connection to the server. Changing the password of the Google account is best done on the PC by logging in there and then going to the settings page.

Using a different Browser

And another thing that can be done to reduce interaction with the mother ship is to use a web browser other than the built-in one. There are several options such as Opera Mobile and Opera Mini. Opera Mini is a proxy based solution so in theory there could be no correlation between the device and the IP address that anyone could track. Unfortunately, that is not quite the case as the Opera Proxy Server includes the original IP address the request was sent from in the HTTP header in an extension. Not so good for privacy unfortunately. But better than nothing.

Summary

Please don't get me wrong, I have come to very much appreciate Android, it's become a great and flexible mobile OS and its capabilities are stunning. It's open source, it's easy to find the code and I'll rant about that in another post but it's a bit too chatty for my taste with Google. Perhaps if I knew what was going on inside those encrypted connections (they have to be for security reasons obviously) I would feel a bit better but I don't have any insight into that.

So if you apply all the things I've written in this post and the previous one you can quite narrow down the interaction but you can't stop it completely this way. If you want to use the Android market, you have to input account details after which the device starts to become chatty. Perhaps the Amazon market and GetJar are replacements that are worth looking at from a privacy point of view? Also, turning off location services is not such a good idea if you want to use Google Maps. Without the option to report the location of Wi-Fi access points occasionally, the functionality is also disabled in Google Maps and usability is greatly diminished as only GPS is used at this point and acquiring a first location fix takes a bit of time. You can of course switch background location reporting off and on as you need it but that's cumbersome. Yes, an ease of usability vs. privacy tradeoff…

Every now and then you can read in the press that the different Android versions out there in the wild and different implementations are a challenge for programmers. Few if any of these reports, however, give specific examples. So how big is the issue really? I can’t give a definitive answer, either, but in the course of putting together my cell logger program I encountered two differences across devices and Android versions that really shouldn’t be there.

The first difference is something I can’t do anything about. In theory, there are standardized handler functions that can be used to get notifications of signal strength changes and standardized methods for retrieving cell-id and other network related data they don’t work in the same way across devices. The signal strength change notification failed to work completely on one device. I would only get one notification when the program started with a completely wrong signal strength value. Afterward the handler was never called again. On all other devices, the behavior was as desired and signal strength changes were always reported as they happened. Strange.

When it comes neighboring cell information there is also a great variety of implementations. On some devices I get full neighboring cell information for GSM while nothing at all for UMTS. On other devices, even GSM neighboring information is missing. This issues might stem from a different implementation of the Radio Interface Layer (RIL) which is in the hands of the device manufacturer and not Google itself.

The second thing, which I was able to fix, is the way registered handlers are treated when the app goes to the background. One most devices, the handler gets removed automatically and has to be put in place again once the app comes to the foreground again. On another device, the handler was not removed and thus my app coming to the foreground again would install another one, resulting in several copies of the data in the log file. The issue can be fixed by removing the registered handler before the app goes to the background which works on all devices I have tested it on. But why the difference? Is it an Android version issue or did the manufacturer meddle with something? I can’t say.

In summary, I can confirm the reports that an Android device is not equal to another Android device of a different manufacturer from an app point of view. My examples might be benign but if I already stumbled over those two things with my 300 line program, there is likely to be more divergence. So one has to be a bit careful and test the app across a number of different models before it is released. Not what a programmer really wants to do…

If you are into Android programming, have you encountered similar things as well?

The first version of my Android experimental cell logger app is ready so it's time to publish it. I've put the source under the GNU public license so feel free to experiment with it. As you are reading this post the easiest way to get the app and the source is as follows:

• For convenience you can pick it up in the Android market. Just search for "cell logger" and you'll find the app with my name as publisher. Or, just click here.
• The bundled .apk file for installation. If you've come here with your Android device, click on the link and it will install right onto your device. If you've never downloaded an app outside the Android market you'll get an error message saying you have to allow installation of apps from unknown sources. Once done the app will install.
• And here's the source code. It's the full Eclipse development folder so you'll find everything inside to compile and run it from there.

So what does it take to publish an app on the market and how difficult it is? You can find the detailed version from a programming perspective here. In essence pretty much everything for publication is already done when you compile the app into an ".apk" file and use a real set of self generated signature keys. These are a precondition anyway if the app runs on anything but the Android simulator. So this step was already done as well. In addition, the market requires two screen shots and a 512×512 pixel logo. I decided to generate the logo out of one of the screenshots by cropping and resizing. It's a research project so I decided that this will do for the moment.

From an administration point of view first thing that is required to publish the app in the market is a Google account. I decided to create a new one and it just takes a couple of minutes. Once done, the next step is to create an Android market account via the publisher page of the Android market web site. Access to the market as a publisher costs $25 and probably keeps out those who just want to look. Also, you need to share credit card details and for the general Google account a mobile phone number has to be given that is verified via an SMS. And finally an email address has to be given that is also verified. In other words, everyone who publishes something in the market can be identified unless someone really goes the extra mile and uses an unregisted throw away SIM card, stolen credit card credentials and an anonymous email account.

Once the registration is done, the app is published in a few minutes. The ".apk" file has to be uploaded, a description should be entered, the 512 pixel logo and two screen shots must be added and that's it. Very simple and straight forward.

Today I wanted to add a feed to my feed-reader of a website I newly discovered but couldn’t find the RSS symbol being displayed next to the URL of the page in Firefox 4. Could it be that the web site has no RSS feed? It turned out that the page did have a feed but that the RSS symbol / button was removed from the default settings in Firefox 4. It can be added manually again by right-clicking on a free part of the navigation toolbar, selecting “customize” and then drag and drop the RSS icon into the toolbar. So I wonder what this means!? Is the discovery of new RSS feeds a function only used by a tiny fraction of users or, in other words, has reading blogs and news web sites via aggregators not really picked up?

This is the 5th and final part in my series on programming on Android. By now my app is pretty much complete and does what I set out to do: It get's notified when the signal strength of the cellular network changes and displays the signal strength, cell-id, location area code, network operator id and network operator name. In this final part I'll describe how one other important feature works, saving the gathered data in a file, and some usability features.

Obviously seeing the current LAC and Cell-ID on the screen is nice, but having a record of cell and signal changes would be nice as well for further analysis later on. In other words, the data needs to be written to a file that can then be retrieved and further processed on the PC. The best place to store the file is on the flash disk which can then be either accessed via a USB cable or by removing it and connecting it via an adapter to a PC. Here's to code that does that: 

try {
  File root = Environment.getExternalStorageDirectory();
  if (root.canWrite()){
      File logfile = new File(root, filename);
      FileWriter logwriter = new
            FileWriter(logfile, true); /* true=append */
      BufferedWriter out = new BufferedWriter(logwriter);
               
      /* now save the data buffer into the file */
      out.write(LocalFileWriteBufferStr);
      out.close();
  }
}   
catch (IOException e) {
   /* don't do anything for the moment */
}

Writing a file could go wrong in which case Java would throw and exception and terminate the program if unhandled. Hence the need for the "try" construct. In the first line I search for the directory name of the flash disk with the getExternalStorageDirectory() method which is part of the Android API. If the disk is present and writable I then open a file in append mode, i.e. if it already exists all data is appended. Then, the standard Java out.write() and out.close() methods are used to write the data to the file and then close it again.

As you can see I have opted to open the file, write something and then to close it again immediately. I've done this as it's entirely possible that the flash disk is removed while the app is running, either physically or when a USB cable is connected to the PC. To reduce the wear on the flash memory of frequent file operations, I use a cache string in memory and only write to the file after a couple of kilobytes of data has accumulated. The downside of this is that one has to be aware when the app is closed and restarted, e.g. when the user turns the device and the screen is switched from portrait to landscape or when the app goes to the background. In those cases the data has to be written to the file, too.

And finally, a couple of words on the menu of the app. There are three entries in the menu: "About", "Reset Counters" and "Toggle Debug Mode".

 The code for them looks as follows:

@Override
public boolean onCreateOptionsMenu(Menu menu) {
    menu.add(0, RESET_COUNTER, 0, "Rest Counters");
    menu.add(0, ABOUT, 0, "About");
    menu.add(0, TOGGLE_DEBUG, 0, "Toggle Debug Mode");
    return true;
}
   
@Override
public boolean onOptionsItemSelected (MenuItem item) {

    switch (item.getItemId()) {
        case RESET_COUNTER:
               
            NumberOfCellChanges = 0;
            […]
               
            return true;
              
        case ABOUT:
            AlertDialog.Builder builder = new
                   AlertDialog.Builder(this);
            […]
               
            AlertDialog alert = builder.create();
            alert.show();

            return true;

        case TOGGLE_DEBUG:
            /* Toggle the debug behavior of the program
               when the user selects this menu item */
            if (outputDebugInfo == false) {
                outputDebugInfo = true;
            }
            else {
                outputDebugInfo = false;
            }
                   
       default:
            return super.onOptionsItemSelected(item);

    }
}

There we go, the app is almost complete now. A bit more polishing and then I'll post it on the blog together with the source and publish it in the Android market for easy access.