Category: Uncategorized

Obviously the main feature of my first self written Android App is to do some drive testing to see how many cells are out there and how good the network coverage in general is on a particular stretch. So in the past couple of days I took my apps for drive testing to see how many cells cover my way from and to work.

The distance between my home and work is 38 km. To see if there are differences in the number of 2G and 3G cells I did the exercise a couple of times to get the number for each network technology. And to see if there are differences between networks I ran the exercise with SIM cards of two network operators. As a result I could see around 63 cells on my way to work, both in GSM and UMTS.In other words the network density of 2G and 3G seems to be quite similar, at least in the part of the world I live.

Also there hasn't been a significant difference in the number of cells between the two networks so from a capacity and coverage point of view the networks are very similar. Perhaps I should run the exercise again with the other two networks as their marketing and budget approaches might be different.

The 64 cells and 38 km distance can be translated into 38 km/63 cells = 600m/cell. In other words, each cells covers on average 600m of my way to work. On my trip I go through cities and also a few kilometers through the countryside so it's likely the cells in the cities are a bit smaller while the cells outside are a bit larger.

This does not mean, however, that there is a cell tower every 600 meters as a base station usually has three sectorized cells each covering 180 120 degrees. So the average distance from one cell tower to another is somewhere between this value and and its double. It's somewhere in between as I imagine that while the trip covers two sectors of some base stations, it will only touch a single sector of others.

Ever wondered if your Wi-Fi Access Point at home is already in Google's Wifi location database? If so, have a look at this web page where you can type in the BSSID/MAC address of your access point. If present in the database, the location will then be shown on a map. With most Android devices collecting location information on the go, I wonder how long it takes until a new Wi-Fi access point shows up in the database!? Hm, an interesting experiment…

Note: The MAC address to be typed in here is usually not the MAC address that you see in Wireshark if you communicate with the web front end of your access point but a separate ID that you either have to find in the configuration menu of the router or if by using Wireshark in Wi-Fi sniffing mode. This exposes the real Wi-Fi MAC header that contains the source, destination and the BSSID MAC addresses instead of only source and destination.

In a previous post I have started looking at with whom and Android based device communicates in the background after it has been switched on. Unsurprisingly, it starts talking with a couple of Google services as detailed in part 1. Naturally, I was wondering if and how that can be reduced again using my private Wi-Fi access point trace setup.

Google Talk

Of particular interest in this regards was first of all Google Talk. In the default configuration, my Android phone automatically registers me to Google Talk after I power on the mobile phone and lists me as available. Further, as part of the default configuration, it updates my Google Talk status as "away" whenever the screen saver activates and again back to "available" every time the screen saver is disabled. In other words, an interaction with Google whenever I use the phone so Google could potentially use the data to track when and how often I use my phone. This behavior can be switched off by starting Google Talk and disabling it in the settings. It's also possible to disable the "auto sign-in on startup of the device" and I did that as well because I don't use Google talk at all. Despite this, my device still starts talking to Google Talk when I power it on. Not sure why that is, the communication is encrypted, so it's not possible to tell. But the question remains, why does it contact Google Talk if I don't want an auto log-on?

Android.Clients.Google.Com

Another interaction that can be reduced is the initial interaction with http://android.clients.google.com when the device starts. By changing the password of the account that is used for the Android market, calendar synch, address book synch, Google Talk, etc., the initial interaction will fail and the device remains quite silent afterward. Even Google Talk remains silent and doesn't attempt to establish a connection to the server. Changing the password of the Google account is best done on the PC by logging in there and then going to the settings page.

Using a different Browser

And another thing that can be done to reduce interaction with the mother ship is to use a web browser other than the built-in one. There are several options such as Opera Mobile and Opera Mini. Opera Mini is a proxy based solution so in theory there could be no correlation between the device and the IP address that anyone could track. Unfortunately, that is not quite the case as the Opera Proxy Server includes the original IP address the request was sent from in the HTTP header in an extension. Not so good for privacy unfortunately. But better than nothing.

Summary

Please don't get me wrong, I have come to very much appreciate Android, it's become a great and flexible mobile OS and its capabilities are stunning. It's open source, it's easy to find the code and I'll rant about that in another post but it's a bit too chatty for my taste with Google. Perhaps if I knew what was going on inside those encrypted connections (they have to be for security reasons obviously) I would feel a bit better but I don't have any insight into that.

So if you apply all the things I've written in this post and the previous one you can quite narrow down the interaction but you can't stop it completely this way. If you want to use the Android market, you have to input account details after which the device starts to become chatty. Perhaps the Amazon market and GetJar are replacements that are worth looking at from a privacy point of view? Also, turning off location services is not such a good idea if you want to use Google Maps. Without the option to report the location of Wi-Fi access points occasionally, the functionality is also disabled in Google Maps and usability is greatly diminished as only GPS is used at this point and acquiring a first location fix takes a bit of time. You can of course switch background location reporting off and on as you need it but that's cumbersome. Yes, an ease of usability vs. privacy tradeoff…

Every now and then you can read in the press that the different Android versions out there in the wild and different implementations are a challenge for programmers. Few if any of these reports, however, give specific examples. So how big is the issue really? I can’t give a definitive answer, either, but in the course of putting together my cell logger program I encountered two differences across devices and Android versions that really shouldn’t be there.

The first difference is something I can’t do anything about. In theory, there are standardized handler functions that can be used to get notifications of signal strength changes and standardized methods for retrieving cell-id and other network related data they don’t work in the same way across devices. The signal strength change notification failed to work completely on one device. I would only get one notification when the program started with a completely wrong signal strength value. Afterward the handler was never called again. On all other devices, the behavior was as desired and signal strength changes were always reported as they happened. Strange.

When it comes neighboring cell information there is also a great variety of implementations. On some devices I get full neighboring cell information for GSM while nothing at all for UMTS. On other devices, even GSM neighboring information is missing. This issues might stem from a different implementation of the Radio Interface Layer (RIL) which is in the hands of the device manufacturer and not Google itself.

The second thing, which I was able to fix, is the way registered handlers are treated when the app goes to the background. One most devices, the handler gets removed automatically and has to be put in place again once the app comes to the foreground again. On another device, the handler was not removed and thus my app coming to the foreground again would install another one, resulting in several copies of the data in the log file. The issue can be fixed by removing the registered handler before the app goes to the background which works on all devices I have tested it on. But why the difference? Is it an Android version issue or did the manufacturer meddle with something? I can’t say.

In summary, I can confirm the reports that an Android device is not equal to another Android device of a different manufacturer from an app point of view. My examples might be benign but if I already stumbled over those two things with my 300 line program, there is likely to be more divergence. So one has to be a bit careful and test the app across a number of different models before it is released. Not what a programmer really wants to do…

If you are into Android programming, have you encountered similar things as well?

The first version of my Android experimental cell logger app is ready so it's time to publish it. I've put the source under the GNU public license so feel free to experiment with it. As you are reading this post the easiest way to get the app and the source is as follows:

• For convenience you can pick it up in the Android market. Just search for "cell logger" and you'll find the app with my name as publisher. Or, just click here.
• The bundled .apk file for installation. If you've come here with your Android device, click on the link and it will install right onto your device. If you've never downloaded an app outside the Android market you'll get an error message saying you have to allow installation of apps from unknown sources. Once done the app will install.
• And here's the source code. It's the full Eclipse development folder so you'll find everything inside to compile and run it from there.

So what does it take to publish an app on the market and how difficult it is? You can find the detailed version from a programming perspective here. In essence pretty much everything for publication is already done when you compile the app into an ".apk" file and use a real set of self generated signature keys. These are a precondition anyway if the app runs on anything but the Android simulator. So this step was already done as well. In addition, the market requires two screen shots and a 512×512 pixel logo. I decided to generate the logo out of one of the screenshots by cropping and resizing. It's a research project so I decided that this will do for the moment.

From an administration point of view first thing that is required to publish the app in the market is a Google account. I decided to create a new one and it just takes a couple of minutes. Once done, the next step is to create an Android market account via the publisher page of the Android market web site. Access to the market as a publisher costs $25 and probably keeps out those who just want to look. Also, you need to share credit card details and for the general Google account a mobile phone number has to be given that is verified via an SMS. And finally an email address has to be given that is also verified. In other words, everyone who publishes something in the market can be identified unless someone really goes the extra mile and uses an unregisted throw away SIM card, stolen credit card credentials and an anonymous email account.

Once the registration is done, the app is published in a few minutes. The ".apk" file has to be uploaded, a description should be entered, the 512 pixel logo and two screen shots must be added and that's it. Very simple and straight forward.

Today I wanted to add a feed to my feed-reader of a website I newly discovered but couldn’t find the RSS symbol being displayed next to the URL of the page in Firefox 4. Could it be that the web site has no RSS feed? It turned out that the page did have a feed but that the RSS symbol / button was removed from the default settings in Firefox 4. It can be added manually again by right-clicking on a free part of the navigation toolbar, selecting “customize” and then drag and drop the RSS icon into the toolbar. So I wonder what this means!? Is the discovery of new RSS feeds a function only used by a tiny fraction of users or, in other words, has reading blogs and news web sites via aggregators not really picked up?

This is the 5th and final part in my series on programming on Android. By now my app is pretty much complete and does what I set out to do: It get's notified when the signal strength of the cellular network changes and displays the signal strength, cell-id, location area code, network operator id and network operator name. In this final part I'll describe how one other important feature works, saving the gathered data in a file, and some usability features.

Obviously seeing the current LAC and Cell-ID on the screen is nice, but having a record of cell and signal changes would be nice as well for further analysis later on. In other words, the data needs to be written to a file that can then be retrieved and further processed on the PC. The best place to store the file is on the flash disk which can then be either accessed via a USB cable or by removing it and connecting it via an adapter to a PC. Here's to code that does that: 

try {
  File root = Environment.getExternalStorageDirectory();
  if (root.canWrite()){
      File logfile = new File(root, filename);
      FileWriter logwriter = new
            FileWriter(logfile, true); /* true=append */
      BufferedWriter out = new BufferedWriter(logwriter);
               
      /* now save the data buffer into the file */
      out.write(LocalFileWriteBufferStr);
      out.close();
  }
}   
catch (IOException e) {
   /* don't do anything for the moment */
}

Writing a file could go wrong in which case Java would throw and exception and terminate the program if unhandled. Hence the need for the "try" construct. In the first line I search for the directory name of the flash disk with the getExternalStorageDirectory() method which is part of the Android API. If the disk is present and writable I then open a file in append mode, i.e. if it already exists all data is appended. Then, the standard Java out.write() and out.close() methods are used to write the data to the file and then close it again.

As you can see I have opted to open the file, write something and then to close it again immediately. I've done this as it's entirely possible that the flash disk is removed while the app is running, either physically or when a USB cable is connected to the PC. To reduce the wear on the flash memory of frequent file operations, I use a cache string in memory and only write to the file after a couple of kilobytes of data has accumulated. The downside of this is that one has to be aware when the app is closed and restarted, e.g. when the user turns the device and the screen is switched from portrait to landscape or when the app goes to the background. In those cases the data has to be written to the file, too.

And finally, a couple of words on the menu of the app. There are three entries in the menu: "About", "Reset Counters" and "Toggle Debug Mode".

 The code for them looks as follows:

@Override
public boolean onCreateOptionsMenu(Menu menu) {
    menu.add(0, RESET_COUNTER, 0, "Rest Counters");
    menu.add(0, ABOUT, 0, "About");
    menu.add(0, TOGGLE_DEBUG, 0, "Toggle Debug Mode");
    return true;
}
   
@Override
public boolean onOptionsItemSelected (MenuItem item) {

    switch (item.getItemId()) {
        case RESET_COUNTER:
               
            NumberOfCellChanges = 0;
            […]
               
            return true;
              
        case ABOUT:
            AlertDialog.Builder builder = new
                   AlertDialog.Builder(this);
            […]
               
            AlertDialog alert = builder.create();
            alert.show();

            return true;

        case TOGGLE_DEBUG:
            /* Toggle the debug behavior of the program
               when the user selects this menu item */
            if (outputDebugInfo == false) {
                outputDebugInfo = true;
            }
            else {
                outputDebugInfo = false;
            }
                   
       default:
            return super.onOptionsItemSelected(item);

    }
}

There we go, the app is almost complete now. A bit more polishing and then I'll post it on the blog together with the source and publish it in the Android market for easy access.

Back in 2006, two network operators in Germany who so far only had GSM spectrum in the 1800 MHz range each got assigned a 5 MHz chunk of 900 MHz spectrum to improve rural coverage and urban in-house coverage. Also the move was necessary to equalize the playing field with the other two network operators who already owned 900 MHz spectrum from the beginning. It's 5 years later now so I wondered if and how they have made good use of it.

While I was recently on the countryside in Southern Germany I had a look in a number of places where 900 MHz spectrum would have been very beneficial. While O2 made consistent use of 900 MHz spectrum there in addition to some 1800 MHz carriers, E-Plus only used it very occasionally. I wouldn't have expected that as, after all, it's E-Plus who's always pointing out that they are at such a disadvantage.

The picture is even more interesting in cities. Here, 900 MHz spectrum significantly improves indoor coverage as can be observed with UMTS 2100 coverage being lost indoors much sooner than UMTS. Again, O2 has made use of their 900 MHz spectrum in Cologne in addition to their 1800 MHz spectrum. And again, no traces of a 900 MHz GSM deployment from E-Plus 5 years after they've been awarded spectrum.

On the other hand, their lack of 900 MHz deployment offers some interesting possibilities for UMTS deployment strategies. They've talked about that before in the press but only seeing is believing.

In part 3 I've looked at the entry method of my app that is called whenever the app is started and how it registers a listener object to network status events. In this part I'll take a look at the listener object itself.

The code for the listener object is part of my app and is a private class that is only visible to my app but not to the outside world. Here's a code snippet that shows how it looks like:

private class MyPhoneStateListener extends PhoneStateListener {
 
  /* Get the Signal strength from the provider,
     each time there is an update */
  @Override
  public void onSignalStrengthsChanged(SignalStrength signalStrength) {

  […]
         
  try {
    /* output signal strength value directly on canvas of
       the main activity */
    outputText = "v7, number of updates: " +
    String.valueOf(NumberOfSignalStrengthUpdates);
    NumberOfSignalStrengthUpdates += 1;
            
    outputText += "rnrnNetwork Operator: " +
                   Tel.getNetworkOperator() +
                   " "+ Tel.getNetworkOperatorName() + "rn";
    outputText += "Network Type: " +
                   String.valueOf(Tel.getNetworkType()) +
                   "rnrn";

   […] 

To keep things simple the app only uses one listener method in the listener object, "onSignalStrengthsChanges". As the name implies this method is called whenever the signal strength of the current cell changes. It is "overridden" as I don't want Android to execute the default actions of this method but I want to do my own things in my instance of the class. When this method is called I also get other network status information and compare it to previously received values. If, for example, the cell-id has changed, I increase a corresponding counter variable accordingly.

Reading network status information seems to be a dangerous thing. If network coverage is lost and the method is called for whatever reason, one of the status query methods throws an error, known as an "exception" in Java. If that exception is not handled, the application will be terminated with an error message presented to the user. To prevent that from happening, all network status querries are performed in a "try – catch" construct. If an exception occurs in the "try" part of the code, execution continues in the "catch" part where the exception can be handled. In the case of this app, the code for the "catch" part is rather short as it doesn't do anything with the exception and just hopes for a better day when the device finds the network again.

To retrieve the network status information I am interested in, I call the following methods provided by the Android API:

• TelephonyManager.getNetworkOperator()
• TelephonyManager.getNetworkOperatorName()
• TelephonyManager.getNetworkOperatorType()
• SignalStrength.getGsmSignalStrength() –> works for UMTS as well…
• GsmCellLocation.getCid()
• GsmLellLocation.getLac()

There's also objects and methods in the API to get information about neighboring cells. However, this doesn't seem to be implemented consistently accross different devices and network technologies. In fact I tried this on several different types and models and only one would give me neighboring cell information and then only for GSM but not for UMTS. Here's the code for it:

   /* Neighbor Cell Stuff */
   List<NeighboringCellInfo> nbcell = Tel.getNeighboringCellInfo ();
   outputText += "Number of Neighbors: "  +
                  String.valueOf(nbcell.size()) + "rn";
   Iterator<NeighboringCellInfo> it = nbcell.iterator();
   while (it.hasNext()) {
     outputText += String.valueOf((it.next()getCid())) + "rn";
   }

And finally, the output text I've generated needs to end up on the screen. There are several ways to do it and I've decided to go for the straight forward approach:

   /* And finally, output the generated string with all the
      info retrieved */
   TextView tv = new TextView(getApplicationContext());
   tv.setText(outputText);
   setContentView(tv);

There we go, this is the core of the app to get network status information via the Android API. I guess one more part in this series is necessary to describe some bells and whistles for smooth user interaction and of course the release of the complete code so you can play around with it as well if you like.

If you are a frequent reader of this blog you are probably aware that I prefer to keep my personal data close to myself. I don't want my personal address book, calendar, etc. etc. with Google or anyone else in the cloud, I don't want anyone collecting location information, anonymously or not without my consent and I don't want my devices to frequently call the mother ship. In this day and age, that's quite a challenge with iOS and Android sending information back into the cloud if the settings are left unchanged. So I was wondering how this data collection works in practice today and how much can be prevented from going out.

To have a closer look at what's going on I set-up a test environment with my Android 2.2 based phone connected to the Internet via Wi-Fi. The Wi-Fi access point was not directly connected to the Internet, however, but instead to the Ethernet port of a notebook that had Internet connection sharing enabled via its own Wi-Fi interface and a second Wi-Fi access point. This way, Wireshark can be used to trace the communication between the mobile device and the rest of the world. To see how Android interacts with Google and the rest of the world and how this can be scaled down I tested three scenarios:

• A device fully configured with a Google account, but calendar and address book synchronization disabled. Most people won't even deactivate this synchronization but I figured that's where most people who prefer to keep their data to themselves would start.
• As above but the Google account inaccessible due to a change of the password.
• After a factory reset without a Google account configured.

Lets take a look at the first scenario:

When the device is switched on, the first thing it does is to connect to synchronize the clock with an NTP time server (north-america.pool.ntp.org). Next, it retrieves GPS ephemeris information (xtra1.gpsonextra.net). This is interesting since gpsonextra.net is a domain registed by Qualcomm (and not Google), the maker of the radio chipset of the device.

After that, the device registers with Google (android.clients.google.com). For the interchange of most information, SSL is used so it's not possible to see what kind of data is interchanged inside the encrypted channel.

If registration with Google was successful the device then registers with Google Talk (mtalk.google.com) as Google Talk starts automatically in the background. This even happens when I configure Google Talk not to log me in automatically. I should add that I never registered to use Google Talk so I guess that comes as part of the Android/Google account.

And finally, the device starts talking to another time synchronization service (time-nw.nist.gov). Since this doesn't happen when I set the date and time to be configured manually, this interaction is probably there to check and correct the Android date and time. Again, this is not a Google service, NIST is the National (US) Institute for Standards and Technology. From that I infer that the NTP interaction above might be for the Qualcomm GPS calibration and not for Android itself.

All these interactions take place in the first few seconds after Internet connectivity has been established. After that, the device remains quiet until an app is started that interacts with a server on the network.

I wasn't able to trace an interaction with the Google location server that is reported to scan for WiFi SSIDs and Cell-IDs on a regular basis if this is not disabled in the configuration. This is most likely because my setup was stationary so the device was not able to collect information.

Put together, there is quite some interaction going on with the default settings. As mentioned above, many things can be disabled in the configuration. In a follow-up post or two I'll have a look at what can be prevented from going out and how that limits the use of the device.