OpenStreetMap (for Android) Part 4: Recording A Route And Import To OSM

While it very rarely happens where I tend to go but still every now and then there are roads and paths I take that are not in Openstreetmap. To my pleasure I recently discovered how easy it is to add them to the OSM database with the Openstreetmap for Android (OSM) GPX plugin.

The pictures below show how it's done. First, activate "Record your trips" in the Osmand 'Plugins' settings. Afterward a "GPX" button is shown in the map on the right hand side that can be tapped-on. Once tapped-on, the path that is recorded is shown in turquoise color on the map and the GPX button is replaced by the distance to the location where the GPX recording was started. To stop the recording, a tap on the distance indicator brings up a pop-up menu from which "save" can be selected (the third option in the pop-up menu, don't mix it up with 'stop' which doesn't save anything). The resulting GPX file is put into the Osmand folder in internal storage or on the SD card.

Once the GPX file is on a PC it can simply be dropped on the Openstreetmap web site after 'editing' mode has been activated. As can be seen on the last screenshot the my GPX track is exactly on the railway track of the train I was sitting in when I took the GPX recording.

Osm1 Osm2 Osm3

VoLTE: The Locality Advantage

VoLTE certainly is an 'interesting' beast and continues to occupy the minds of thousands of engineers to make it work in networks around the globe. I wonder how this monumental effort stacks up against other VoIP services such as Facetime, Skype and others that are centrally managed!? That probably costs a lot less money and they are already up and running for many years now. So far I've seen one major advantage of VoLTE over other voice over IP service on mobile devices: It can do a fallback to 2G and 3G circuit switched channels once you run out of LTE network coverage.

Recently, another advantage came to my mind: Instead of globally managed from a single country there are several independent VoLTE systems in each country. While that creates all sorts of problematic disadvantages, the advantage of such a distributed system is that there is no single point of failure and no single point of control and interception. So if somebody wants to spy on VoLTE installations they have to make the extra effort to break into each and every system independently. Not that I think that this is beyond the capabilities both in terms of knowledge and man power of some organizations but it's a lot harder to do and a lot more difficult to conceal.

Not that VoLTE is an end to end encrypted system but at least it makes it a more local problem.

OpenStreetMap (for Android) Part 3: Points of Interest and Editing

In part 3 on my series on OpenStreeMap let's have a closer look at Openstreemap for Android, or OSMAND for short. OSMAND is open source and hosted over at Github. Apart from the great navigation system it has been for me over the years (see point 8 here) it also serves me pretty well when searching for 'points of interest' (POI) such as supermarkets, nearby restaurants, etc. when I'm traveling.

What I didn't notice so far, however, is that POIs can also be shown in the map instead of just being searched for. The first screenshot on the left shows the default map, the second shows how the map looks like when all types of POIs are displayed and the third screenshot shows the left 'swipe-in' menu from which the display of POIs can be enabled. In my examples all types of POIs are shown but the menu also offers a list from which a subset can be selected, e.g. only restaurants and supermarkets. When touching one of the orange POIs, additional information is presented such as the name of the restaurant and, if available, other information such as opening hours, URL of the web site, phone number, etc. All very useful.

Screenshot_2015-03-05-07-42-49 Screenshot_2015-03-05-07-42-41 Screenshot_2015-03-05-07-43-01For those of you who would like to add, modify or delete POIs that are no longer there, OSMAND offers the option to do just that while one is on site. To add additional information to a POI, a long press brings up an edit menu. Editing POIs requires an Openstreetmap account and to configure the log-in credentials in the OSMAND settings on the device. The only quirck is that changes are not immediately visible on the map as it is an offline copy. Changes are uploaded immediately, however, and can be seen after a few minutes on the main OSM web site. It takes a couple of days for them to appear in the next update for the downloadable maps, however. Still, very useful, too.

What Happens To The PDP context when going to WiFi?

I was recently asked by a colleague if a mobile device keeps its PDP context with the UMTS network or the Default Bearer with the LTE network when it goes to Wifi. In the past the question was easily answered but today there are a number of different scenarios. As I'm sure that I will be asked again in the future I decided to write down a note here for easy reference.

Let's start with UMTS to Wi-Fi. Most devices on the market today deactivate the PDP context once a connection with a Wi-Fi network has been established. In practice this means that the IP address goes away and a new one is assigned when the device establishes a new PDP context again. You can easily check this as follows: While connected to UMTS go to a web pages such as "whatismyip.com" to find out your current IP address. Then switch to Wi-Fi, reload the page and you will see a different IP address. Once done, disable Wi-Fi again, wait until 3G connectivity has been reestablished and again reload the page. Again the IP address has changed. While this behavior can be observed with most smartphones and tablets, some keep the PDP context when changing to Wi-Fi so the IP address that was assigned before switching to Wi-Fi is shown again after disabling it again.

With LTE things are different in most mobile networks. Unlike UMTS that could live with a device not having a PDP context due to its dual circuit switched / packet switched nature, LTE requires a device to always have a default bearer, i.e. an IP address. Even if data connectivity is disabled in the menu or Wi-Fi is used the device keeps the IP address in the baseband chip for future use. Again, this can be easily verified with reloading pages such as "whatismyip.com" after changing networks. Some networks however, assign two default bearers for various reasons. In such cases the second default bearer is usually used for Internet connectivity. In such a configuration the second default bearer is disconnected when switching to Wi-Fi so one will observe different IP addresses before and after using a Wi-Fi network.

Not as straight forward anymore than it used to be.

OpenStreetMap – Part 2: It’s Just a Databasse – The Services Are Elsewhere

This is part 2 of my post series on Openstreetmap. Before having had a closer look I always assumed that Openstreetmap was something similar like Google maps, just open, free (as in freedom) and not as privacy invading. While this is true, I was quite amazed to find out that the structure of the project is quite unlike that of commercial map services.

Let's have a look at Google maps and other popular map services first. These are usually integrated products, i.e. Google owns the program, the render the map, they own the database behind the maps and the information where shops, hotels etc. that are are put into the map.

Openstreetmap on the other hand first and foremost is only one thing: A database. Yes, when you go to Openstreetmap.org you can see a map and do great things with it. But it's not as versatile as Google maps and others, perhaps, despite the routing capabilities that were recently added. And that's not it's intention anyway, it's sort of a technology demonstrator for the database. The applications that run on the Openstreetmap database are not developed by Openstreetmap, they are done by third parties. Openstreetmaps for Android (OSMAND) is a good example, it's not done by the entity known as Openstreetmaps. And Osmand is just one of many programs using the OSM database. OSM describes this in more detail in their FAQ so have a look here if you want to dig deeper.

And here's a link to a very long list of OSM-based services and a link to Android Apps that use OSM data of which OSMAND is just one. Lots of interesting stuff to discover there!

OpenStreetMap – Part 1: Routing on OSM

Osm-routingNot being a fan of being tracked online, especially not location wise, I've been using OpenStreetMap (OSM) and OSM for Android (OSMAND) a lot over the years. I've grown especially fond of OSMAND as it enables navigation with offline maps and both the program and the map data is open source and open data respectively. There are some things, however that others could do better on the PC, so I still had to resort to Google Maps if I wanted to get a first idea how a route could look like or to find out the distance between two locations. I don't use a Google account, my browser forgets Google's cookies whenever I close it and I regularly get a new IP address so privacy is more or less still retained. Nevertheless I would always have preferred an open solution. So you probably understand my joy when OpenStreetMap recently announced that they have integrated routing on their main page. Look for the arrow icon next to the "go" button. It's not a new feature, I take it that it's been available for quite some time on Open Source Routing Machine but I have to admit I didn't know until recently.

Actually there is a lot of stuff I didn't know about OSM and OSMAND and after watching the recording of the OSM presentation at 31C3 (in German only…) I had a closer look and found lots of amazing things that I have since made good use of. That's why I titled this post "part 1" and I will write down some notes about other topics such as editing the map via OSMAND and on the desktop, adding features, services that use OSM, etc. in a couple of follow-ups.

OCSP, Stapling And Android That Doesn’t Care

When surfing to an https protected website, most desktop browsers today make use of the Online Certificate Status Protocol (OCSP) to check the validity of the authentication certificate that was sent by the web site. There is lots of debate about whether this feature is useful or not but there's also a privacy aspect to this. Let me quote from Wikipedia:

"OCSP checking also impairs privacy, since it requires the client to contact a third party (the CA) to confirm certificate validity. A way to verify validity without disclosing browsing behavior would be desirable for some groups of users."

I guess I'm part of this group which is why I had a closer look at the OCSP Stapling feature after upgrading my Owncloud server to Ubuntu 14.04 which included an Apache web server update that supports the feature.

What is OCSP Stapling And How Is It Configured in Apache and Nginx?

In short, OCSP stapling means that the web server requests the OCSP information from the CA's OCSP server and then includes it as part of the TLS session establishment when a web browser sends a request for an https encrypted page. The advantage is that the web browser no longer has to send a request to the Certificate Authority to check the validity of the certificate that it has received from the website which in turn protects my privacy. Agreed, this one's part of the last 5% when it comes to privacy protection but every bit counts… Configuring OCSP stapling is actually quite straight forward and this post over at Digitalocean goes into the details including how to verify that everything is working.

Works Well On The Desktop But Android Doesn't Care

On the desktop, both Firefox and Thunderbird, the two programs I use most together with my Owncloud at home make use of the feature and no longer reach out to the Certificate Authority. A Wireshark trace nicely showed how the OCSP information is included during the TLS session establishment. Mission accomplished.

On the mobile side, Android doesn't seem to care at all about OCSP. That should probably not be very surprising as Google has disabled OCSP checking already back in 2012 in their Chrome desktop browser as well. No privacy issues here, good.

An interesting twist is the Opera Mobile browser on Android: When accessing my website it requests the OCSP status information during TLS session establishment and receives it. When going to another https site, however, which does not supply OCSP status information, there is no separate OCSP check as a consequence. That kind of defeats the purpose. But at least there's no privacy issue here.

Android USB Tethering to Connect a Raspberry Pi to the Internet

USB - TetheringThese days the easiest way to connect a PC or Raspberry Pi for that matter to the Internet via a smartphone is to use Wi-Fi tethering.  But there are still scenarios in which Wi-Fi is not ideal, e.g. when the air is already pretty "busy". Another way that I recently discovered is to use USB tethering that many Android phones also support in addition to the ubiquitous Wi-Fi tethering. The screenshot on the left shows CyanogenMod's Android menu to activate "USB Tethering". Luckily, Raspbian already comes with drivers for it so the new network interface is recognized immediately. The only thing that is required is an entry in /etc/network/interfaces so the USB0 interfaces gets an IP address from the phone:

auto usb0
iface usb0 inet dhcp

Owncloud Benchmarking On A New Raspberry Pi 2 vs. The Original One vs. a NUC

A month ago, the new Raspberry Pi 2 was released and needless to say I couldn't wait to get my hands on one to see by how much the new quad core processor and the 1 GB of RAM would speed up Owncloud.

Back in September 2014 I ran a similar benchmark, comparing Owncloud on a Raspberry Pi Model B, a BananaPi and an Intel Celeron based NUC. While the BananaPi wasn't as fast as the NUC it nevertheless ran Owncloud much quicker than the original Raspberry Pi. With the new Pi 2's hardware specifications now closely matching or perhaps even exceeding that of the Raspberry Pi when it comes to its 4 processor cores vs. the 2 cores of the BananaPi it was time for another Benchmark.

The use cases I used for this benchmark are the same as those used previously. The results are not fully comparable, however, as I have upgraded Owncloud from version 7 to version 8 in the meantime. Also, I have upgraded the NUC from Ubuntu 12.02 to 14.04 and the Truecrypt container was replaced by a dm-crypt partition. And finally, I've decided to run all tests over Wi-Fi instead of over an Ethernet cable as that's how I access my servers anyway.

An interesting thing to mention at this point is that after performing a software and kernel upgrade of my Raspian / Owncloud image on an SD card it runs on both the old and the new Pi. For the benchmark I used the same SD card in both the old and new Raspi which excludes differences due to different software installations and flash speeds. The Owncloud installation is identical on the NUC and the two Raspberry Pis as I used rsync to copy the Owncloud installation (/var/www/owncloud) and Owncloud's data directory from the NUC to the SD card that I then used to boot both Raspberry Pis.

Login Test

Like in the benchmark back in November the first test was about how quickly I could access my Owncloud account after typing-in username and password:

  • NUC: 3 seconds
  • Raspi 2: 5 seconds
  • Raspi: 18 seconds.

Displaying Address Book Entries

Getting my 300 address book entries out of the database and onto a web page took the following time on the three devices:

  • NUC: 5 seconds
  • Raspi 2: 9 seconds
  • Raspi: 26 seconds

Opening the Calendar

Entries of 5 different calendars are displayed on my calendar page and the time it took until all entries were shown on the web page are significantly different:

  • NUC: 4 seconds
  • Raspi 2: 13 seconds
  • Raspi: 104 seconds

Picture Uploads

For this test I uploaded 28 jpg images into a new folder on my Owncloud instance with a total file size of 71 MB. After each picture was uploaded a thumbnail was generated on the server and shown on the web page. And here's how the three systems fared:

  • NUC: 45 seconds
  • Raspi 2: 75 seconds
  • Raspi 1: 509 seconds

This is almost a 7 times speedup between the old and the new Raspberry Pi, due to the 4 CPU cores that are used simultaneously during the process. The two screenshots below (click to enlarge) show CPU usage on the old Raspberry Pi with a single processor and on the new Raspberry Pi with 4 processors.

Raspi1-OC-Upload-MultitaksingOn both systems, several Apache web server tasks are actively working on different activities of the multiple file upload activity. On the old Raspi they all have to share a single CPU core and each can therefore only use around 20% of the CPU's capacity.

 

Raspi2-OC-Upload-MultitaksingOn the Raspberry 2 the picture looks completely different. Instead of sharing a single CPU, several Apache web server tasks are running simultaneously and independently on several cores as can be seen in four bar graphs that represent CPU activity and the amount of processor time (CPU%) used by each. No wonder it is so much faster!

Show Link

The final test I ran was how long it takes to show the page with all pictures I have just uploaded if it is accessed via a "sharing link". This is the typical "I take pictures, upload them to the cloud and then share it with others" scenario. As it takes some time to generate and display the thumbnails of the uploaded pictures I have two results per system below, one for the page to show up in the web browser and one for the time it takes until all thumbnails are loaded:

  • NUC: 2 seconds (7 seconds until thumbnails are loaded)
  • Raspi 2: 5 seconds (16 incl. thumbs)
  • Raspi: 14 seconds (85 inc. thumbnails)

Summary

In all scenarios the new Raspberry Pi 2 ran significantly and noticeably faster than the old Pi. While not as fast as the much more expensive NUC system I can fully recommend the new Pi as an Owncloud server for home. It's still possible by all means to run an Owncloud on a previous generation Pi but it takes patience. As there is almost no price difference between the new and the old Pi, setting up an Owncloud server at home today on very inexpensive hardware yields much better results and usability than just a year ago. At the time I spent around 200 euros to move my Owncloud instance from an old Pi to a NUC. With the Pi 2 now available, I'm not sure if I would do it again for this reason.

More Details on What’s In Front of the Baseband – Part 2

In the previous post I had a couple of links to interesting information on what is between an antenna and the baseband radio chip in today's smartphones. As a quick follow-up here's my cheat-sheet which lists the components which is best used with this diagram over at Anandtech:

Downlink (Receive, RX) Path:

RF Front End Components

  • Antennas (e.g. 2 required for diversity or LTE MIMO)
  • Optional:  Antenna Tuner – Matches the (changing) impedance of the antenna with the transceiver. Impedance scenarios can be loaded dynamically when the environment (hand, grip, body location) changes.
  • Antenna switch – Pipes the signal to/from the antenna to band specific filters and power amplifiers for the band currently used.
  • Duplexers – Sit in each path after the antenna switch to combine/split the uplink and downlink signal that is transmitted/received from the antenna(s). Also, the duplexers include band specific filters.

Transceiver – includes the following sub-components:

  • Low noise Power Amplifiers (PA), several present, each covering different band(s). 
  • Down-converter: converts the frequency of the signal to a baseband frequency (to remove the carrier frequency (700 MHz to 5 GHz). Have a look at the Superhetrodyne receiver article over at Wikipedia for more information.
  • Another Power Amplifier stage
  • (Latest development from a Qualcomm 9×45 point of view: No companion chip necessary anymore for LTE carrier aggregation!)

After all of these steps the analog signal is ready to be given to the baseband modem split into in-phase and quadrature components. It's still analogue so the first task of the baseband chip (e.g. the 9×15 / 25 / 35 / 45 Qualcomm MDMs) convert the signal from analog to digital and it then starts it's magic to decode the information (which is outside the scope of this cheat-sheet).

Uplink (Transmit, TX) Path:

  • Baseband: The digital baseband chips's last stage is a digital to analog converter to deliver a weak analog signal to the transmitter chain of the transceiver chip.
  • Transciever:
  • Narrow-band amplifiers
  • Up-converter: Raises the signal to the transmit frequency
  • Driver amplifier raises the signal before it leaves the transceiver
  • Power Amplifiers: A switch inside the transceiver chip forwards the output signal to one of several power amplifiers, each dedicated to a specific frequency range. Several power amplifiers are typically included in a single chip outside of the transceiver chip. Optional: An envelope tracker chip can control the power amplifiers to reduce power requirements. The envelope tracker gets information of how much power to apply directly from the modem (i.e. the component before the transceiver!)
  • Duplexer: Mixes transmit and receive signals as they use the same antenna
  • Antenna switch
  • Antenna