OpenStreetMap – Part 2: It’s Just a Databasse – The Services Are Elsewhere

This is part 2 of my post series on Openstreetmap. Before having had a closer look I always assumed that Openstreetmap was something similar like Google maps, just open, free (as in freedom) and not as privacy invading. While this is true, I was quite amazed to find out that the structure of the project is quite unlike that of commercial map services.

Let's have a look at Google maps and other popular map services first. These are usually integrated products, i.e. Google owns the program, the render the map, they own the database behind the maps and the information where shops, hotels etc. that are are put into the map.

Openstreetmap on the other hand first and foremost is only one thing: A database. Yes, when you go to Openstreetmap.org you can see a map and do great things with it. But it's not as versatile as Google maps and others, perhaps, despite the routing capabilities that were recently added. And that's not it's intention anyway, it's sort of a technology demonstrator for the database. The applications that run on the Openstreetmap database are not developed by Openstreetmap, they are done by third parties. Openstreetmaps for Android (OSMAND) is a good example, it's not done by the entity known as Openstreetmaps. And Osmand is just one of many programs using the OSM database. OSM describes this in more detail in their FAQ so have a look here if you want to dig deeper.

And here's a link to a very long list of OSM-based services and a link to Android Apps that use OSM data of which OSMAND is just one. Lots of interesting stuff to discover there!

OpenStreetMap – Part 1: Routing on OSM

Osm-routingNot being a fan of being tracked online, especially not location wise, I've been using OpenStreetMap (OSM) and OSM for Android (OSMAND) a lot over the years. I've grown especially fond of OSMAND as it enables navigation with offline maps and both the program and the map data is open source and open data respectively. There are some things, however that others could do better on the PC, so I still had to resort to Google Maps if I wanted to get a first idea how a route could look like or to find out the distance between two locations. I don't use a Google account, my browser forgets Google's cookies whenever I close it and I regularly get a new IP address so privacy is more or less still retained. Nevertheless I would always have preferred an open solution. So you probably understand my joy when OpenStreetMap recently announced that they have integrated routing on their main page. Look for the arrow icon next to the "go" button. It's not a new feature, I take it that it's been available for quite some time on Open Source Routing Machine but I have to admit I didn't know until recently.

Actually there is a lot of stuff I didn't know about OSM and OSMAND and after watching the recording of the OSM presentation at 31C3 (in German only…) I had a closer look and found lots of amazing things that I have since made good use of. That's why I titled this post "part 1" and I will write down some notes about other topics such as editing the map via OSMAND and on the desktop, adding features, services that use OSM, etc. in a couple of follow-ups.

OCSP, Stapling And Android That Doesn’t Care

When surfing to an https protected website, most desktop browsers today make use of the Online Certificate Status Protocol (OCSP) to check the validity of the authentication certificate that was sent by the web site. There is lots of debate about whether this feature is useful or not but there's also a privacy aspect to this. Let me quote from Wikipedia:

"OCSP checking also impairs privacy, since it requires the client to contact a third party (the CA) to confirm certificate validity. A way to verify validity without disclosing browsing behavior would be desirable for some groups of users."

I guess I'm part of this group which is why I had a closer look at the OCSP Stapling feature after upgrading my Owncloud server to Ubuntu 14.04 which included an Apache web server update that supports the feature.

What is OCSP Stapling And How Is It Configured in Apache and Nginx?

In short, OCSP stapling means that the web server requests the OCSP information from the CA's OCSP server and then includes it as part of the TLS session establishment when a web browser sends a request for an https encrypted page. The advantage is that the web browser no longer has to send a request to the Certificate Authority to check the validity of the certificate that it has received from the website which in turn protects my privacy. Agreed, this one's part of the last 5% when it comes to privacy protection but every bit counts… Configuring OCSP stapling is actually quite straight forward and this post over at Digitalocean goes into the details including how to verify that everything is working.

Works Well On The Desktop But Android Doesn't Care

On the desktop, both Firefox and Thunderbird, the two programs I use most together with my Owncloud at home make use of the feature and no longer reach out to the Certificate Authority. A Wireshark trace nicely showed how the OCSP information is included during the TLS session establishment. Mission accomplished.

On the mobile side, Android doesn't seem to care at all about OCSP. That should probably not be very surprising as Google has disabled OCSP checking already back in 2012 in their Chrome desktop browser as well. No privacy issues here, good.

An interesting twist is the Opera Mobile browser on Android: When accessing my website it requests the OCSP status information during TLS session establishment and receives it. When going to another https site, however, which does not supply OCSP status information, there is no separate OCSP check as a consequence. That kind of defeats the purpose. But at least there's no privacy issue here.

Android USB Tethering to Connect a Raspberry Pi to the Internet

USB - TetheringThese days the easiest way to connect a PC or Raspberry Pi for that matter to the Internet via a smartphone is to use Wi-Fi tethering.  But there are still scenarios in which Wi-Fi is not ideal, e.g. when the air is already pretty "busy". Another way that I recently discovered is to use USB tethering that many Android phones also support in addition to the ubiquitous Wi-Fi tethering. The screenshot on the left shows CyanogenMod's Android menu to activate "USB Tethering". Luckily, Raspbian already comes with drivers for it so the new network interface is recognized immediately. The only thing that is required is an entry in /etc/network/interfaces so the USB0 interfaces gets an IP address from the phone:

auto usb0
iface usb0 inet dhcp

Owncloud Benchmarking On A New Raspberry Pi 2 vs. The Original One vs. a NUC

A month ago, the new Raspberry Pi 2 was released and needless to say I couldn't wait to get my hands on one to see by how much the new quad core processor and the 1 GB of RAM would speed up Owncloud.

Back in September 2014 I ran a similar benchmark, comparing Owncloud on a Raspberry Pi Model B, a BananaPi and an Intel Celeron based NUC. While the BananaPi wasn't as fast as the NUC it nevertheless ran Owncloud much quicker than the original Raspberry Pi. With the new Pi 2's hardware specifications now closely matching or perhaps even exceeding that of the Raspberry Pi when it comes to its 4 processor cores vs. the 2 cores of the BananaPi it was time for another Benchmark.

The use cases I used for this benchmark are the same as those used previously. The results are not fully comparable, however, as I have upgraded Owncloud from version 7 to version 8 in the meantime. Also, I have upgraded the NUC from Ubuntu 12.02 to 14.04 and the Truecrypt container was replaced by a dm-crypt partition. And finally, I've decided to run all tests over Wi-Fi instead of over an Ethernet cable as that's how I access my servers anyway.

An interesting thing to mention at this point is that after performing a software and kernel upgrade of my Raspian / Owncloud image on an SD card it runs on both the old and the new Pi. For the benchmark I used the same SD card in both the old and new Raspi which excludes differences due to different software installations and flash speeds. The Owncloud installation is identical on the NUC and the two Raspberry Pis as I used rsync to copy the Owncloud installation (/var/www/owncloud) and Owncloud's data directory from the NUC to the SD card that I then used to boot both Raspberry Pis.

Login Test

Like in the benchmark back in November the first test was about how quickly I could access my Owncloud account after typing-in username and password:

  • NUC: 3 seconds
  • Raspi 2: 5 seconds
  • Raspi: 18 seconds.

Displaying Address Book Entries

Getting my 300 address book entries out of the database and onto a web page took the following time on the three devices:

  • NUC: 5 seconds
  • Raspi 2: 9 seconds
  • Raspi: 26 seconds

Opening the Calendar

Entries of 5 different calendars are displayed on my calendar page and the time it took until all entries were shown on the web page are significantly different:

  • NUC: 4 seconds
  • Raspi 2: 13 seconds
  • Raspi: 104 seconds

Picture Uploads

For this test I uploaded 28 jpg images into a new folder on my Owncloud instance with a total file size of 71 MB. After each picture was uploaded a thumbnail was generated on the server and shown on the web page. And here's how the three systems fared:

  • NUC: 45 seconds
  • Raspi 2: 75 seconds
  • Raspi 1: 509 seconds

This is almost a 7 times speedup between the old and the new Raspberry Pi, due to the 4 CPU cores that are used simultaneously during the process. The two screenshots below (click to enlarge) show CPU usage on the old Raspberry Pi with a single processor and on the new Raspberry Pi with 4 processors.

Raspi1-OC-Upload-MultitaksingOn both systems, several Apache web server tasks are actively working on different activities of the multiple file upload activity. On the old Raspi they all have to share a single CPU core and each can therefore only use around 20% of the CPU's capacity.

 

Raspi2-OC-Upload-MultitaksingOn the Raspberry 2 the picture looks completely different. Instead of sharing a single CPU, several Apache web server tasks are running simultaneously and independently on several cores as can be seen in four bar graphs that represent CPU activity and the amount of processor time (CPU%) used by each. No wonder it is so much faster!

Show Link

The final test I ran was how long it takes to show the page with all pictures I have just uploaded if it is accessed via a "sharing link". This is the typical "I take pictures, upload them to the cloud and then share it with others" scenario. As it takes some time to generate and display the thumbnails of the uploaded pictures I have two results per system below, one for the page to show up in the web browser and one for the time it takes until all thumbnails are loaded:

  • NUC: 2 seconds (7 seconds until thumbnails are loaded)
  • Raspi 2: 5 seconds (16 incl. thumbs)
  • Raspi: 14 seconds (85 inc. thumbnails)

Summary

In all scenarios the new Raspberry Pi 2 ran significantly and noticeably faster than the old Pi. While not as fast as the much more expensive NUC system I can fully recommend the new Pi as an Owncloud server for home. It's still possible by all means to run an Owncloud on a previous generation Pi but it takes patience. As there is almost no price difference between the new and the old Pi, setting up an Owncloud server at home today on very inexpensive hardware yields much better results and usability than just a year ago. At the time I spent around 200 euros to move my Owncloud instance from an old Pi to a NUC. With the Pi 2 now available, I'm not sure if I would do it again for this reason.

More Details on What’s In Front of the Baseband – Part 2

In the previous post I had a couple of links to interesting information on what is between an antenna and the baseband radio chip in today's smartphones. As a quick follow-up here's my cheat-sheet which lists the components which is best used with this diagram over at Anandtech:

Downlink (Receive, RX) Path:

RF Front End Components

  • Antennas (e.g. 2 required for diversity or LTE MIMO)
  • Optional:  Antenna Tuner – Matches the (changing) impedance of the antenna with the transceiver. Impedance scenarios can be loaded dynamically when the environment (hand, grip, body location) changes.
  • Antenna switch – Pipes the signal to/from the antenna to band specific filters and power amplifiers for the band currently used.
  • Duplexers – Sit in each path after the antenna switch to combine/split the uplink and downlink signal that is transmitted/received from the antenna(s). Also, the duplexers include band specific filters.

Transceiver – includes the following sub-components:

  • Low noise Power Amplifiers (PA), several present, each covering different band(s). 
  • Down-converter: converts the frequency of the signal to a baseband frequency (to remove the carrier frequency (700 MHz to 5 GHz). Have a look at the Superhetrodyne receiver article over at Wikipedia for more information.
  • Another Power Amplifier stage
  • (Latest development from a Qualcomm 9×45 point of view: No companion chip necessary anymore for LTE carrier aggregation!)

After all of these steps the analog signal is ready to be given to the baseband modem split into in-phase and quadrature components. It's still analogue so the first task of the baseband chip (e.g. the 9×15 / 25 / 35 / 45 Qualcomm MDMs) convert the signal from analog to digital and it then starts it's magic to decode the information (which is outside the scope of this cheat-sheet).

Uplink (Transmit, TX) Path:

  • Baseband: The digital baseband chips's last stage is a digital to analog converter to deliver a weak analog signal to the transmitter chain of the transceiver chip.
  • Transciever:
  • Narrow-band amplifiers
  • Up-converter: Raises the signal to the transmit frequency
  • Driver amplifier raises the signal before it leaves the transceiver
  • Power Amplifiers: A switch inside the transceiver chip forwards the output signal to one of several power amplifiers, each dedicated to a specific frequency range. Several power amplifiers are typically included in a single chip outside of the transceiver chip. Optional: An envelope tracker chip can control the power amplifiers to reduce power requirements. The envelope tracker gets information of how much power to apply directly from the modem (i.e. the component before the transceiver!)
  • Duplexer: Mixes transmit and receive signals as they use the same antenna
  • Antenna switch
  • Antenna

More Details on What’s In Front of the Baseband

Back in July 2014 I wrote a post with a link over to Anandtech with a great introduction to the components and chips that sit in front of the baseband modem chip of a modern smartphone. Now the team over at Anandtech has greatly expanded on this and have written another article on the topic.

If you are interested but new to the topic it's worth to read the older post first as it includes a great diagram that shows how the components are connected together and to then continue with their latest post.

And once you've done that it's worth having a look at the tear-down of the iPhone 6 over at iFixit where they show how most of the components discussed in the posts above are used in this device. By my standards, the iPhone 6 has the most advanced modem front end to date as it supports 20 LTE bands, more than any other device currently on the market I'm aware of.

And if you are still looking for additional background information have a look at Wikipedia for extra knowledge on Superhetrodyne receivers and Envelope Tracking.

Preferring 3G over DSL – But The Channel Has Become Quite Busy

Last year I bought a prepaid SIM card in Austria with unlimited Internet access for 18 euros a month. It's done me a great service and has done so again recently when I was staying in Austria for a few weeks again. The place I was staying had a 5 Mbit/s DSL line but I nevertheless preferred to use the 3G connectivity instead.

You might wonder why but there's a simple reason: The uplink! While the DSL line only provided a few hundred kilobits per second, I could upload my documents at almost 10 times the speed over 3G, i.e. around 3 Mbit/s.

But things were getting quite tight over the air. In the early morning hours I could easily achieve the 10 Mbit/s in the downlink direction which was the subscribed limit. During daytime and in the evening, however, I could only get around 2-3 Mbit/s, despite the dual-carrier configuration of the 3G base station. In other words, things are getting quite busy. Time for LTE to arrive in the Austrian country side.

The FCC Makes It Crystal Clear That Blocking Wi-Fi In Hotels Is Illegal

Last year it was discovered that a Marriott owned hotel in the US was intentionally blocking private Wi-Fi access points in an attempt to force their guests to use their heavily overpriced Wi-Fi system. After a complaint to the FCC an investigation was started that led to a $600.000 fine and a public consultation. Cisco, Aruba Networks and Ruckus Networks came to Marriott's help and defended the practice, probably because they are selling the weapons, i.e. the equipment that detects other hotspots and send de-authentication packets to devices that use them. Shame on them!

Fortunately lots of companies such as Microsoft, Google and CTIA took the side of liberty and freedom of choice by positioning themselves against such active 'jamming' systems. A couple of days ago the FCC has now published a final 2 page 'enforcement advisory' with crystal clear wording:

"Warning – Wi-Fi Blocking is Prohibited

Persons and Businesses Causing Intentional Interference to Wi-Fi hotspots are Subject to Enforcement Action

[…] the Bureau is protecting consumers by aggressively investigating and acting
against such unlawful intentional interference. […]"

Thanks FCC and yes, Marriott you'll stay on my list of companies I will not do business with anymore!

via Heise here and here

A Raspi to the Rescue during Meetings with Crappy Internet Connectivity

Raspi-at-meetingQuite often I'm in multi-day meetings with lots of participants. In recent years more and more hotels have bought Wi-Fi equipment that can handle 80-100 participants with twice the number of Wi-Fi devices in a single room. Unfortunately, there's still ample opportunity to be trapped in meetings for several days where the Wi-Fi fails once the room starts filling up. For such cases I've developed a solution that can fix the issues for everyone but unfortunately it's not always possible to put it into action.

In such cases I'm now resorting to plan B, which is a Raspberry Pi that acts as a Wi-Fi access point for my devices that tunnels my traffic into the Internet via smartphone tethering, i.e. a second Wi-Fi link. In addition, the Raspberry acts as a VPN tunnel aggregator so all my data is transported over an encrypted tunnel to my home and only from there to the 'unprotected' Internet. If you are interested in the details and the scripts to configure a Raspberry Pi of your own for a similar purpose have a look at this blog post from a couple of months ago.

While this helps me it unfortunately doesn't help the other meeting participants as data over cellular is still too expensive to give 100 people access. But with the introduction of affordable global roaming data rates in 2014 I now have at least a solution for myself. This requires, of course, cellular reception in the meeting room which is sometimes also a challenge…