A couple of days ago I suddenly had strange intermittent problems to connect to my services at home. It worked one minute and the servers were not found the next. At first I thought it must be a DSL problem but I quickly figured out that this was not the case at all when I checked the DNS resolution for my domains. Here’s the response of two DNS queries which were one minute apart.
$ ping www.some-domain.com PING www.some-domain.com (217.253.68.195) 56(84) bytes of data. [...] $ ping www.some-domain.com PING www.some-domain.com (79.216.56.147) 56(84) bytes of data.
What should have been the same IP address suddenly oscillated between two entirely different IP addresses. What’s more is that this happened for several of my domain names via the local DNS resolvers, over Google’s 8.8.8.8 DNS server and also over two VPN tunnels over which yet other DNS resolvers were used.
While most of the time I got the correct IP address, every now and then I got a second, invalid IP address, which would not lead to my servers at home. I soon realized that the invalid IP address was the IP address I previously had before my DSL router re-synchronized the day before. But why was I still getting this IP address when making DNS requests and only sometimes?
While working on the issue, things returned back to normal, so I can’t be 100% sure what happened. But I strongly suspect one of the authoritative DNS servers for my domains was acting up. In case you have no idea which DNS servers supply the ultimate truth for your domains, you can find out as follows:
$ dig NS +short some-domain.com ns5.inwx.net. ns4.inwx.com. ns2.inwx.de. ns.inwx.de. ns3.inwx.eu.
This is the result I got as my domains are hosted at inwx. With a ‘short’ bash command I then queried all servers directly instead of going through the DNS chain and got the following result:
$ dig www.some-domain.com @ns.inwx.de +short; \ dig www.some-domain.com @ns2.inwx.de +short; \ dig www.some-domain.com @ns3.inwx.eu +short; \ dig www.some-domain.com @ns4.inwx.com +short; \ dig www.some-domain.com @ns5.inwx.net +short 79.216.56.147 79.216.56.147 79.216.56.147 no answer 79.216.56.147
This was the result I got by the time things went back to normal. I suspect that the 4th DNS server that was taken offline once my DNS hoster realized that it was serving corrupted data. After a while that name server came online again and returned the correct IP address for my domain names as well. Things have been stable since then but I decided to run a DNS query once a minute and log the results anyway.
$ while true; do dig www.some-domain.com +short; sleep 61; done
Just to make sure…
This makes one realize just how dependent one is on the the DNS servers working correctly. And in case you wonder about the creative title for this post, it was inspired by this.