IPv6 Snow Leopard versus IPv6 Lion

While digging into IPv6 support for the iPad, I stumbled on some unexpected results on my MBP. For starters, the default behavior I knew from a Snow Leopard MBP (10.6.8) is very different from what I observed on my Lion MBP (10.7.3).

To understand what follows, here's a brief description of my home network.

Regular IPv4 internet access is provided through VDSL over a b-box2 IAD. This multipurpose internet access device is part from the tripple-play offered by my ISP. It's a modem, router, firewall, dhcp server, dns proxy, etc... the box doesn't support IPv6 and neither does my ISP.

On my internal network (RFC1918 address space), I have an Apple Time Capsule, which has this odd feature that it can work as an IPv6 tunneling devices, and once configured, it will become the IPv6 router in the network. It takes the hassle out of IPv6 by taking the role of a Router Advertisement daemon and doubles as an IPv6 firewall and proxy name server. A /64 IPv6 network is assigned by Hurricane Electric to my Time Capsule and it's this prefix that is offered on the LAN through the Time Capsule.

Snow Leopard MacOS X 10.6.8 IPv6 behavior

Snow Leopard support for IPv6 is basic and straightforward. It will automagically configure itself when there's a Router Advertisement daemon on the LAN as shown in the following screenshot.

The IPv6 address is typically composed of the v6 prefix + the MAC address of the host. Besides auto-configuring its IP address and figuring out where the router is, there's nothing more to do to get the Internet6 at the doorstep. It's the most basic implementation of the Neighbor Discovery Protocol.

A dual stacked Snow Leopard will then default over IPv6 as per RFC3484. This means that if an application like Safari is asked to go to a site like http://www.ripe.net, it will go over IPv6 if it runs on a dual stacked system.

ripe.net shows your source IP address in its home page.

From a technical perspective and in a perfect world RFC3484 could be the desired behavior for a quick adaptation of IPv6. Unfortunately, the IPv6-world is not perfect and this has a negative influence on user experience.

1. there's this concern about privacy and auto-configured IPv6 addresses. As the MAC address is inserted in the IPv6 address, one could potentially follow a computer on the IPv6 Internet.
2. the auto configuration for IPv6 on Snow Leopard doesn't take into account DNS, which is the second most important thing after getting an IP address and the default router. RFC6106 described DNS extensions on IPv6 router advertisements.
3. what with bad IPv6 implementations or rogue radvd? From a user experience point of view there should be absolutely no difference between IPv4 and IPv6. Users don't care about IP addresses and that's how it should be. In practice, the IPv6 network has not yet reached the maturaty needed for a smooth transition as not all ISPs support IPv6 and the IPv6 network itself is not fully meshed.

No support for RFC6106 under Snow Leopard.

Lion MacOS X 10.7.3 IPv6 behavior

Whilst writing my blog entry on IPv6 support on iDevices I noticed that Lion was behaving very differently from Snow Leopard. One of the most important differences is that Lion no longer defaults over IPv6 on a dual stacked system. While looking into this, I first believed to have bumped into a bug as Lion would often go over IPv6 first and fall back later to IPv4 for no apparent reason.

It took me quite some digging and googling to have a better understanding of Lion's IPv6 implementation.

For starters, Lion support RFC6106 DNS extensions on IPv6 router advertisements, this completes the auto-configuration features for both IPv4 and IPv6.

DNS servers through DHCP (v4) and RA (v6)

In practice Lion will use mostly the IPv4 address unless it is not available.

Secondly Lion supports the privacy extensions as described in RFC3041 which results in 2 IPv6 addresses per interface. One is still based on the MAC address, the other is based on a randomized value.

Privacy extension add a second IPv6 address

On the LAN, it will use the IPv6 MAC generated address, beyond the LAN it will use the randomized address.

The behavior that puzzled me the most is Lion's unwillingness to default over IPv6 like Snow Leopard does. I noticed this while visiting the ripe.net website. The first time it shows my IPv6 source IP address:

Visiting ripe.net over IPv6, privacy included.

When the page is reloaded, it will no longer show my IPv6 address, but the source address is now my IPv4 address:

Revisiting ripe.net... now over IPv4???

It took a while before I finally found what's causing this. It has nothing to do with the name servers and their order (the IPv4 DHCP configured name server is always taken before the IPv6 name server, simply because of the order in the /etc/resolv.conf file), although it does help to flush the DNS cache to reproduce the behavior.

For your information, the DNS cache can be flushed with the following command in a terminal window: dscacheutil -flushcache.

After quite some Googling I stumbled on an excerpt of a mailing list explaining the Lion behavior: http://lists.apple.com/archives/ipv6-dev/2011/Jul/msg00020.html. In short, Lion measures the round trip time over IPv4 and IPv6 and the fastest wins. It's an implementation of an algorithm called "Happy Eyeballs" which is described in a draft IETF standard: http://tools.ietf.org/html/draft-ietf-v6ops-happy-eyeballs-07.

To understand the reason for "Happy Eyeballs", have a look at "Dual Stack Esotropia", an article written by glh at APNIC labs. The "Hampering Eyeballs" article on the RIPE Labs website by Emile Aben clarifies the challenge even more. In short, the algorithm is used in the application layer and was designed to handle optimized protocol selection (IPv4 versus IPv6 on dual stacked set-ups). It helps increasing user experience by monitoring which connectivity works best, not in throughput but in responsiveness.

Lion does this by by keeping track of RTT for every destination that was asked. This information is visible through the nettop -n -m route command on the cli. The command is interesting as it gives a dynamic view on the OS's routing table.

As a net result, in my set-up IPv6 looses under Lion, simply because the tunneled set-up is always slower.

According to the little literature available on the subject only very few applications and Operating Systems have deployed a version of "Happy Eyeballs". MacOS X Lion is the only operating system with the algorithm deployed. Chrome and Firefox have also an implementation which works very nicely.

Conclusion

From a user's perspective, "Happy Eyeballs" is a blessing in a dual stacked world. Under the hood, the applications figure out which connectivity works best and use it accordingly.

But from a technology point of view it's a bad thing, as it confirms there are issues with IPv6. The v6 network is not yet up to par and there are still some major peering problems, dividing the v6 network effectively in multiple Internets. Remember the Hurricane Electric versus Cogent issue? Cricket Liu had a blog post on the issue last year; one year later, it's still a problem.

traceroute6 from HE to COGENT.

Thanks to "Happy Eyeballs" I don't notice this issue under Lion. Thanks to "Happy Eyeballs" fixing the real challenges in the v6 cloud are minimized for an end user and pressure is reduced to get the v6 Internet as reliable and meshed as the v4 Internet.

In the end, "Happy Eyeballs" are a necessary evil to get IPv6 working on a dual-stacked end-user device. Being so user centric; no wonder Apple implemented it in Lion.

IPv6... we're not there yet.