Firefox Nightly Secure DNS Experimental Results

A previous post discussed a planned Firefox Nightly experiment involving secure DNS via the DNS over HTTPS (DoH) protocol. That experiment is now complete and this post discusses the results.

Browser users are currently experiencing spying and spoofing of their DNS information due to reliance on the unsecured traditional DNS protocol. A paper from the 2018 Usenix Security Symposium  provides a new data point on how often DNS is actively interfered with – to say nothing of the passive data collection that it also endures. DoH will let Firefox securely and privately obtain DNS information from one or more services that it trusts to give correct answers and keep the interaction private.

Using a trusted DoH cloud based service in place of traditional DNS is a significant change in how networking operates and it raises many things to consider as we go forward when selecting servers (see “Moving Forward” at the end of this post). However, the initial experiment focused on validating two separate important technical questions:

1. Does the use of a cloud DNS service perform well enough to replace traditional DNS?
2. Does the use of a cloud DNS service create additional connection errors?

During July, about 25,000 Firefox Nightly 63 users who had previously agreed to be part of NIghtly experiments participated in some aspect of this study. Cloudflare operated the DoH servers that were used according to the privacy policy they have agreed to with Mozilla. Each user was additionally given information directly in the browser about the project. That information included the service provider, and an opportunity to decline participation in the study.

The experiment generated over a billion DoH transactions and is now closed. You can continue to manually enable DoH on your copy of Firefox Nightly if you like. See the bottom of the original announcement for instructions.

Results

Using HTTPS with a cloud service provider had only a minor performance impact on the majority of non-cached DNS queries as compared to traditional DNS. Most queries were around 6 milliseconds slower, which is an acceptable cost for the benefits of securing the data. However, the slowest DNS transactions performed much better with the new DoH based system than the traditional one – sometimes hundreds of milliseconds better.

The above chart shows the net improvement of the DoH performance distribution vs the traditional DNS performance distribution. The fastest DNS exchanges are at the left of the chart and the slowest at the right. The slowest 20% of DNS exchanges are radically improved (improvements of several seconds are truncated for chart formatting reasons at the extreme), while the majority of exchanges exhibit a small tolerable amount of overhead when using a cloud service. This is a good result.

We hypothesize the improvements at the tail of the distribution are derived from 2 advantages DoH has compared to traditional DNS. First, is the consistency of the service operation – when dealing with thousands of different operating system defined resolvers there are surely some that are overloaded, unmaintained, or forwarded to strange locations. Second, HTTP’s use of modern loss recovery and congestion control allow it to better operate on very busy or low quality networks.

The experiment also considered connection error rates and found that users using the DoH cloud service in ‘soft-fail’ mode experienced no statistically significant different rate of connection errors than users in a control group using traditional DNS. Soft-fail mode primarily uses DoH, but it will fallback to traditional DNS when a name does not resolve correctly or when a connection to the DoH provided address fails. The connection error rate measures whether an HTTP channel can be successfully established from a name and therefore incorporates the fallbacks into its measurements. These fallbacks are needed to ensure seamless operation in the presence of firewalled services and captive portals.

Moving Forward

We’re committed long term to building a larger ecosystem of trusted DoH providers that live up to a high standard of data handling. We’re also working on privacy preserving ways of dividing the DNS transactions between a set of providers, and/or partnering with servers geographically. Future experiments will likely reflect this work as we continue to move towards a future with secured DNS deployed for all of our users.