DuckDuckGo removes carve-out for Microsoft tracking scripts after securing policy change

A few months on from a tracking controversy hitting privacy-centric search veteran, DuckDuckGo, the company has announced it’s been able to amend terms with Microsoft, its search syndication partner, that had previously meant its mobile browsers and browser extensions were prevented from blocking advertising requests made by Microsoft scripts on third party sites.

In a blog post pledging “more privacy and transparency for DuckDuckGo web tracking protections”, founder and CEO, Gabe Weinberg, writes: “Over the next week, we will expand the third-party tracking scripts we block from loading on websites to include scripts from Microsoft in our browsing apps (iOS and Android) and our browser extensions (Chrome, Firefox, Safari, Edge and Opera), with beta apps to follow in the coming month.”

“This expands our 3rd-Party Tracker Loading Protection, which blocks identified tracking scripts from Facebook, Google, and other companies from loading on third-party websites, to now include third-party Microsoft tracking scripts. This web tracking protection is not offered by most other popular browsers by default and sits on top of many other DuckDuckGo protections,” he added.

DDG claims this third party tracker loading protection is not offered by most other popular browsers by default.

“Most browsers’ default tracking protection focuses on cookie and fingerprinting protections that only restrict third-party tracking scripts after they load in your browser. Unfortunately, that level of protection leaves information like your IP address and other identifiers sent with loading requests vulnerable to profiling. Our 3rd-Party Tracker Loading Protection helps address this vulnerability, by stopping most 3rd-party trackers from loading in the first place, providing significantly more protection,” Weinberg writes in the blog post.

“Previously, we were limited in how we could apply our 3rd-Party Tracker Loading Protection on Microsoft tracking scripts due to a policy requirement related to our use of Bing as a source for our private search results. We’re glad this is no longer the case. We have not had, and do not have, any similar limitation with any other company.”

“Microsoft scripts were never embedded in our search engine or apps, which do not track you,” he adds. “Websites insert these scripts for their own purposes, and so they never sent any information to DuckDuckGo. Since we were already restricting Microsoft tracking through our other web tracking protections, like blocking Microsoft’s third-party cookies in our browsers, this update means we’re now doing much more to block trackers than most other browsers.

Asked if DDG will be publishing its new contract with Microsoft, or whether it’s still bound by an NDA, Weinberg said: “Nothing else has changed and we don’t have other information to share on this.”

The carve-out for DDG’s search supplier was picked up in May via an independent audit conducted by privacy researcher, Zach Edwards.

At the time DDG ‘fessed up to anomaly but said it essentially had no choice to accept Microsoft’s terms, although it also said it wasn’t happy about the restriction and hoped to be able to remove it in the future.  

Asked whether the publicity generated by the controversy helped persuade the tech giant to relax the restriction on its ability to block Microsoft ad scripts on non-Microsoft sites, DDG referred us back to Microsoft.

When we put the same question to the tech giant a spokeswoman told us:

Microsoft has policies in place to ensure that we balance the needs of our publishers with the needs of our advertisers to accurately track conversions on our network. We have been partnering with DuckDuckGo to understand the implications of this policy and we are pleased to have arrived at a solution that addresses those concerns.

In a transparency-focused steps being announced today, DDG said it’s publishing its tracker protection list — available here on Github — although the company told us the information was available before but suggested it’s easier to find now.

It also sent us the following list of domains where it said it will be blocking Microsoft tracking requests:

Despite this expansion of DDG’s ability to block Microsoft tracking requests, there are still instances where Microsoft ad scripts are not blocked by DDG’s tools by default — related to processes used by advertisers to track conversions (i.e. to determine whether an ad click actually led to a purchase).

“To evaluate whether an ad on DuckDuckGo is effective, advertisers want to know if their ad clicks turn into purchases (conversions). To see this within Microsoft Advertising, they use Microsoft scripts from the bat.bing.com domain,” explains Weinberg in the blog post. “Currently, if an advertiser wants to detect conversions for their own ads that are shown on DuckDuckGo, 3rd-Party Tracker Loading Protection will not block bat.bing.com requests from loading on the advertiser’s website following DuckDuckGo ad clicks, but these requests are blocked in all other contexts. For anyone who wants to avoid this, it’s possible to disable ads in DuckDuckGo search settings.

DDG says it wants to go further to protect user privacy around ad conversion tracking — but admits this won’t happen any time soon. In the blog post Weinberg writes that “eventually” it wants to be able to replace the current process for ad conversions checks by migrating to a new architecture for assessing ad effectiveness privately.

“To eventually replace the reliance on bat.bing.com for evaluating ad effectiveness, we’ve started working on an architecture for private ad conversions that can be externally validated as non-profiling,” he says.

DDG is by no means alone here. Across the industry, all sorts of moves are afoot to evolve/rethink adtech infrastructure in response to privacy backlash — and to rising regulatory risk attached to individual tracking — efforts such as Google’s multi-year push to replace support for tracking cookies in Chrome with an alternative adtech stack (aka its ‘Privacy Sandbox’ proposal; which remains a (delayed) work in progress).

“DuckDuckGo isn’t alone in trying to solve this issue; Safari is working on Private Click Measurement (PCM) and Firefox is working on Interoperable Private Attribution (IPA). We hope these efforts can help move the entire digital ad industry forward to making privacy the default,” adds Weinberg. “We think this work is important because it means we can improve the advertising-based business model that countless companies rely on to provide free services, making it more private instead of throwing it out entirely.”

Asked about the timeline for developing such an infrastructure, he says: “We don’t have a timeline to share right now but it’s not an imminent announcement.”

Despite DDG’s assertion that viewing ads via its browsers is “anonymous”, its ad disclosure page confirms that it passes some personal data (IP address and user string) to Microsoft, its ad partner — for “accounting purposes” (aka “to charge the advertiser and pay us for proper clicks, which includes detection of improper clicks”, as Weinberg puts it).

“Per our ad page, Microsoft has committed [that] “when you click on a Microsoft-provided ad that appears on DuckDuckGo, Microsoft Advertising does not associate your ad-click behavior with a user profile. It also does not store or share that information other than for accounting purposes,” he says when pressed on what guarantees he has from Microsoft that user data passed for ad conversions doesn’t end up being repurposed for broader tracking and profiling of individuals.

In back and forth with TechCrunch, DDG also repeatedly emphasizied that its policy states that Microsoft does not link this data to a behavioral profile (or, indeed, share a user’s actual IP address etc).

However Weinberg concedes there are limits on how much control DDG can have over what happens to data once it’s passed — given, for example, the adtech ecosystem’s penchant for sharing (and synching) pseudonymized identifiers (e.g. hashes of identifiers) in order that digital activity may still be linked back to individual profiles, say after a few hops through a chain of third party data processors/enrichers, and thereby removing an earlier privacy screen… So, tl;dr, trying to shield your users’ privacy from prying third parties whilst operating in an ad ecosystem that’s been designed for pervasive surveillance (and allowed to sprawl all over the place) remains a massive firefight. 

“Staying anonymous ‘through the adtech ecosystem’ is a different story because once someone clicks on a site (whether or not they got there through DuckDuckGo search), they become subject to the website owner’s privacy policy and related practices,” Weinberg admits. “In our browsers, we try to limit that through our web privacy protections but we cannot control what the website owner (the ‘first party’) does, which could be sharing data with third-parties in the ad tech ecosystem.”

“The ad disclosure page makes clear viewing ads is anonymous and further covers ad clicks, which has a commitment from Microsoft to not profile users on ad click, which includes any behavioral profiling by them or others. This commitment includes not passing that data on to anyone,” DDG also claims.

“Our privacy policy states that viewing all search results (including ads) is anonymous, and Microsoft Advertising (or anyone else) does not get anything that can de-anonymize user searches at that time (including full IP address) in terms of being able to tie individual searches to individuals or together into a search history,” it adds.

In further developments being highlighted by the company today, DDG said it’s updated the Privacy Dashboard that’s displayed in its apps and extensions — to show “more information” about third-party requests, per its blog post.

“Using the updated Privacy Dashboard, users can see which third-party requests have been blocked from loading and which other third-party requests have loaded, with reasons for both when available,” Weinberg writes on that.

It has also relaunched its help page — with a promise that the overhauled content offers “a comprehensive explanation of all the web tracking protections we provide across platforms”.

“Users now have one place to look if they want to understand the different kinds of web privacy protections we offer on the platforms they use. This page also explains how different web tracking protections are offered based on what is technically possible on each platform, as well as what’s in development for this part of our product roadmap,” its blog post suggests.