The “Not Provided” Search Scam

With the recent FTC decision to not pursue Google for rigging search results in their favor, TOTSB was reminded to check in on their clever 2011 organic search scam.  If you too, have been having trouble understanding your organic search traffic with continued growth in “not provided”, you are not the only one.

With the recent FTC decision to not pursue Google for rigging search results in their favor, TOTSB was reminded to check in on their clever 2011 organic search scam.  If you too, have been having trouble understanding your organic search traffic with continued growth in “not provided”, you are not the only one. The global change was done under the auspices of security (sounds familiar) but had the back-handed benefit of hampering digital marketers understanding of the very organic keywords that were used to find their sites from the search engine. It is a classic algorithm: 3-for-me-and-1-for-you.

In case, you missed it on October 11, 2011 Google decided to implement Secure Sockets Layer technology for their authenticated (Google, Gmail, YouTube, etc…) search users. In practice, this effectively meant that the referral string that is otherwise passed by Web servers to browsers indicating the referring page URL is truncated. Specifically, while the search engines domain is still included in the referral string, now the actual search queries are excluded. Of course, they certainly track on their end what their users clicked on – this is valuable insight for PageRank..

While often mistaken as an analytics problem, this is actually the case for all site analytics systems. Moreover, this is an organic search problem only; as long as you are paying for Google’s paid search advertising you can have the full referral data inclusive of search queries.

40% and Growing
Back in late 2011, a drone originally calculated the impact as single-digit percentage. Sounds manageable but as more and more stories emerge about much higher percentages this lack of search results transparency becomes more troubling. For your reference, TOTSB decided to take a look at our own site (in both Google Analytics and Piwik) and were shocked to learn as much as 4x the expected amount…and it is  trending higher. Below you can see the “Not Provided” percentage volume increasing since early-2011.

An indepdent SEO firm’s prepared a study that looked at many Web sites – but the same problem persists.

Implications
It is pretty clear that if you are interested in optimizing your organic search presence, the hand that giveth has taketh away. With Google’s dominant position in the search market, it essentially means that about half of your organic search keyword results cannot be understood right now. Worse, this remarkably could continue higher.

The net effect of this move is that Google is denying site owners (the providers of free content to the search engine) their referral information. It is absolutely outrageous that otherwise discerning digital marketers allow this to happen – perhaps class action legal action will emerge. Maybe the W3C or the IAB should get involved and speak out about this perversion of data control.

What to Do?
It is easy enough to track your own Web site’s numbers but beyond that advertisers should start playing hardball and complain – especially those buying paid search. Some are already making noise, including a petition where people can take action for more organic referral transparency; you can even buy a (Not Provided) T-Shirt. In addition, the organization fairsearch.org will hopefully include this in the scope of their work.