This is a follow up to “Target Long Tail Searches with Keyword Patterns”
To get a sense of the scale of the long tail in search, Dustin Woodard recently put together an analysis of U.S. search data collected by Hitwise over a 3 month period, during which they measured 14 million different search terms. How did these break down?
- Top 100 terms: 5.7% of the all search traffic
- Top 500 terms: 8.9% of the all search traffic
- Top …
This is a follow up to “Target Long Tail Searches with Keyword Patterns”
To get a sense of the scale of the long tail in search, Dustin Woodard recently put together an analysis of U.S. search data collected by Hitwise over a 3 month period, during which they measured 14 million different search terms. How did these break down?
- Top 100 terms: 5.7% of the all search traffic
- Top 500 terms: 8.9% of the all search traffic
- Top 1,000 terms: 10.6% of the all search traffic
- Top 10,000 terms: 18.5% of the all search traffic
This means if you had a monopoly over the top 1,000 search terms across all search engines (which is impossible), you’d still be missing out on 89.4% of all search traffic. There’s so much traffic in the tail it is hard to even comprehend. To illustrate, if search were represented by a tiny lizard with a one-inch head, the tail of that lizard would stretch for 221 miles.
Yesterday, we described the concept of search patterns and how you can use them to summarize this type of long tail text data. Today, we will walk through a case study we put together to explain how Concentrate’s pattern discovery feature will help you find new competitive insights.
You can replicate this study yourself by signing up for the Plus version of Concentrate and loading competitive search data from providers like Hitwise, Compete, Keyword Discovery, or comScore. The input search data used in our analysis consisted of a sample of unique queries leading to clicks on top travel domains during Spring 2006, along with their frequency of occurrence (the chart is truncated after the 20th query):
Raw search data: most frequent queries by site
We loaded the full dataset of queries into Concentrate to generate summary patterns for each of 5 top travel sites. After each file of unique queries and associated metrics is loaded, the application generates reports which include summary statistics based on the head (top 50) and tail queries for each site. This is a good way to start looking at the data if we want to get a sense of each site’s long tail search strategy:
Head vs. tail queries for top travel sites
It appears that the long tail makes up the overwhelming majority of traffic for the travel planning and review sites, but is a much smaller percentage for transaction focused sites like Expedia and Travelocity. Measuring the size of the head and tail gives us a rough idea what is going on, but we need to dig deeper if we want to benchmark where we stand in various categories and produce actionable insights. Inspired by a recent New York Times infographic “Words They Used”, our data visualization guru, Chris Gemignani, downloaded the Pattern CSV file that Concentrate generated for each of these sites and created the following view of competition in the travel search sector:
Comparing travel searches by pattern
This chart compares the proportion of searches that go to each travel site for the top 25 patterns in the travel sector. The site getting the most traffic for each pattern is highlighted. Only searches that wound up at one of these five travel sites are considered.
The difference in search pattern profiles for these sites is striking. Tripadvisor leads the pack in the long tail, which makes sense given the huge amount of long tail user generated content on the site. TripAdvisor owns most of the pattern categories, but Yahoo Travel and Hotel-Guides take the lead in niche areas like maps and hotels. Traffic to Expedia and Travelocity is largely composed of navigational and branded queries (not shown). The only long tail patterns they have significant share for are “[x] ticket”, and “cheap [x]”.
The input data we used reflects referrals to these sites from a sample population of users who clicked on search engine result pages. Factors which will affect the number and type of search referrals a site received in this data include: how representative the sample is of the population of U.S. searchers as a whole, how much relevant content a site has for a given query pattern, and how well that content ranks in google and other search engines.
If a travel website repeated this study with Concentrate using current competitive data, then uploaded additional search data for their own site including other metrics beyond search frequency (see our demo using Google Analytics), the results might reveal that “things to do in [x]” queries lead to high quality visits and their site has a chance at winning more searches for that pattern. Based on this information they might decide to make a move on TripAdvisor in that content category. Using Concentrate, they could streamline the process by downloading thousands of real queries for this pattern sent to their competitor:
Some queries in TripAdvisor pattern: “things to do in [x]”
Take Action: Some ideas for next steps
- Use Concentrate patterns to segment your sites query referrals, then estimate revenue opportunities for underperforming long tail categories and target your efforts accordingly
- Expand into new content areas based on trends discovered in long tail search patterns over time and ensure your site’s information supply (PDF) matches customer demand
- Get a subscription to Concentrate Plus and repeat this case study for your industry
- Use your industry search patterns as a guide to optimize internal website architecture
- Jumpstart your SEM efforts by using Concentrate to automatically create tight adgroups containing similar queries found within long tail each pattern
- Use your patterns as templates for dynamic filters within your website to show targeted ads, content, related links, or content recommendations based on query referral strings
- If you are a financial analyst or product planner, spot emerging trends in the long tail with Concentrate and use the information to guide decision making