Reflecting on 2010: Searching for Answers

Yes, it’s that time of year when we take a moment to reflect on the past year’s accomplishments and muse about what the next year will bring. Other than milder weather!

I began this year as a Noogler and leave it as a Xoogler. I hope I left Google better than I found it — I’m certainly proud of the improvements my team made to the quality of local authority pages. I also tried to infuse Google with some of the scrappy start-up culture I’d picked up at Endeca, particularly focusing on the hiring process. In information retrieval terms, I’d say that Google’s hiring process does extremely well when it comes to precision, but could use improvement in the areas of recall and efficiency. Still, I’m impressed at how well Google has maintained its quality standards as the company has grown. Finally, I couldn’t help being an extrovert: I developed warm relationships with the lead bloggers covering local search, including Andrew Shotland, David Mihm, Gib Olander, Greg Sterling, and Mike Blumenthal. Indeed, when I announced my departure, Mike wrote a really nice post about the friendship we cultivated over the past year. I hope that he continues to have such relationships with my former co-workers.

Looking back at what was on my mind when this year began, I had lots of questions around exploratory, mobile, real-time, social/collaborative search. I also wondered whether it was possible to offer more transparency in relevance ranking without losing ground in the battle against spam and black-hat SEO.

I’m as bullish as ever on the value of exploratory search: part of why I joined LinkedIn is that a significant fraction of the site’s value comes from supporting users’ exploratory search needs. I also published a position paper at the SIGIR 2010 Workshop on Simulation of Interaction proposing the use of query performance prediction to model the fidelity of communication between user and system, thus helping HCIR researchers to simulate query refinement with standard test collections. And of course exploratory search was a major theme at the HCIR 2010 workshop, not only providing the basis for the first HCIR Challenge, but even extending to new territory with Max Wilson and David Elsweiler’s work on casual leisure searching.

As for mobile search, I’d say that 2010 has been the year of “mobile first“. Thanks to a generous gift from my former employer, I’ve become a regular user of the mobile web–and of search in particular. To my surprise, the communication bottleneck has not been screen real estate, but rather the difficulty of entering text. And innovative approaches like voice search and Swype go a long way to mitigate that difficulty.

On to real-time search. Not surprisingly, my favorite innovation in this space is LinkedIn Signal, which offers exploratory search for Twitter. I still struggle to find use cases that emphasize the “real-time” aspect of Twitter and other microblogging services, but I am convinced that the path to utility lies in tools that support organization, analysis, and exploration.

On the social/collaborative front, I’m happy to work for a company whose charter includes “supporting mediated search by linking people to people, rather than directly to information”. While the biggest event in this space in 2010 was Facebook’s introduction of the Like button, I’m not convinced that “likes” have supplanted links. I’m still looking to niche players like Topsy and Blekko to push innovation in this space.

Speaking of Blekko, they’ve made an impressive attempt to increase the transparency of relevance ranking. But, as I blogged earlier this year, I think that, at least for the time being, Google is making the right decision to keep some of its details secret. Now that web search is essentially a duopoly (at least in the US), I believe the real test of the value of transparency to users will be whether one of the two parties employs it as competitive differentiator.

What’s in store for 2011? LinkedIn CEO Jeff Weiner has a vision of using data science to provide a “Pandora for people“, and that’s a vision I’m eager to help realize. Not surprisingly, when I blogged in 2008 about where Google wasn’t good enough, two of the four areas I cited were finding jobs and find employees. Even then I recognized that LinkedIn was the best at both. But LinkedIn can be so much more, and I am looking forward to working with an incredible team and incredible data on a delightful set of information science challenges.

Happy New Year! I hope that 2011 brings you great answers — and great questions!