A Netflix Prize win is nigh: How they did it

June 28, 2009
45 Views

Breaking news: On Friday, Netflix Prize team “BellKor’s Pragmatic Chaos” passed the mark, qualifying for the $1,000,000 prize. The team includes last February’s PAW speaker Andreas Töscher.

But they haven’t won yet. Their qualification triggers a 30-day count-down during which all teams have a final chance to improve their efforts.

The Netflix Prize is an open contest in product recommendations. The competition has provided an incentive to teams worldwide to improve the state-of-the-art, and, in a sense, work as an extended R&D effort for Netflix to improve their movie recommendation accuracy.

How they did it: Joining forces. Only by an international collaboration, and the combining of methodologies, did the current leading team hit the mark. The team is composed of four teams that have also competed independently, located in the U.S., Canada, Austria and Israel.

Combined methodology made simple. Each team has developed an intricate approach. Once they agreed to collaborate, how hard did they have to work to integrate their systems? Actually, not hard at all. Rather than dig in, think hard, and assess where one system’s weaknesses may be compensated for by another team’s

Breaking news: On Friday, Netflix Prize team “BellKor’s Pragmatic Chaos” passed the mark, qualifying for the $1,000,000 prize. The team includes last February’s PAW speaker Andreas Töscher.

But they haven’t won yet. Their qualification triggers a 30-day count-down during which all teams have a final chance to improve their efforts.

The Netflix Prize is an open contest in product recommendations. The competition has provided an incentive to teams worldwide to improve the state-of-the-art, and, in a sense, work as an extended R&D effort for Netflix to improve their movie recommendation accuracy.

How they did it: Joining forces. Only by an international collaboration, and the combining of methodologies, did the current leading team hit the mark. The team is composed of four teams that have also competed independently, located in the U.S., Canada, Austria and Israel.

Combined methodology made simple. Each team has developed an intricate approach. Once they agreed to collaborate, how hard did they have to work to integrate their systems? Actually, not hard at all. Rather than dig in, think hard, and assess where one system’s weaknesses may be compensated for by another team’s strengths, they let predictive modeling do it – at least, that’s how Mr. Töscher indicated they did it when 2 of these sub-teams combined to form “BellKor in BigChaos” and become the leader several months ago (we don’t yet officially know how they did it this time).

In this approach, each system may conveniently be treated as a “black box,” training a new “meta-system” to combine the respective outputs into one better output. This is called “meta-learning” or “ensemble methods”, which elicits the concept of collective intelligence.

So stay tuned – we should know more soon!

You may be interested

How SAP Hana is Driving Big Data Startups
Big Data
298 shares3,142 views
Big Data
298 shares3,142 views

How SAP Hana is Driving Big Data Startups

Ryan Kh - July 20, 2017

The first version of SAP Hana was released in 2010, before Hadoop and other big data extraction tools were introduced.…

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion
Data Management
125 views
Data Management
125 views

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion

Manish Bhickta - July 20, 2017

Physical Data destruction techniques are efficient enough to destroy data, but they can never be considered eco-friendly. On the other…

10 Simple Rules for Creating a Good Data Management Plan
Data Management
69 shares728 views
Data Management
69 shares728 views

10 Simple Rules for Creating a Good Data Management Plan

GloriaKopp - July 20, 2017

Part of business planning is arranging how data will be used in the development of a project. This is why…