Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: The Netflix Prize, Occam’s Razor and PMML
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Predictive Analytics > The Netflix Prize, Occam’s Razor and PMML
Predictive Analytics

The Netflix Prize, Occam’s Razor and PMML

MichaelZeller
MichaelZeller
4 Min Read
SHARE

I just finished reading an excellent posting on the Netflix Tech Blog by Xavier Amatriain and Justin Basilico. Entitled “Netflix Recommendations: Beyond the 5 stars (Part 1)“, it gives a very nice account of the actual application of the 107 algorithms submitted as the winner solution to the Netflix prize.

I just finished reading an excellent posting on the Netflix Tech Blog by Xavier Amatriain and Justin Basilico. Entitled “Netflix Recommendations: Beyond the 5 stars (Part 1)“, it gives a very nice account of the actual application of the 107 algorithms submitted as the winner solution to the Netflix prize.

I recall reading about the prize when the winner team was finally announced. I basically asked myself: “How on earth will Netflix implement all these algorithms and put them to work?” The obvious answer was “they won’t”. Too complicated, too time consuming.

More Read

Research Uncovers Keys to Using Predictive Analytics
Scorecards in PMML: A Primer
Dos and Donts for getting help
Putting predictive analytics to work at Infinity Insurance
March Madness to Achieve Optimized Performance Management

As pointed out by Xavier and Justin, at the end, two algorithms (SVD and Restricted Boltzmann Machine) were selected, based on performance, and eventually made into production where they are busy making recommendations.
In their posting, Xavier and Justin put it clearly:

“We evaluated some of the new methods offline but the additional accuracy gains that we measured did not seem to justify the engineering effort needed to bring them into a production environment.”

On top of that, they go on to mention that the focus of the recommendations algorithm has changed as business has shifted to on-line streaming. With that, the input data has changed considerably as well as customer behavior. Basically, the proposed solution (an intricate combination of 107 algorithms) no longer applies since the rules of the game have changed. That’s the very nature of business.  

You may be wondering about Occam’s razor at this point. How does it relate to the Netflix Prize? Simple, the razor “asserts that one should proceed to simpler theories until simplicity can be traded for greater explanatory power. The simplest available theory need not be the most accurate” (Wikipedia).

Obviously, the razor does not apply to a contest in which participants are battling each other for the prize. The more accurate, the better, right? But, how about real life? We know that every predictive solution has a cost. They take time to implement and deploy.

But, can we somehow create a data mining contest in which Occam’s razor is taken into account? Is that possible? I believe the answer is “yes” and it involves the use of open standards. For example, if the proposed solutions were to be delivered in PMML (the Predictive Model Markup Language) format, they could be put to work immediately. Also, since PMML is supported by all the top data mining tools, re-creating the same solution using existing software would not take an arm and a leg. And, whenever represented in PMML, it can easily be understood. Given that PMML is XML-based, it contains a verbose but accurate representation of all model details. Whenever a solution is represented in PMML, there is no need for an extra document to explain it. The same file can be used for explaining the solution and for deploying it.

Finally, given the agility introduced by the standard itself, predictive solutions can be easily adjusted to new business requirements. In fact, they can even be used to drive new business opportunities. And, that’s a prize worth winning.


TAGGED:pmml
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics and truck accident claims
How Data Analytics Reduces Truck Accidents and Speeds Up Claims
Analytics Big Data Exclusive
predictive analytics for interior designers
Interior Designers Boost Profits with Predictive Analytics
Analytics Exclusive Predictive Analytics
big data and cybercrime
Stopping Lateral Movement in a Data-Heavy, Edge-First World
Big Data Exclusive
AI and data mining
What the Rise of AI Web Scrapers Means for Data Teams
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

ACM Data Mining Talk: Representing Solutions with PMML

2 Min Read

Predictive Analytics in your iPhone

3 Min Read

SAIC and Zementis to bring “smarts” to the Smart Grid

6 Min Read

Open Source Analytics Reaches Main Street (and Some Other Trends in Analytics)

8 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?