By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    predictive analytics in dropshipping
    Predictive Analytics Helps New Dropshipping Businesses Thrive
    12 Min Read
    data-driven approach in healthcare
    The Importance of Data-Driven Approaches to Improving Healthcare in Rural Areas
    6 Min Read
    analytics for tax compliance
    Analytics Changes the Calculus of Business Tax Compliance
    8 Min Read
    big data analytics in gaming
    The Role of Big Data Analytics in Gaming
    10 Min Read
    analyst,women,looking,at,kpi,data,on,computer,screen
    Promising Benefits of Predictive Analytics in Asset Management
    11 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Predictive Analytics Q & A with Gregory Piatetsky-Shapiro
Share
Notification Show More
Latest News
ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence
How Big Data Is Transforming the Maritime Industry
How Big Data Is Transforming the Maritime Industry
Big Data
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Predictive Analytics > Predictive Analytics Q & A with Gregory Piatetsky-Shapiro
AnalyticsPredictive Analytics

Predictive Analytics Q & A with Gregory Piatetsky-Shapiro

Brett Stupakevich
Last updated: 2012/03/06 at 5:49 PM
Brett Stupakevich
8 Min Read
SHARE

gps6 photo (predictive analytics data scientist 2 big data advanced analytics )With this week’s Predictive Analytics World Conference (#pawcon) now in full-force, we brought in an industry expert – Gregory Piatetsky-Shapiro of KDNu

gps6 photo (predictive analytics data scientist 2 big data advanced analytics )With this week’s Predictive Analytics World Conference (#pawcon) now in full-force, we brought in an industry expert – Gregory Piatetsky-Shapiro of KDNuggets (an online portal dedicated to knowledge discovery) – to give you insights into how analytics can reveal insights to drive business value and even change the world.

Gregory Piatetsky-Shapiro, PhD (@KDNuggets) is the president of KDnuggets, which provides consulting in the areas of business analytics, data mining, data science, and knowledge discovery. Previously, he led data mining and consulting groups at GTE Laboratories, Knowledge Stream Partners, and Xchange. He has extensive experience developing CRM, customer attrition, cross-sell, segmentation and other models for some of the leading banks, insurance companies, and telecommunications companies.

He is also the editor and publisher of KDnuggets News, the leading newsletter on data mining and knowledge discovery (published since 1993), and the KDnuggets.com website (published since 1997), a top resource for data mining and analytics news, software, jobs, courses, data, education, and more. Gregory has over 60 publications, with over 10,000 citations, including two best-selling books and several edited collections on topics related to data mining and knowledge discovery.

Q − You recently said on Twitter that detective Sherlock Holmes would have been “a good data scientist.” What skills do you think he possessed that anyone can use to discover the unknowns in data?

More Read

Predictive Analytics

Why Every College Student Should Study Predictive Analytics

Answers to the Most Frequently Asked PAW Questions
How BI and Data Analytics Gurus Used Twitter in February
Predictive Analytics on Big Data – What Does the Future Hold?
Predictive Analytics World New York City Conference Announces Speaker Line-Up

A −I read most of Arthur Conan Doyle books on Sherlock Holmes as a kid, and loved his powers of deduction. Holmes was a keen observer of facts and had good logic. He also was on the cutting edge of the science in his time; so today’s Sherlock Holmes would be analyzing social networks of the suspects (and perhaps hacking them) in addition to looking for fingerprints. Finally, he had great intuition and knew how to reject wrong hypotheses. No matter how initially appealing they appeared, if these hypotheses were not supported by facts, he rejected them.

Q − Holmes was known for seeing “hidden” information and identifying opportunities and threats. In the most recent film, the viewer was given a glimpse of his method of calculating the next move before it happened in sort of a flash-forward scene. Do you think today’s predictive analytics technologies and data visualizations give us this “extra sense” that Holmes had? What does this do for companies in competitive situations?

A − Of course, analytics is much easier when we predict behavior of inanimate objects – asteroids, viruses, etc. Predicting human behavior is much more difficult. Predictive analytics today gives us the ability to predict future behavior, but for non-trivial predictions they are only accurate in aggregate, not for individuals.

For example, say Verizon’s monthly customer churn rate is 2%. That means that every month 2 out of 100 customers switch. With analytics, we can select a group of customers where churning is 5-7 times more frequent. The difference is that the expected churn rate in the analytics-selected list can be 10%-14% versus 2% in a random list. Of the 100 selected customers we expect only 10-14 to churn. 

This has enormous business value because these customers can be contacted much more efficiently and personalized offers can be made to retain them. However, this example shows the limits of analytics in predicting human behavior “in aggregate” –  it is far from perfect. But it does NOT need to be perfect to be useful.

Analytics can be used when there are many thousands of similar customers. When we have a single person, statistical methods are not relevant and need to be augmented with knowledge and rules. Ultimately, a combination of artificial intelligence and statistical methods can be a powerful predictor.

Q − You also said recently on Twitter that you think the “data scientist” title is provocative but misleading. You suggested data engineer as a description. Can you distinguish the roles for us and give us a “for instance” of when a company would involve a data engineer or scientist?

I am not saying “data scientist” is provocative – I think it is a great title for marketing and branding. But the people hired in industry under this title do very applied things – a combination of code tuning, business insight, and knowing how to extract business value from data. This is very applied and a more accurate (if less sexy) description is “data engineer” (or perhaps “data wizard” or “data wrangler”).

People who do actual science and publish in top journals or conferences almost never use the “data scientist” title. They more frequently use “knowledge discovery” or “data mining” in their position or lab description. For example, in our academic and research positions listings, there are over a dozen positions with “data mining” and only one with “data scientist” in the title.

Q − In your Analytics and Data Mining Industry Overview presentation, you said that big data is a second industrial revolution and that it will help us do old things better like churn prediction, customer modeling, make recommendations, detect fraud and security/intelligence. Where do you see these “old things” changing the most (e.g. industries or markets)? What lessons do we still have to learn?

We already see huge changes in advertising and behavioral targeting, and most consumer-oriented business (retail, telecom, etc.), are actively leveraging big data to get more value from the customers. However, sometimes the increased accuracy may backfire as in the recent story of the Target retail chain using changes in buying behavior to learn that a teen was pregnant before her father did. I don’t know exactly where the line is that Target has crossed, but as a society we will have to adjust to our changing privacy in the era of big data.

Q − You have a storied background in “knowledge discovery” from developing models to predicting child support non-payment and attrition to predicting drug effectiveness to developing methods for identifying online auction fraud. How can companies capitalize on knowledge discovery? How do you feel the term knowledge discovery applies to “predictive analytics” today?

I coined the term “knowledge discovery” to emphasize that what we want to find in data is some understandable knowledge and not just incomprehensible patterns. Much of the big data analysis and many of the best predictive algorithms, unfortunately, produce results that are incomprehensible. So we have a tension between accuracy and understandability, but I think that better understanding of predictive models will contribute to increased trust for such models, and may also help resolve some privacy issues by providing increased transparency.

Amanda Brandon
Spotfire Blogging Team

TAGGED: kdnuggets, predictive analytics world
Brett Stupakevich March 6, 2012
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

Predictive Analytics
Predictive Analytics

Why Every College Student Should Study Predictive Analytics

6 Min Read

Answers to the Most Frequently Asked PAW Questions

4 Min Read

How BI and Data Analytics Gurus Used Twitter in February

4 Min Read

Predictive Analytics on Big Data – What Does the Future Hold?

0 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence
giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?