By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Fast Logistic Regression on Big Data with Commodity Hardware? No Problem.
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Fast Logistic Regression on Big Data with Commodity Hardware? No Problem.
Analytics

Fast Logistic Regression on Big Data with Commodity Hardware? No Problem.

DavidMSmith
Last updated: 2011/07/18 at 2:58 PM
DavidMSmith
3 Min Read
SHARE

You might think that doing advanced statistical analysis on Big Data is out of reach for those of us without access to expensive hardware and software. For example, back in April SAS was proud to demonstrate being able to run logistic regression on a billion records (and “just a few” variables) in less than 80 seconds.

You might think that doing advanced statistical analysis on Big Data is out of reach for those of us without access to expensive hardware and software. For example, back in April SAS was proud to demonstrate being able to run logistic regression on a billion records (and “just a few” variables) in less than 80 seconds. But that feat required some serious hardware: two racks of Greenplum’s Data Computing Appliance (DCA). Each rack of the DCA has 16 servers, 192 Intel cores, and 768 GB of RAM, and pricing starts at $1 million. Add on SAS license fees for its High Performance Analytics suite, and you’re talking serious money.

We’re currently beta testing Revolution R Enterprise 5.0, which includes new features for using the power of a cluster of commodity hardware machines running Windows Server to perform statistical analysis on huge data sets. In the video below, Revolution Analytics’ Sue Ranney takes the beta for a spin, and uses the RevoScaleR package to run a logistic regression on 1.2 billion records of data on our 5-node cluster:

 

For comparison, each of the five nodes in our cluster has 16 GB of RAM with an Intel Xeon E3-1230, 3.2Ghz 8M cache quad-core processor, 16 Gb of RAM and a 1 TB hard drive. Total hardware cost: around $5,000. All the machines are running Windows Server 2008 with the Windows HPC Pack and Revolution R Enterprise 5.0 beta 1.

More Read

Investing in Data Center Efficiencies: Part Two

How to succeed in the enterprise without really trying: Apple’s crunch
Intel’s Next Generation Chip Architecture

And the time for that 1.2 billion row regression? 75 seconds: just as fast, and at less than 1% of the hardware cost.  See the details in the video linked below.

Revolution Analytics YouTube Channel: Logistic Regression in R with a Billion Rows

TAGGED: hardware
DavidMSmith July 18, 2011
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

Investing in Data Center Efficiencies: Part Two

6 Min Read

How to succeed in the enterprise without really trying: Apple’s crunch

5 Min Read

Intel’s Next Generation Chip Architecture

8 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence
data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?