By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Behind the scenes of REvolution’s 64-bit Windows port of R
Share
Notification Show More
Latest News
big data mac performance
Data-Driven Tips to Optimize the Speed of Macs
News
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
Artificial Intelligence
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > Behind the scenes of REvolution’s 64-bit Windows port of R
Data MiningPredictive Analytics

Behind the scenes of REvolution’s 64-bit Windows port of R

DavidMSmith
Last updated: 2009/04/13 at 8:36 PM
DavidMSmith
9 Min Read
SHARE

I’m very happy to report that after 6 months of development by a team of 8 developers (at its peak) and two months of beta testing, we’re ready to release REvolution R Enterprise for the 64-bit Windows platform.

Although much of the work of porting R to a 64-bit codebase was done several years ago by the R Core Team for the Linux and MacOS platforms, working binaries for the 64-bit Windows platform have proven problematic until now. The main problem has been finding a reliable toolchain to compile R on 64-bit Windows. R is routinely compiled for 32-bit Windows systems with the free mingcw compiler (a version of gcc for Windows), but it has proven unreliable for 64-bit builds. 

The REvolution development team applied their expertise in the R source code and Intel’s C++ and Fortran compilers to build R for 64-bit Windows. R is a large program, and building it on a compiler other than gcc required a several changes to the core R code (which have of course been released back to the R Project as GPL sources). On the other hand, using the Intel compilers allowed us to take advantage of the architecture-native optimizations Intel’s own compiler provides, boosting the performance of R. To f…

More Read

supply chain analytics

Automotive Industry Uses Analytics To Solve Pressing Supply Chain Issues

How can CIOs Build Business Value with Business Analytics?
Seven Benefits of Using AI to Perform Text Analysis
7 Data Lineage Tool Tips For Preventing Human Error in Data Processing
Data Analytics Plays a Vital Role in Teacher Verification Software

I’m very happy to report that after 6 months of development by a team of 8 developers (at its peak) and two months of beta testing, we’re ready to release REvolution R Enterprise for the 64-bit Windows platform.

Although much of the work of porting R to a 64-bit codebase was done several years ago by the R Core Team for the Linux and MacOS platforms, working binaries for the 64-bit Windows platform have proven problematic until now. The main problem has been finding a reliable toolchain to compile R on 64-bit Windows. R is routinely compiled for 32-bit Windows systems with the free mingcw compiler (a version of gcc for Windows), but it has proven unreliable for 64-bit builds. 

The REvolution development team applied their expertise in the R source code and Intel’s C++ and Fortran compilers to build R for 64-bit Windows. R is a large program, and building it on a compiler other than gcc required a several changes to the core R code (which have of course been released back to the R Project as GPL sources). On the other hand, using the Intel compilers allowed us to take advantage of the architecture-native optimizations Intel’s own compiler provides, boosting the performance of R. To further improve performance, we have integrated the Intel MKL numerical libraries: in addition to providing key optimizations of core mathematical routines, the library enables R to use multiple processors in parallel, further speeding up some mathematical functions on multicore workstations.

Another major challenge in porting R to 64-bit Windows has been in porting the packages. As we’ve been working on this project, we heard from many customers who were interested in using REvolution R on 64-bit Windows … provided the packages they used were also available. Of course, for just about every customer this was a different set of packages. To address this, we set about porting every R package to 64-bit Windows and making binary versions available. R has over 2000 packages (counting those from the BioConductor project), so this was a huge undertaking. 

Packages that contained only R code run without modification, but those that contained compiled code were time-consuming to port. Some packages are integrations with other tools which themselves needed to be ported to 64-bit Windows (as was the case with graphviz, a visualization tool that many of the BioConductor packages depend on). Porting code is always a good way to expose hard-to-find bugs, and we’ve found and fixed quite a few along the way (and communicated those changes back to the package authors). The final tally of 64-bit Windows packages that will be available on release isn’t yet complete, but already stands at over 1500. (Unfortunately, it will not be possible for us to port all packages: some have license terms that disallow it, and others integrate with third-party tools that can’t be ported.)

It’s been a long and tricky project, but we’re very excited to be able to release this port. I want to congratulate our development team on a job well done, and give thanks to the many beta testers that gave such detailed feedback. Uwe Ligges, a frequent R contributor and member of the beta test team, had this to say about the project:

“REvolution has worked hard to get R ported to x64 Windows andenables high performance useRs to work on very memory consumingapplications even under Windows. During the time I had the opportunityto test the very well working beta version on the winbuilder (64-bit)machine that is used to build CRAN binary packages for 32-bit R,REvolution developers were extremely responsive and helpful. Packageupdates that were required in order to work with REvolution’s R versionhad been released at REvolution’s binary package repository within aday or two. Congratulations to REvolution for having such a wellworking team of developers!”

On behalf of all of us here at REvolution, I also want to acknowledge and thank the R Core team who created R in the first place (and therefore most of the code in REvolution R). In particular I’d like to thank Brian Ripley and those others who took the lead in porting R to 64-bit systems at a time when 64-bit applications were still rare, giving the R community early access to large-scale data analysis. (Thanks also to Brian Ripley for his detailed comments during the beta test.) I also want to give thanks to all the package developers who have enriched the R ecosystem, and especially those that have helped us during the package porting project.

Based on our experience with R users and the downloads from our CRAN mirror, Windows is the largest installed platform for R, and a 64-bit version of REvolution R Enterprise will help R users process much larger data sets than before. With some tweaking, 32-bit Windows machines can access up to 3Gb of RAM, which in practice limits R on 32-bit Windows to datasets smaller than a gigabyte. On the 64-bit Windows platform those limitations disappear: REvolution R Enterprise there can manage much larger datasets using a greatly expanded virtual-memory address space, and the fact that you can install and use much more than 4Gb of RAM can make those analyses run even faster. (Look for some specific examples in a subsequent post.)

REvolution R Enterprise 2.0 will be available to subscribers tomorrow, April 14. This release will also include a major update to ParallelR (more about that later). For details about subscriptions, please contact our Sales team. Academic pricing is also available.

REvolution Computing: What’s new in REvolution R Enterprise 2.0

TAGGED: analytics, open source, r
DavidMSmith April 13, 2009
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

big data mac performance
Data-Driven Tips to Optimize the Speed of Macs
News
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
Artificial Intelligence
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

supply chain analytics
Analytics

Automotive Industry Uses Analytics To Solve Pressing Supply Chain Issues

6 Min Read
Analytics

How can CIOs Build Business Value with Business Analytics?

8 Min Read
text analytics
Text Analytics

Seven Benefits of Using AI to Perform Text Analysis

9 Min Read
data lineage tool
Big Data

7 Data Lineage Tool Tips For Preventing Human Error in Data Processing

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?