By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: How to Program MapReduce Jobs in Hadoop with R
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Software > Hadoop > How to Program MapReduce Jobs in Hadoop with R
AnalyticsHadoopR Programming Language

How to Program MapReduce Jobs in Hadoop with R

DavidMSmith
Last updated: 2011/09/13 at 5:50 PM
DavidMSmith
3 Min Read
SHARE

MapReduce is a powerful programming framework for efficiently processing very large amounts of data stored in the Hadoop distributed filesystem.

MapReduce is a powerful programming framework for efficiently processing very large amounts of data stored in the Hadoop distributed filesystem. But while several programming frameworks for Hadoop exist, few are tuned to the needs of data analysts who typically work in the R environment as opposed to general-purpose languages like Java.

That’s why the dev team at Revolution Analytics created the RHadoop project, to give R progammers powerful open-source tools to analyze data stored in Hadoop. RHadoop provides a new R package called rmr, whose goals are:

  • To provide map-reduce programmers the easiest, most productive, most elegant way to write map reduce jobs. Programs written using the rmr package may need one-two orders of magnitude less code than Java, while being written in a readable, reusable and extensible language.
  • To give R programmers a way to access the map-reduce programming paradigm and way to work on big data sets in a way that is natural for data analysts working in R.

Together with its companion packages rhdfs and rhbase (for working with HDFS and HBASE datastores, respectively, in R) the rmr package provides a way for data analysts to access massive, fault tolerant parallelism without needing to master distributed programming. By providing an abstraction layer on top of all of the Hadoop implementation details, the rmr package lets the R programmer focus on the data analysis of very large data sets.

More Read

Hadoop vs Spark

Big Data New Age: Hadoop vs Spark

100 Petabytes of Data in Poop?
The Fallacy of the Data Scientist Shortage
What Is a Data Scientist (and What Isn’t)?
Ring in the New Year with New Data Products

If you want to get started with MapReduce programming in R, this tutorial on rmr shows simple equivalents to the R functions lapply and tapply in map-reduce form. It also gives some simplified, but practical examples of doing linear and logistic regression and k-means clustering via map-reduce. For more advanced map-reduce programmers, these pages on efficient rmr techniques and writing composable mapreduce jobs will also be of interest.

The rmr package is available for download from the github repository under the open-source Apache license, and we encourage other Hadoop developers to get involved with the RHadoop project.

Note: As an introduction to the RHadoop, project lead Antonio Piccolbini will join Revolution Analytics CTO David Champagne for a webinar Wednesday, September 21. Register here for a live introduction to the rmr package and how to use it to analyze big data sets within the map-reduce framework.

githib: Revolution Analytics RHadoop Project

TAGGED: MapReduce
DavidMSmith September 13, 2011
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

Hadoop vs Spark
Big DataHadoopMapReduceProgramming

Big Data New Age: Hadoop vs Spark

5 Min Read

100 Petabytes of Data in Poop?

6 Min Read

The Fallacy of the Data Scientist Shortage

8 Min Read

What Is a Data Scientist (and What Isn’t)?

7 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?