The latest in the O’Reilly “Short Cuts” series, and the first devoted to R, is Data Mashups in R. Written by Jeremy Leipzig and Ziao-Yi Lu, this 30-page article is an excellent and very practical example of integrating messy data from varied sources, using R or REvolution R.
It’s not designed as a manual-style introduction to R. But while working through the fully-detailed example it presents, even programmers unfamiliar to R will get a good sense of the practical capabilities of R when working with real-life data sources. “Learning by doing” is a great way to bootstrap your knowledge of any language, and a concrete, practical example like this really helps with that process.
The example used is a particularly timely one: how to automate the process of downloading foreclosure data from a public website, and presenting it in graphical form, like this:
In practical terms, creating this graphic necessitates the integration of data from various sources:
- Downloading an HTML file of foreclosed addresses from a public web-site (with download.file) and extracting addresses in messy formats (with grep);
- Downloading geolocation data from a Yahoo web-service and parsing the …
The latest in the O’Reilly “Short Cuts” series, and the first devoted to R, is Data Mashups in R. Written by Jeremy Leipzig and Ziao-Yi Lu, this 30-page article is an excellent and very practical example of integrating messy data from varied sources, using R or REvolution R.
It’s not designed as a manual-style introduction to R. But while working through the fully-detailed example it presents, even programmers unfamiliar to R will get a good sense of the practical capabilities of R when working with real-life data sources. “Learning by doing” is a great way to bootstrap your knowledge of any language, and a concrete, practical example like this really helps with that process.
The example used is a particularly timely one: how to automate the process of downloading foreclosure data from a public website, and presenting it in graphical form, like this:
In practical terms, creating this graphic necessitates the integration of data from various sources:
- Downloading an HTML file of foreclosed addresses from a public web-site (with download.file) and extracting addresses in messy formats (with grep);
- Downloading geolocation data from a Yahoo web-service and parsing the XML result (with xmlTreeParse);
- Downloading an ESRI shape file of Philadelphia and its census tracts, and plotting a map (with the maptools package);
- Matching the individual addresses to census tracts and counting the number of foreclosures in each (with plotPolys).
One of the great things about the article is that it’s not presented as a theoretical exercise where everything just works the first time. It takes pains to cover not just how to accomplish the task itself, but also how to deal with problems you’re likely to encounter along the way (especially if you adapt the methods to your own needs). Practical advice is given on how to deal with connection problems caused by firewalls and proxies, how to deal gracefully with the failure of servers to respond (using tryCatch), and how to detect error messages from the Yahoo web service when it fails to recognize addresses. The authors also let you know where they found the information of how to deal with such problems, so it also serves as a guide for finding help resources in R.
My only criticisms are minor ones. The article leads the user through downloading Census data, matching it to the tracts, and performing some simply summary statistics and exploratory graphics, but stops short of any actual statistical analysis. And cutting-and-pasting R code from a PDF document is a pain: there’s no associated script file (as far as I know).
But for experienced R users looking for tips on integrating messy data sources from the Web, or for programmers new to R looking for a practical example to work through as an introduction to R, Data Mashups in R is well worth $4.99 to download.
O’Reilly Short Cuts: Data Mashups in R