Benchmarking Revolution R for Data Mining

The blog Heuristically Andrew puts Revolution R through its paces by running some benchmarks versus open-source R for data mining applications. The benchmarks set out to answer the following question:

I recently upgraded my notebook (where I often use R for data mining) and was faced with two questions: for the fastest speed for building models, do I use the R or Revolution R, and do I enable Hyper-Threading?

The post includes benchmark code for several computations in both R and Revolution R, with and without hyperthreading enabled, and the results are summarized in the following chart:

The height of each bar is execution time, so a smaller bar is better. First thing to notice is that using hyper-threading or not doesn’t make a whole lot of difference. (In each group, compare bar 1 with bar 2, and bar 3 with bar 4: not much difference.) That’s in line with my experience, too: while hyper-threading can make one processor appear as two processors to the operating system, I’ve never seen it improve the performance of computations that would normally consume nearly 100% of the processor anyway.

As shown by these benchmarks, Revolution R offers a nice performance boost for building tree models with the ctree function. This is a function from a user-written CRAN package (party), and I’d guess that the underlying code makes extensive use of native R operations like those in our R peformance benchmarks that benefit from the parallel MKL math library linked with Revolution R. (It’s also possible that underlying C or C++ code calls level-2 BLAS functions directly, which would also benefit.) If a package doesn’t make much use of such matrix operators (like the earth function for MARS models), you won’t much of a speedup.

The k function — a simple, tight arithmetic loop that doesn’t do any heavy lifting with matrices — has is of the same class. It was interesting to see the benefits of the new compile function in R 2.13: the lk function is a byte-compiled version of k, and byte-compilation offers some nice benefits. Dirk Eddelbuettel ran some similar tests soon after R 2.13.0 was released. (Byte-compilation isn’t currently available in Revolution R, which will upgrade to R 2.13 after the final 2.13.x patch is released by the Core Group.)

The post concludes:

I already used Revolution R on Amazon EC2, and for now, I plan to switch to Revolution R on Windows too. I’ll leave HTT enabled in case it helps other tasks.

If you’d like to do the same, you can purchase Revolution R Enterprise online, or download it for free if you’re at an academic institution. Check out the post at the Heuristically Andrew blog for the full details and code of this interesting benchmarking exercise.

Heuristically Andrew: Benchmarking R, Revolution R, And HyperThreading For Data Mining