SDC @ Strata – Doug Cutting on Hadoop, Apache and Open Source

Doug Cutting addressed the Wednesday plenary audience for 10 minutes and spoke about Hadoop. He carries the title of “Architect” on his bio at Strata. It goes on as follows:

Doug is a founder of several Apache open source projects, including Lucene, Nutch, Hadoop and Avro. Doug currently works for Cloudera, and previously worked at Yahoo!, Excite, Apple and Xerox PARC. Doug holds a Bachelor’s degree from Stanford University and presently chairs the Board of the Apache Software Foundation.

Doug Cutting addressed the Wednesday plenary audience for 10 minutes and spoke about Hadoop. He carries the title of “Architect” on his bio at Strata. It goes on as follows:

Doug is a founder of several Apache open source projects, including Lucene, Nutch, Hadoop and Avro. Doug currently works for Cloudera, and previously worked at Yahoo!, Excite, Apple and Xerox PARC. Doug holds a Bachelor’s degree from Stanford University and presently chairs the Board of the Apache Software Foundation.

He described how hardware has evolved rapidly to deal with big data, making possible multi-nodesystems coordinated by software. Huge lots of raw data can now be stored and manipulated with largely open source software tools. 

As Doug declared, “This ecosystem is now The System.”

Hadoop, he says, has become the kernel- part of the distributed system for Big Data. Other elements are being added to Hadoop

The Apache Software Foundation is a non-profit organization hosting about 100 different projects. It tries to promote the development of healthy communities around projects. Quality and usage, Doug maintains, are present in healthy communities. As such, Apache allows – indeed encourages – competing products to be developed. Apache is a loose federation of projects, none of which are critical. The way they roll, any product can be dismissed if something better is developed. 

Free software, Doug said, encourages experimentation.

He would like to hear more about how people are using Hadoop, and what ideas they have found to be impractical without Hadoop. 

“What data are you ignoring?” – Doug wants to know.