Microsoft Focuses Big Data Efforts on Hadoop

Microsoft has shown once again that it can act decisively and smartly improve its offerings in doing so. It has “disrupted” itself in a way that will be positive for the community and probably the Microsoft bottom line.

In a November 11 blog post, Microsoft announced that it would not be moving forward with a production release of LINQ to HPC, something that was an alternative distributed Big Data platform. Codenamed Dryad, LINQ to HPC would have been Microsoft’s proprietary alternative to Hadoop, designed to run Big Data jobs on clusters of Windows servers. Instead, they are focusing their efforts on developing Hadoop distributions for Windows Servers and Azure, a platform for building, hosting, and scaling applications in Microsoft datacenters.

The Microsoft post read in part:

Hadoop has emerged as a great platform for analyzing unstructured data or large volumes of data at low cost, which aligns well with Microsoft’s vision for its Information Platform.  It also has a vibrant community of users and developers eager to innovate on this platform.  Microsoft is keen to not only contribute to this vibrant community, but also help its adoption in the Enterprise.  We expect a preview version on Windows Azure available by end of the calendar year.

In the post, Microsoft said that it chose to stick with Hadoop because of its proven low cost, high performance, and vibrant community. Microsoft’s Hadoop stacks will be open sourced through the Apache Software Foundation like the original code and will work with Microsoft’s business-intelligence tools such as Excel, PowerPivot, and PowerView. With its contributions, Microsoft hopes to broaden Hadoop’s adoption in the enterprise by simplifying the downloading, installation, and configuration of members of the Hadoop ecosystem including HDFS, Hive, and Pig. Microsoft also plans to enable users to write MapReduce jobs in Javascript. Expect a test build of Hadoop for Windows Azure by the end of the year and one for Windows Server in early 2012.