Discussion of Big Data in the Geospatial Intelligence Domain

The US Geospatial Intelligence Foundation (USGIF) held a Geoint Community Week June 4-8 2012 which included a technology day. The afternoon of 07 June was dedicated to fast-paced “ignite” style presentations from some of the key community thought leaders on geospatial data. At the end of these presentations I was asked to summarize and discuss the gist of the many presentations and to provide a briefing capturing key thoughts.

Some issues that came out of these discussions:

1) As long predicted, something about human nature causes people to latch onto great terms like “Big Data” and then some seek to twist the meaning to mean what they want it to. One thing we discussed at this event is that the community needs to decide what they want the term to mean and then enforce some discipline and rigor in its use. Our recommendation: keep using the definition in Wikipedia, and use that platform to suggest changes to the definition as required so a wide swath of the community can think through its meaning.

2) The ignite format of presentations for this sort of capability worked well since it forced us all to refine what we wanted to talk about.

3) In general, participants in the presentations and in the discussions afterwards all hit on the fact that whatever you call “it” is not the important thing. What “it” is is the improved ability to use more data to influence information, and you can call the successful new approaches to doing that just about anything you want.

4) Although there are many capabilities that “support” Big Data approaches, if we call everything that has to do with modern data “Big Data” then the term will lose all meaning fast. There was not total agreement in the group about this, but my favored approach is to say “Big Data” means new approaches primarily based around new open source frameworks like Apache Hadoop and the many related capabilities (Hive, Hbase, etc).