# Looking at Trees to Understand the Forest

> David Simon (of The Wire fame) has sucked me into another brilliant television series with Generation Kill. It is the story of a Marine recon unit at the beginning of the Iraq war. At the heart of all the action, the seven-part…
> URL: https://www.smartdatacollective.com/25594/
> Last modified: 2010-03-19T07:06:00+00:00

David Simon (of [The Wire](http://en.wikipedia.org/wiki/The_Wire) fame) has sucked me into another brilliant television series with [Generation Kill](http://en.wikipedia.org/wiki/Generation_Kill_%28TV_series%29). It is the story of a Marine recon unit at the beginning of the Iraq war. At the heart of all the action, the seven-part miniseries offers an intimate and honest profiles of individual Marines.

The characters don’t so much displace stereotypes as reveal texture and insight about the unique qualities of individual Marines.

		
							
					
*
				
					
		
The series got me thinking once again about different ways to analyze data. Almost four years ago, I posted a couple blog posts ([Part 1](http://www.juiceanalytics.com/writing/customer-sparklines/) and [Part 2](http://www.juiceanalytics.com/writing/a-missing-link-in-business-analytics-part-2/)) making a case for analyzing and visualizing data at a granular level to uncover patterns and behaviors. Generation Kill is a case study in looking closely at the individual trees to understand the forest.

			
				
#### More Read

				
							
					
						
					[](https://www.smartdatacollective.com/real-time-analytics-and-storm-tracking-how-utilities-can-keep-customers-out-dark/)
				
						
					[Real-Time Analytics and Storm Tracking: How Utilities Can Keep Customers Out of the Dark](https://www.smartdatacollective.com/real-time-analytics-and-storm-tracking-how-utilities-can-keep-customers-out-dark/)
				
				
		*		[How Your Hadoop Distribution Could Lose Your Data Forever](https://www.smartdatacollective.com/how-your-hadoop-distribution-could-lose-your-data-forever/)
				
				
		**		[How Amazon Has Shaped the Big Data Landscape](https://www.smartdatacollective.com/how-amazon-shaped-big-data-landscape/)
				
				
		**		[Why Overfitting is More Dangerous than Just Poor Accuracy [PART 1]](https://www.smartdatacollective.com/why-overfitting-more-dangerous-just-poor-accuracy-part-i/)
				
				
		**		[Tipping Points](https://www.smartdatacollective.com/22587/)
				
						
			
		
Analytics is a journey of exploration–a continuous series of iterations with the goal of deeper understanding based on better questions and more targeted analyses. Einstein said:

“To raise new questions, new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advance in science.”

How to arrive at new questions?

In the previous blog post, I described examples from online learning, credit cards usage, and football film study to show how granular analysis can spur new questions. I’ve stumbled across a series of new examples recently:

		
							
					

				
					
		
**Surveys.** Survey analysis is hard work–just ask Ken who recently presented [results](http://www.juiceanalytics.com/writing/survey-results-are-viz-pundits-really-helping/) from Juice’s survey on the practice of information visualization in organizations. If a survey is mostly about understanding your audience, rolling up responses by questions can’t be the only approach (though it is the most common). [Cross tabs](http://en.wikipedia.org/wiki/Cross_tabulation) (“displays the joint distribution of two or more variables”) are one direction to go. Another approach is to look for people who share common characteristics or patterns in their responses.

Macrofocus’ [SurveyVisualizer](http://macrofocus.com/public/products/surveyvisualizer/) is the most innovative survey analysis tool I’ve seen and it emphasizes data at a granular level.

“All the analysis elements are always shown as grey lines in the background. This provides an overview of the ranges and spreads of the individual values for each node, and facilitates the detection of outliers.” (from Visualization of Large-Scale Customer Satisfaction Surveys Using a Parallel Coordinate Tree)

**Medical research.** Research studies are conducted against carefully defined target and control populations with aggregate statistics across these populations required for conclusions. However, the ability to review the patterns of diagnoses and procedures at the individual patient-level can help test assumptions about the target population and refine the parameters of a study. Better model inputs; better results.

		
							
					

				
					
		
**Speech analytics.** Michel Guillet at [Nexidia](http://www.nexidia.com/) recently told me about their approach to speech data:

Nexidia’s speech analytics can mine thousands of hours of audio to categorize, correlate or spot trends. However, it is quite often in identifying and listening to a lone outlier that the application provides its most valuable insights. Some examples of outliers can be the very long call of a particular call type, the extremely abrupt one, the one with the most languages spoken or the one where  no one is speaking at all. An outlier can change your hypotheses and put you in a different direction…perhaps a better one.  Nexidia’s reporting and analysis tools offer many different methodologies including histograms, analysis of means charts and flexible filtration by meta-data to identify outliers in large amounts of data. In addition, Nexidia’s ad-hoc search functionality allows users to search an entire body of audio content at any time, which is often helpful to find the “smoking gun” or a single recording which can make or break an argument.

Of course you can’t be assured of a full or accurate picture when looking at granular data, but somewhere between standard aggregation-based analysis and granular views lies the truth.