By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Assisted Insight: The Future of Data Discovery
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Visualization > Assisted Insight: The Future of Data Discovery
Business IntelligenceData ManagementData Visualization

Assisted Insight: The Future of Data Discovery

Trevor Lohrbeer
Last updated: 2014/08/01 at 9:07 PM
Trevor Lohrbeer
7 Min Read
Image
SHARE

ImageBusinesses don’t want “data discovery”, “visualization” or “analytics” tools – businesses want one thing: insight.

The challenge has been how to provide that insight. Twenty years ago the processing power of machines was limited. To overcome this limitation, traditional business intelligence providers created cubes and OLAP.

ImageBusinesses don’t want “data discovery”, “visualization” or “analytics” tools – businesses want one thing: insight.

The challenge has been how to provide that insight. Twenty years ago the processing power of machines was limited. To overcome this limitation, traditional business intelligence providers created cubes and OLAP.

More Read

ai in ppc advertising

5 Proven Tips for Utilizing AI with PPC Advertising in 2023

5 Ways AI Technology Has Disrupted Website Development
Four Strategies For Effective Database Compliance
Fortifying Enterprise Digital Security Against Hackers Weaponizing AI
Data Visualization Boosts Business Scalability with Sales Mapping

Over the past decade, those limitations have lifted and interactive data visualizations tools that use in-memory data models have blossomed. The paradigm shifted, and with it, new powerhouses like Tableau Software have arisen.

In the next decade an equally profound shift will occur. As processing power increases and cloud computing takes hold, new capabilities will emerge—new capabilities that will drive new paradigms. One of those will be Assisted Insight.

What is Assisted Insight?

Assisted insight is the process of presenting users with computer-generated insights in a way that guides them when analyzing data. Using a combination of framing, algorithms and visualizations, assisted insight will accelerate the discovery of new insights and uncover insights that previously remained hidden.

With new user interfaces, users will frame their problem in business terms instead of mathematical terms. This will allow algorithms to dig deeper and visualizations to be presented to maximize new insights. If users enter their assumptions and expectations, data discovery software can use that framing to find the unexpected.

By combining this framing with statistical and machine learning algorithms, data discovery software will create a list of potential insights, ranked by confidence, for a user to explore. With the massive amount of computing power users have at their disposal today, thousands of potential data combinations can already be explored by algorithms in the same time it would take a user to explore one manually.

Once insights have been identified, the framing and nature of the insight will be used to configure interactive visualizations that communicate those insights to the user using best practices. Users will explore these visualizations to verify the insight and gain a nuanced understanding of it.

Finally, users will use the visualizations to refine the insights by changing the parameters of the analysis and dig deeper into the data. A cycle emerges of the user refining their framing of the problem, software proposing potential insights, and users exploring & refining those insights further.

Framing the Problem

Framing refers to how a user defines their analysis. Traditionally, data analysis is done using a “mathematical” frame. Metrics are low or high, statistics like averages and medians are calculated (assuming distributions are bell-shaped) and the user is presented with a series of numbers that they need to translate into their problem space.

By framing the analysis using a “problem” frame, a user can work directly in the concepts they use to drive their business: metrics are “good” or “bad”, distributions are “normal” or “abnormal”, values are “expected” or “unexpected”, and the user is presented with visualizations configured to communicate the conceptual space of the problem.

Generating Insights with Algorithms

Algorithms can help users quickly explore potential combinations and correlations within their data to find unknown trends, anomalies and relationships. But algorithms are only as good as their input—garbage in, garbage out.

By adding the meaning of the data—the assumptions, expectations and known relationships—into the analysis, algorithms can help users find business insights, not mathematical insights.

Using Visualization to Refine Insights

Visualization helps make data consumable. By using colors, shapes and other visual aspects that align with the meaning of the data, comprehension of insights is enhanced. Users can be shown insights faster by highlighting potential insights and being given natural language explanations.

But not all potential insights will be valuable. Most won’t be.

Visualizations can give the context of an insight to a user so they gain a nuanced understanding and a spring pad for further exploration. Through direct manipulation and other techniques, users can refine not only the visualization itself, but the framing of the problem. This new framing can be used to drive the algorithms to create new potential insights.

Assisted Insight Today

At present, no one has combined all these elements to create true assisted insight. But companies are moving fast to develop these capabilities.

IT operations analytics vendors like Netuitive have developed behavior learning engines that use algorithms to learn what is “normal” vs “abnormal” in real-time, machine-generated data. Using a constrained problem space allows them to build the framing of the problem directly into their software. Making this generic requires new user interfaces to allow users to enter their framing, and new algorithms for data problems that don’t map easily onto the IT operations analytics space.

Business intelligence startups have started developing “automated insight” functionality by using algorithms to find potential insights and automatically configure visualizations for those insights.

Within data discovery, vendors have focused on natural language queries and storyboarding—but these don’t help uncover new insights. Natural language queries speed the process of configuring a visualization once a user has an idea what to look for, but do nothing to help users find hidden insights. Story-boarding helps to communicate insights once they are found, not to generate new ones.

The Future of Assisted Insight

It will take several more years before assisted insight becomes a standard feature of business intelligence and data discovery suites. The vendors who develop it early will have the edge. They’ll have had more time to improve their user interfaces, refine their algorithms and develop new visualizations.

But the pieces are there, just waiting for someone to put them together. Over the next few months, I’ll be writing about those pieces and how to combine them. Follow me to get notified about future posts.

Trevor Lohrbeer August 1, 2014
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

ai in ppc advertising
Artificial Intelligence

5 Proven Tips for Utilizing AI with PPC Advertising in 2023

10 Min Read
ai in web design
Artificial Intelligence

5 Ways AI Technology Has Disrupted Website Development

7 Min Read
database compliance guide
Data Management

Four Strategies For Effective Database Compliance

8 Min Read
Digital Security From Weaponized AI
Security

Fortifying Enterprise Digital Security Against Hackers Weaponizing AI

11 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?