By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Visualizing Lexical Novelty in Literature
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Visualization > Visualizing Lexical Novelty in Literature
Data VisualizationText Analytics

Visualizing Lexical Novelty in Literature

matthewhurst
Last updated: 2011/09/19 at 7:45 AM
matthewhurst
4 Min Read
SHARE

Novels are full of new characters, new locations and new expressions. The discourse between characters involves new ideas being exchanged. We can get a hint of this by tracking the introduction of new terms in a novel. In the below visualizations (in which each column represents a chapter and each small block a paragraph of text), I maintain a variable which represents novelty. When a paragraph contains more than 25% new terms (i.e. words that have not been observed thus far) this variable is set at its maximum of 1.

Novels are full of new characters, new locations and new expressions. The discourse between characters involves new ideas being exchanged. We can get a hint of this by tracking the introduction of new terms in a novel. In the below visualizations (in which each column represents a chapter and each small block a paragraph of text), I maintain a variable which represents novelty. When a paragraph contains more than 25% new terms (i.e. words that have not been observed thus far) this variable is set at its maximum of 1. Otherwise, the variable decays. The variable is used to colour the paragraph with red being 1.0 and blue being 0. The result is that we can get an idea of the introduction of new ideas in novels.

In the first book – Austen’s Sense and Sensibility – we can see two things. Firstly, the start of the book keeps a pretty good degree of novelty for the first few chapters. Secondly, each chapter introduces something new.

The second book – Stevenson’s Kidnapped – shows a different pattern. While it starts off with reasonable novelty, this then dies out for most of the book with spurts of interest here and there.

More Read

data visualization for small business

Data Visualization Boosts Business Scalability with Sales Mapping

5 Best Practices for Extracting, Analyzing, and Visualizing Data
Seven Benefits of Using AI to Perform Text Analysis
10 Important Ways Data Visualization Can Benefit Your Content Strategy
How to Bring Presentation Data to Life with Powered Template

What is surprising to me (if we take any real meaning from this approach) is that Austen’s Emma – the third book – is strong out of the gate (the first 18 chapters) but fails to break the 25% novelty ceiling thereafter.

[Note that these results are preliminary and I’m going to do more validation and testing.]

Update: see below the original visualization for an updated version with more accurate results.

LexicalNovelty 
Update: After looking at the above results I drilled down on the strange behaviour in Emma. It turns out that Emma as multiple volumes within which the chapter counter reset to I. Consequently I was picking up chapter titles (I, II, III, VI, V, etc.) as novel terms the first go round and this was driving the visualization. I’ve since modified the algorithm to firstly ignore text blocks (paragraphs) with fewer than 5 words and secondly, given it a more dynamic colour scheme.

This improvement still highlights some key differences (again, in as much as the algorithm is correct). However, these differences are now somewhat changed from the first set of observations. Note also that the threshold for novelty has been decreased to 0.1.

LexicalNovelty2 

matthewhurst September 19, 2011
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

data visualization for small business
Data Visualization

Data Visualization Boosts Business Scalability with Sales Mapping

7 Min Read
big data visualization
Data Visualization

5 Best Practices for Extracting, Analyzing, and Visualizing Data

6 Min Read
text analytics
Text Analytics

Seven Benefits of Using AI to Perform Text Analysis

9 Min Read
Data Visualization
Data Visualization

10 Important Ways Data Visualization Can Benefit Your Content Strategy

13 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence
ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?