By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Crossing the Language Chasm: Extracting Information from Foreign-Language Text
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Data Management > Best Practices > Crossing the Language Chasm: Extracting Information from Foreign-Language Text
AnalyticsBest PracticesData MiningExclusiveMarket ResearchPredictive AnalyticsSentiment AnalyticsText AnalyticsUnstructured DataWeb Analytics

Crossing the Language Chasm: Extracting Information from Foreign-Language Text

metabrown
Last updated: 2019/03/28 at 8:11 PM
metabrown
6 Min Read
SHARE

Just a few years ago, text analytics was unknown to most business people, even most data analysts. Today, more and more organizations are not only aware of text analytics, but trying it themselves. With increasing awareness comes expanding expectations: identifying the subject matter of text isn’t enough, the pressure is on to identify sentiment, to find significance in tiny snips of text, to mine massive quantities of text in real time, and to analyze text written in languages that the analyst cannot read.

Think about that last item one moment: analyze text written in languages the analyst cannot read.  

The main drive behind text analytics so far has been speed – conquering large quantities of text that human investigators simply don’t have time to read and interpret. That’s about to change. As text analytics enters the toolkits of businesses and governments, they are asking for something more – cross-lingual text analytics, tools that enable analysts to mine text written in languages that they themselves do not understand.

Who wants cross-lingual analytics?

More Read

data analytics in sports industry

Here’s How Data Analytics In Sports Is Changing The Game

Advances in Data Analytics Are Rapidly Transforming Nursing
Data Analytics Technology Proves Benefits of an MBA
Anti-Spoofing is Crucial for Data-Driven Businesses
3 AI-Based Strategies to Develop Software in Uncertain Times

According to this translation company in Budapest, market researchers from Western countries studying consumer preferences in Asia, and vice-versa.

English speakers in the U.S. interested in the political concerns of recent immigrants.

Intelligence agencies in Freedonia searching for signs of terrorist activity in Moosylvania.

… And so on.

How are they going about this? Some are translating the text to their own language and then applying their usual text analysis process to the translated text. That approach has major problems. If you’ve ever read machine translated text, you know it’s less than perfect, much less than perfect. If text mining is hard with text in its original form, imagine the quality issues that crop up when the text has been machine translated. What’s more, even machine translation can be costly when applied to massive quantities of text. High cost for poor quality is bad news; the good news is that there is better way to go about it.

Cross-lingual text analytics is more accurate and less costly when as much of the process is performed in the original language of the text as possible. Keep translation to a bare minimum, and use tools built for working with the language you require. Read this outline of the process, and notice how translation is minimized:

The Cross-Lingual Text Analytics Process

Define the topic .

What is the subject matter of the investigation? If the topic is, say, soft drinks, what terms are used to describe soft drinks in English? The list must be complete – so it would include not just the term “soft drinks,” but also “soda,” “pop,” “lemonade” and many others. Some of the terms will be brand names, others slang. And the specific sense of the each term must be identified; you are interested in soda as a fizzy drink, not an ingredient for baking.

Translate the topic

When a thorough and unambiguous definition of the subject matter is complete, it is translated to the target language.

Search

Translated terms form the basis for a text search to identify and retrieve relevant text.

Categorize

Working within the original language of the text, extract information such as subject matter and sentiment in the text. This information can be used to categorize or score text as required.

Summarize

Summarization includes tasks such as calculating the proportion of the text mentioning specific terms or estimating the percentages of positive and negative sentiment.

Tag

Tagging is the annotation of text, adding codes to denote specific functional parts such as titles or names.

Translate text

When all other steps have been performed, the analyst may choose to translate some portion of the original text for examination.

(Note that many investigations do not require all of these steps.)

That’s the right way to go about it. Carefully define and translate search concepts, rather than sloppily translate a mass of text. Use the most efficient search process available. Evaluate text in its original form, not a poorly translated version. And translate only a minimal quantity of relevant text, or none at all. This cross-lingual text analytics process is efficient and accurate.

In a climate of global interaction and competition, much of the information we need is embedded in languages we don’t understand. Extracting that information is not optional, it’s a necessity. Cross-lingual text analytics is in your future, perhaps your near future. Go about it in the wrong way, and you’ll waste resources and produce nothing of value. Understanding and using the right process maximizes the chance of success for your project and your organization.

More on text analytics:

©2011 Meta S. Brown

metabrown October 25, 2011
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

data analytics in sports industry
Big Data

Here’s How Data Analytics In Sports Is Changing The Game

6 Min Read
data analytics on nursing career
Analytics

Advances in Data Analytics Are Rapidly Transforming Nursing

8 Min Read
data analytics reveals the benefits of MBA
Analytics

Data Analytics Technology Proves Benefits of an MBA

9 Min Read
anti-spoofing tips
Security

Anti-Spoofing is Crucial for Data-Driven Businesses

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data
AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?