The sentiment on US Economy from Twitter

November 2, 2009
53 Views

Is the economic crisis over? What is the sentiment of people regarding US Economy and the future? These are some of the questions that many people ask these days and the signs are somewhat mixed. Dow Jones is close to the 10000 mark and some US Economy Indices show that the worse is behind. But do people feel the same?

To answer these questions 10000 tweets containing the word economy were collected with the purpose of finding out what people think and how they feel about the US Economy and the economic crisis. The following web chart shows some of the results:


PositiveSentiment is an annotation type that includes all words that suggest positivity such as good, better, advances, while the opposite annotation (NegativeSentiment) exists for all keywords that suggest negativity.

The bolder the lines between words the heavier the association. To get an idea of how people feel, look at the line that connects NegativeSentiment and the word still, which implies that the strongest sentiment is that US Economy is still under big problems.

Some other findings :

  • US President tells that the economy gets better but people don’t feel the same.
  • Economy cannot be getting better while at the same ..



Is the economic crisis over? What is the sentiment of people regarding US Economy and the future? These are some of the questions that many people ask these days and the signs are somewhat mixed. Dow Jones is close to the 10000 mark and some US Economy Indices show that the worse is behind. But do people feel the same?

To answer these questions 10000 tweets containing the word economy were collected with the purpose of finding out what people think and how they feel about the US Economy and the economic crisis. The following web chart shows some of the results:


PositiveSentiment is an annotation type that includes all words that suggest positivity such as good, better, advances, while the opposite annotation (NegativeSentiment) exists for all keywords that suggest negativity.

The bolder the lines between words the heavier the association. To get an idea of how people feel, look at the line that connects NegativeSentiment and the word still, which implies that the strongest sentiment is that US Economy is still under big problems.

Some other findings :

  • US President tells that the economy gets better but people don’t feel the same.
  • Economy cannot be getting better while at the same time there are layoffs.
  • People expressing very negative feelings after losing their jobs.

Notice also the association between NegativeSentiment and people, job, money, sales. Interesting insights can also be found if brand names and product categories are also taken into account: In this analysis a specific brand was found that was associated with word sales and a good overall sentiment. Buying behavior can also be found regarding consumer intentions when the time is right.

You will also find that an association exists between finance_institution keywords (implying keyword Fed) and PositiveSentiment. This association exists because a number of re-tweets is about the Fed signaling the start of exit from recession and its impact on housing. Interesting also is the association between the words fool and annotation PositiveSentiment (…)

Specific tweets were removed such as spam tweets (that try to sell investing products). Re-tweets were kept intact since we are making the assumption that if someone re-tweets – say, a positive sentiment tweet – then he/she also feels the same – positive – sentiment. Tweets that were jokes were identified, marked accordingly and removed.

As with many examples in the past, the software that was used consisted of GATE (for annotating unstructured text from Tweets) but also SPSS Clementine (now PASW Modeller). Here is the setup from GATE :


Specific rules (JAPE) were used that identify and annotate accordingly negative and positive sentiment. Consider the following sentences:

  • The economy is most likely bad at the moment
  • If the economy is great then why so many people can’t find a job?

The first sentence has clearly a negative sentiment since the word bad exists. However the second phrase contains the word great, so a specific matching rule should take into consideration the word If and annotate this phrase as one having negative sentiment despite the presence of word great.

After running GATE, here is how the now-structured data look like from a smaller sample of the original dataset (notice the highlighted record and the IfGood flag) :


With data in a structured form as the one depicted above, we are then ready to identify which tweets were found having a positive or negative sentiment, see erroneous annotations, take corrective actions, and finally analyze the information and extract knowledge from it.

Link to original post

You may be interested

Big Data Revolution in Agriculture Industry: Opportunities and Challenges
Analytics
69 shares1,947 views
Analytics
69 shares1,947 views

Big Data Revolution in Agriculture Industry: Opportunities and Challenges

Kayla Matthews - July 24, 2017

Big data is all about efficiency. There are many types of data available, and many ways to use that information.…

How SAP Hana is Driving Big Data Startups
Big Data
298 shares3,200 views
Big Data
298 shares3,200 views

How SAP Hana is Driving Big Data Startups

Ryan Kh - July 20, 2017

The first version of SAP Hana was released in 2010, before Hadoop and other big data extraction tools were introduced.…

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion
Data Management
156 views
Data Management
156 views

Data Erasing Software vs Physical Destruction: Sustainable Way of Data Deletion

Manish Bhickta - July 20, 2017

Physical Data destruction techniques are efficient enough to destroy data, but they can never be considered eco-friendly. On the other…