February 8, 2023
In this article, we unpack Sentiment analysis - what is it, what are the different approaches to Sentiment analysis, why do we need it and why is it gaining momentum.
In this article, you will read about:
Sentiment analysis is the process of determining whether textual data conveys a particular attitude, which can be positive, negative or neutral. It is also a way to assess to what degree an event can have a negative or positive impact in the course of future developments.
The technology that enables sentiment analysis is Natural Language Processing (NLP) - a branch of Language AI that trains computers to analyze large amounts of textual data and extract meaning from it. Thanks to NLP, machines can “comprehend” if a text about an entity (such as a company, a person, a currency) conveys a particular sentiment. It is also capable of identifying events (an executive hire or a quarterly results disclosure) and assessing the sentiment around that event, as reflected by the text, using sentiment scores . What is truly remarkable is that NLP is able to do this assessment at scale and in real time .
In marketing and communications , sentiment analysis is often used to understand the public perception of a certain topic, such as a product or political candidate, by analyzing large amounts of social media data. Sentiment analysis can also be used to identify customer sentiment about a particular product or brand.
In the finance and investment world , sentiment analytics refers to quantifying subjective information resulting from financial textual data, such as news articles, earnings call transcripts or job postings. The goal is to have a quantified view of the perceptions about a particular stock, sector, market or an event (a merger, an earnings release) and then systematically use this information to make more informed investment decisions, manage risk, predict market trends or macro indicators.
Traditional approaches are based on heuristic methods, which involve fixed rules created by humans. Modern ones are less heuristic and more machine-driven.
Traditional Natural Language Processing
This approach involves the use of NLP to identify positive and negative words and phrases in the text. It uses a Rule Base that assigns specific words, phrases, combinations and other word-level definitions to predefined sentiment values.
The expert consensus approach
Expert consensus combines complex NLP event detection models with human-produced sentiment values.
It is a technique used to determine the ranges of sentiment scores for a probable event (acquisitions, product launches, fall or rise in earnings). It involves surveying a panel of industry experts to get them to rate events. The goal is to determine if they are positive or negative and the expected impact they would have on a market or stock.
Expert consensus models produce sentiment scores determined by systematically matching events, typically categorized by financial experts as having a short-term positive or negative financial or economic impact.
The Deep Learning approach
This is the most advanced approach to sentiment analysis, based on deep neural networks - incredibly complex models that digest vast amounts of inputs all at once. Essentially, computers are being taught to recognize not only positive or negative words but also make sense of the context and semantics . Moreover, these language models are able to make predictions based on patterns learned from experience.
In order to be reliable, models need to be trained by using millions of sentences. So the quality of the Sentiment analysis will depend on the quality of the datasets that models are being trained on, as well as on the accuracy and completeness of the taxonomy.
Approaches to Sentiment analysis often overlap
There’s not one size fits all. Natural language is complex, and the meaning of words and phrases can vary depending on the context in which they are used. This complexity makes it challenging to develop a single, universal method for sentiment analysis. Hence, multiple approaches coexist and often overlap. For instance, some models combine traditional NLP and modern ML capabilities with sentiment analysis based on expert consensus. For instance, an NLP engine would scan textual data to identify events and then automatically match them with the expert consensus evaluation grid. Then, through deep learning technology, Sentiment Models are able to learn from the experts’ evaluations. They are able to do so by using vast training sets and purpose-built taxonomies.
In the financial industry, investment opportunity windows rarely extend beyond a few days, and with dozens of models running concurrently, the number of triggers to buy or sell (called signals) can quickly become overwhelming.
Earnings season is perhaps one of the best examples to illustrate that. Every quarter, companies disclose their earnings reports. In a matter of hours, traders need to quickly process vast amounts of new information and react.
In practice, it’s impossible for them to cover the entire flow of information, all the more that multiple sources, from actual transcripts to press reports, may convey slightly different perspectives.
Sentiment analysis empowers traders to assess unstructured content from multiple sources at scale, and fast. This helps them make more informed decisions, capture alpha and tackle emerging risks. It is akin to an auto-pilot in charge of reviewing large amounts of data and collecting insights to deliver an actionable brief to decision makers.
As more and more organizations see the immense value behind tracking and leveraging opportunities in real time, sentiment analysis has seen significant traction. This trend is expected to continue, being fueled by significant advancements in NLP and ML, coupled with a wealth of insightful data sets available, such as earnings transcripts, jobs data or news content.
Traders, in particular, have found sentiment analysis to be a valuable tool for making informed trades. By systematically embedding sentiment analytics data, they are gaining an edge in anticipating how the market is likely to move. Sentiment analysis has become an essential instrument in any modern investor's arsenal.
Professor of Finance
Georgia Southern University
Individuals are prone to rely on stories that simplify relevant relationships.
Sentiment analysis falls into what Nicholas Mangee, Professor of Finance Georgia Southern University calls “stories that simplify” in his recent book “How Novelty and Narratives Drive the Stock Market: Black Swans, Animal Spirits and Scapegoats”:
“When there is unforeseeable structural change, investors face inherent ambiguity contemplating both the odds of possible future states and the most appropriate model for forecasting returns conditional on their occurrence. In doing so, individuals are prone to rely on stories that simplify relevant relationships for cognitive tractability, particularly those stories infused with the greatest emotions. Psychological considerations and feelings give narratives life and shape that provide investors with the perception of a more complete picture of the process driving outcomes when relationships have changed in unforeseeable ways. Given the interconnected complexities of capitalistic markets and investors´ imperfection of knowledge, real-world decision-making would be paralyzed otherwise.”
Chief Data Scientist
RavenPack
On top of direction, sentiment also provides information about signal strength.
Peter Hafez, Chief Data Scientist at RavenPack talks about Sentiment as an indicator of direction and signal strength:
“While news volume can provide a proxy for importance, it doesn’t provide any directional information. For that, you need sentiment. On top of direction, sentiment also provides information about signal strength. We typically see that more extreme sentiment stocks outperform less extreme sentiment stocks - which adds that extra strength component.”
Please use your business email. If you don't have one, please email us at info@ravenpack.com.
By providing your personal information and submitting your details, you acknowledge that you have read, understood, and agreed to our Privacy Statement and you accept our Terms and Conditions. We will handle your personal information in compliance with our Privacy Statement. You can exercise your rights of access, rectification, erasure, restriction of processing, data portability, and objection by emailing us at privacy@ravenpack.com in accordance with the GDPRs. You also are agreeing to receive occasional updates and communications from RavenPack about resources, events, products, or services that may be of interest to you.
Your request has been recorded and a team member will be in touch soon.