Introducing Series Paper 2 About Earnings Q&A Sentiment

Following Paper 1: Q&A Language Transparency, RavenPack introduces Paper Series 2: Q&A Earnings Calls Communication Quality - Q&A Sentiment.

The Q&A portion of earnings calls has long been recognized as a critical moment for information discovery, but quantifying its predictive value has remained challenging. Advances in NLP-based sentiment detection now enable precise, large-sample measurement of sentiment within these interactions at unprecedented granularity.

RavenPack analysis focuses on a comprehensive panel of U.S. mid- and large-cap companies from 2015 to 2025, leveraging RavenBERT transformer model to aggregate chunk-level sentiment scores separately for management answers and analyst questions within the Q&A sections of earnings calls.

Key findings

Our results provide robust evidence that firms exhibiting negatively-perceived disclosures, both in management responses and analyst queries, measured by lower block-level sentiment for both groups, consistently underperform their peers.

This finding is validated through the construction of profitable long-short portfolios based on question and answer sentiment metrics, which generate average return spreads of 410 and 370 basis points, respectively, over a weekly effective holding period.

Performance breakdown

For management's answers and analysts' questions, the Sentiment Factor portfolios generate respectively 410 and 370 basis points annually for a weekly effective holding period, with an Information Ratio of 0.78 and 0.75 in excess of the US mid-large cap market.

Sentiment Measure	Ann. Excess Returns	Information Ratio	Effective Holding Period
Analyst Questions (7-day decay)	4.1%	0.78	8.0 days
Management Answers (7-day decay)	3.7%	0.75	8.0 days

Robust across market segments

Additionally, the robustness analysis demonstrates persistence of alpha to universe size, with performance metrics remaining stable and statistically significant across all segments of market capitalizations. These results indicate that the observed alpha is robust and persistent even among larger, more liquid stocks.

Following universe partitioning, the stratified backtests for weekly time decay across ascending market capitalization buckets demonstrate the robustness of the alpha to universe size. Performance metrics remain statistically significant across all segments while generally improving for smaller companies.

Methodology and data source: RavenPack Transcripts Annotations

Our analysis utilizes RavenPack Transcripts Annotations as the data source, which provides a deep level of granularity by offering detailed, sentence-level information. Crucially, these sentence annotations are accompanied by comprehensive metadata for detected entities and events, including sentiment scores, sentiment confidence levels, event categories, and event recency.

We process this data using RavenPack's BERT-based transformer model (RavenBERT). This allows us to aggregate sentence-level sentiment scores, treating management answers and analyst questions within Q&A sections as separate streams. The resulting long-short, dollar-neutral factor-mimicking portfolios based on this methodology demonstrate robust, statistically significant performance across a variety of time decays and universe sizes.