Semantic Graph Based Learning for Trend Prediction from Long Financial Documents

Language models (LMs) have shown great promise in generative and predictive tasks. However, they do not explicitly incorporate semantic relations, and despite the progress in increasing the context size, they still struggle with long documents. Since abstract meaning representation (AMR), which is a graph-based representation of text to preserve its semantic relations, can encode semantic relationships at a deeper level, it can be beneficially utilized by graph neural networks (GNNs) for constructing effective document-level graph representations built upon LM embeddings for predictive tasks. We propose FLAG, an AMR-based framework to generate document-level embeddings via GNNs for long document classification tasks. We construct document-level graphs from sentence-level AMR graphs, endow them with finance-specific LM word embeddings, apply a GNN-based deep learning mechanism, and examine the efficacy of our AMR-based approach in predicting trends from financial documents. Extensive experiments on several different tasks are conducted on two large datasets of quarterly earnings call transcripts. We find that FLAG outperforms fine-tuning LMs directly on text in predicting stock price movement trends, as well as previous work utilizing document graphs and GNNs for text classification. Finally, we demonstrate our AMR-graph-based approach’s potential for explainability via a case study.