Introduction
Financial forecasting is an essential task that helps investors make sound investment decisions and wealth creation. With increasing public interest in trading stocks, cryptocurrencies, bonds, commodities, currencies, crypto coins and non-fungible tokens (NFTs), there have been several attempts to utilize unstructured data for financial forecasting. Unparalleled advances in multimodal deep learning have made it possible to utilize multimedia such as textual reports, news articles, streaming video content, audio conference calls, user social media posts, customer web searches, etc for identifying profit creation opportunities in the market. E.g., how can we leverage new and better information to predict movements in stocks and cryptocurrencies well before others? However, there are several hurdles towards realizing this goal - (1) large volumes of chaotic data, (2) combining text, audio, video, social media posts, and other modalities is non-trivial, (3) long context of media spanning multiple hours, days or even months, (4) user sentiment and media hype-driven stock/crypto price movement and volatility, (5) difficulties with traditional statistical methods (6) misinformation and non-interpretability of financial systems leading to massive losses and bankruptcies.
At the AAAI-2023 Workshop on Multimodal AI for Financial Forecasting (Muffin@AAAI2023), we aim to bring together researchers from natural language processing, computer vision, speech recognition, machine learning, statistics, and quantitative trading communities to expand research on the intersection of AI and financial time series forecasting. We will also organize 2 shared tasks in this workshop – (1) Stock Price and Volatility Prediction post-Monetary Conference Calls and (2) Cryptocurrency Bubble Detection.
This workshop will hold a research track and a shared task track. The research track aims to explore recent advances and challenges of multimodal AI for finance. As this topic is an inherently multi-modal subject, researchers from artificial intelligence, computer vision, speech processing, natural language processing, data mining, statistics, optimization, and other fields are invited to submit papers on recent advances, resources, tools, and challenges on the broad theme of Multimodal AI for finance. The topics of the workshop include but are not limited to the following:
- Transformer models / Self-supervised / Transfer Learning on Financial Data
- Machine Learning for Finance
- Natural Language Processing and Speech Applications for Finance
- Conversational dialogue modeling for Financial Conference Calls
- Social media and User NLP for Finance
- Entity extraction and linking, Named-entity recognition, information extraction, relationship extraction, ontology learning in financial documents
- Financial Document Processing
- Multi-modal financial knowledge discovery
- Financial Event detection from Multimedia
- Visual-linguistic learning for financial video analysis
- Video understanding (human behavior cognition, topic mining, facial expression detection, emotion detection, deception detection, gait and posture analysis, etc.)
- Data annotation, acquisition, augmentation, feature engineering, for financial/time-series analysis
- Bias analysis and mitigation in financial models and data
- Statistical Modeling for Time Series Forecasting
- Interpretability and explainability for financial AI models
- Privacy-preserving AI for finance
-
Submission
-
Short Paper: Up to 4 pages of content including the references.
Upon acceptance, the authors are provided with 1 more page to address the reviewer's comments. -
Long Paper: Up to 8 pages of content including the references.
Upon acceptance, the authors are provided with 1 more page to address the reviewer's comments. -
Shared Task Track: Participants are invited to take part in shared tasks: (1) Financial Prediction from Conference Call Videos and (2) Cryptocurrency Bubble Detection. Participants are invited to submit a system paper of 4-8 pages of content including the references.
- Paper submission deadline: December 23, 2022
- Acceptance notification: January 5, 2023
- Camera-ready submission: January 15, 2023
- Muffin workshop at AAAI 2023: Feb 13, 2022
Authors are invited to submit their unpublished work that represents novel research.
The papers should be written in English using the AAAI-23 author kit
and follow the AAAI 2023 formatting guidelines.
Authors can also submit supplementary materials, including technical appendices,
source codes, datasets, and multimedia appendices.
All submissions, including the main paper and its supplementary materials, should be fully anonymized.
For more information on formatting and anonymity guidelines, please refer to AAAI 2023 call for paper page.
All papers will be double-blind peer-reviewed. Muffin workshop accepts both long papers and short papers:
Two reviewers with the same technical expertise will review each paper.
Authors of the accepted papers will present their work in either the Oral or Poster session.
All accepted papers will appear in the workshop proceedings that will be published on CEUR-WS.
The authors will keep the copyright of their papers that are published on CEUR-WS.
The workshop proceedings will be indexed by DBLP.
Paper must be submitted using EasyChair
Important Dates
All deadlines are “anywhere on earth” (UTC-12)
About the Shared Task
The Multimodal AI for Finance Forecasting (Muffin) workshop will host two shared tasks on challenging multimodal financial forecasting problems using artificial intelligence:
Task-1: Financial Prediction from Conference Call Videos
Monetary policy calls (MPC) provide important insights into the actions taken by a country’s central bank on economic goals related to inflation, employment, prices, and interest rates. Investors and analysts critically analyze these video calls to forecast prices of the stock market, treasury bonds, gold, and currency exchange rates post the conference call. Prior works in the NLP literature have looked at what is being said during press conferences although there is a greater need to focus on how it is being said. The use of multimodal (visual+textual+audio) input to answer this question has been largely limited. Non-verbal behavioral cues from conference videos such as eye movements, facial expressions, postures, gaits, the complexity of language, vocal tone, and facial expressions of the speakers may reflect emotions that subjects may not express through words and have been found to be strongly correlated with enhanced trading activities in the financial markets. Interpreting and extracting information from financial conference calls reveals difficult challenges such as (1) Gap in current multimodal AI methods for simultaneously leveraging visual, vocal, and verbal modalities; (2) Long length of videos (50min to 1 hour) with multi-page text transcripts (3) Need to explore few-shot, semi-supervised, and self-supervised methods due to limited training data; (4) Large variability in conference calls across geographies due to different speakers, demographics, and economic conditions causing unintended bias. To this end, we curated a dataset of video conference calls from 2009 to 2022 released by central banks of 6 major English-speaking economies - USA, Canada, European Union, United Kingdom, New Zealand, and South Africa. The data has been processed to extract video frames, audio recordings, and utterance-aligned text transcripts. The task is to predict the volatility and price movement of stock market indices, gold, currency exchange rates, and bond prices T days after a conference call. We provide a cumulative of 25K data points split across training/development/testing for experimentation.
The train/validation/test corpus can be downloaded from: here .
Relevant research paper: [1] MONOPOLY: Financial Prediction from MONetary POLicY Conference Videos Using Multimodal Cues
Task-2: Cryptocurrency Bubble Detection on Social Media
Cryptocurrency trading presents a new investment opportunity for maximizing profits. The rising ubiquity of speculative trading of cryptocurrencies over social media leads to rapid escalation and crash of price in a short period of time, also called bubbles, causing investment losses and bankruptcy. These crypto bubbles are strongly tied to user sentiment and social media usage as opposed to conventional value-driven stocks and equities. Such financial bubbles are often a result of social media hype and the intensity of contagion among users, rendering both conventional statistical models and contemporary ML models weak as they are not built to deal with large volumes of unstructured, user-generated text on social media. In order to identify and safeguard against such bubbles, we formulate the CryptoBubbles Detection Challenge - a novel multi-span prediction task over future days of time series price data for crypto assets. We have curated a dataset of the 50 most traded crypto coins by volume from the top 9 crypto exchanges such as Binance, Gatio, etc to obtain a time series of prices for 450+ crypto assets over five years accompanied by over 2.4 million related tweets.
The train/validation/test corpus can be downloaded from: here .
Relevant research paper: [2] Cryptocurrency Bubble Detection: A New Stock Market Dataset, Financial Task & Hyperbolic Models
Participation Guidelines
In order to participate, please first fill out this form to register for the shared tasks: here . The team name that is provided in this form will be used in subsequent submissions and communications.
Use the train/validation/test dataset provided in the GitHub pages to design your systems. Final results should be submitted as a 4-8 page paper to the EasyChair Submission Link by the Shared Tasks paper submission deadline (December 15, 2023). All submitted papers will be evaluated based on their (1) novelty, (2) performance results, (3) system description, and (4) reproducibility. Participants are highly encouraged to submit their code and scripts to enhance reproducibility and describe the system parameters in sufficient detail. Accepted submissions will be published as part of workshop proceedings.
Task and Data Citations
If you utilize any of the above datasets or build solutions for / inspired by the above tasks, please cite the relevant work:
[1] Mathur, Puneet, Atula Tejaswi Neerkaje, Malika Chhibber, Ramit Sawhney, Fuming Guo, Franck Dernoncourt, Sanghamitra Dutta and Dinesh Manocha. “MONOPOLY: Financial Prediction from MONetary POLicY Conference Videos Using Multimodal Cues.” Proceedings of the 30th ACM International Conference on Multimedia (2022).
[2] Sawhney, Ramit, Shivam Agarwal, Vivek Mittal, Paolo Rosso, Vikram Nanda, and Sudheer Chava. “Cryptocurrency Bubble Detection: A New Stock Market Dataset, Financial Task & Hyperbolic Models.” NAACL (2022).