A walk through generative AI & LLMs: prospects and challenges

Following up on a previous article on ChatGPT prompting for Equity Research, this article briefly discusses the main prospects for Generative AI with a particular focus on providing a reader-friendly but comprehensive introduction to LLMs (Large Language Models) including the opportunities and challenges that come with them and the many ways of implementing LLMs in an investment organisation.

Generative AI Definitions 

Wikipedia’s entry for Generative AI is rather tepid: “Generative artificial intelligence (AI) is artificial intelligence capable of generating text, images, or other media. Generative AI models learn the patterns and structure of their input training data and then generate new data that has similar characteristics”.

Alternatively, we can ask ChatGPT to provide a definition of Generative AI that is tailored for different audiences:

The jargon of AI-related terms continues to grow and can sometimes confuse readers at the very beginning of their Data Science journey with buzzwords such as AI, Generative AI, NLP and LLMs. In a nutshell, AI is the overarching field that encompasses a wide range of techniques and applications, while Generative AI focuses on data generation. It’s powered by very large machine learning models that are pre-trained on vast amounts of data, commonly referred to as foundation models (FMs). NLP (Natural Language Processing) is a subset of AI specifically concerned with language-related tasks, and LLMs are a type of NLP model known for their language generation capabilities, often powered by deep learning techniques. These concepts are interconnected, and advancements in one area often benefit others, leading to significant progress in the broader field of AI.

Because of space constraints, I recommend the readers to visit the next links by IBM and Nvidia to have a more detailed introduction about Generative AI.

In this article our focus will be LLMs (Large Language Models), which are models that blend the best of both the Generative AI and NLP worlds.

LLMs Introduction

LLMs (Large Language Models) have revolutionised the field of NLP due to their ability to handle various language-related tasks effectively by leveraging deep learning and big data. The evolution of LLMs has not occurred overnight but is the result of many decades of research as pointed in the next timeline (source: author).

After many models and iterations, it was the rise of the Transformer Architecture in the last decade that allowed NLP to dramatically leap forward with models like BERT and its finance-specific versions like FinBERT, introduced in 2019. By the turn of the 2010s, the golden age of LLMs had begun with a zoo of LLMs being introduced in the NLP arena by many parties such as OpenAI (GPT), Google (Bard) or Meta (LLaMA) as can be seen in the chart below (source: A Survey of LLMs).

These state-of-the-art models leverage Transformer Architecture, Transfer Learning and Reinforcement Learning in two stages - pre-training and fine-tuning -  to reach astounding performance across a diverse number of tasks like text generation, transcription, debugging, translation and summarisation among many others:

• Pre-Training:in this stage, LLMs are trained using large datasets for specific completion tasks (e.g. predicting missing words) or the generation of contextually relevant text. This training stage allows them to learn generalist patterns and nuances that are learned by the weights and biases of the model’s architecture. That said, the learned coefficients in this stage are not optimised for any particular task.
• Customisation: this part can be either performed using either “prompt learning” or “supervised fine-tuning”. In any case, these is the stage where the pre-trained model is altered in order to make it excel at particular tasks (e.g. sentiment analysis) within a specific domain (e.g. finance).

Overall, these LLMs share similarities in their foundational concept but also diverge in terms of their specific model architecture and the corpus data utilised during their pre-training stage. To illustrate this fact, the next chart (source: A Survey of LLMs) showcases the differences across multiple LLMs with models like GPT and LLaMA models being trained mostly with web content, yet the former being much more complex with +175bn parameters compared to the latter’s +65bn parameters. Other models like LLaMA and Chinchilla have a similar complexity as measured by their number of parameters, yet their pre-training corpus datasets are significantly different with Chinchilla relying much more on Books & News data (40%) and less in web content (56%). Overall, domain-specific LLMs have showed better performance than generic models for specific tasks, yet generic data is still important for LLMs to be able to extrapolate and think “out-of-the-box”.

The reader can visit a more detailed explanation about GPT model specs in this first article on LLMs focused on ChatGPT3 prompting. 

LLMs Customisation: Prompt Learning vs Fine-Tuning

In the context of explaining model customisation of LLMs, it's important to understand the distinction between the foundational model and a customised model. On the one hand, the foundational model refers to the original, pre-trained Large Language Model (LLM) that has been trained on a vast and diverse corpus of text data e.g. GPT-3. Remember that the foundational model is designed to be a general-purpose language model that can handle various NLP tasks without any specific fine-tuning.

On the other hand, a customised model is created by either adding incremental knowledge or skills, or by including domain-specific knowledge (e.g. finance) to the foundational model. That results in a customised model that is more specialised and optimised for a specific use case e.g. BloombergGPT.

But let’s circle back to our original question: what is the difference between “prompt learning” and “fine-tuning” when customising a foundational LLM? In a nutshell, “prompt learning” is used when adding incremental or new skills to a foundational model is desired; whereas “fine-tuning” is a more resource-intensive customisation technique that retrains the model with a domain-specific dataset that ultimately changes the behaviour of the model dramatically.

“Prompt learning” is a technique used to guide the behaviour of a pre-trained LLM by providing explicit instructions or prompts in the input text. Instead of fine-tuning the model on a task-specific dataset, prompt learning relies on designing well-crafted input prompts that serves as a guide to elicit the desired response from the model without requiring extensive retraining. Some well-known techniques for prompting are:

• Zero Shot Learning: Asking the foundation model to perform a task with no previous example or knowledge. This is simply using the foundation model without any “prompt learning”.
• Few Shot Learning: builds upon “zero-shot” learning by allowing the model to observe a small amount of task-specific examples during inference. Few-shot learning provides a way to adapt a pre-trained model to specific tasks with minimal examples.
• P-tuning: Training a "prompt-model" with a much larger number of examples (+100s, +1000s) covering various scenarios and desired responses that get sent to the foundation model at inference time. P-tuning allows for more controlled adaptation of the model's behaviour while preserving its general language understanding.
• Constrained Generation: additional constraints or requirements are provided to guide the output. For example, you could instruct the model to generate a coherent paragraph while including specific keywords or concepts to match a certain predefined criteria.
• Dialogue management: guides a LLM's responses in a conversation by providing prompts that instruct the model on how to respond to user inputs. This can help in creating more contextually relevant and meaningful interactions.
• RAG (Retrieval-Augmented-Generation): this approach was introduced by Facebook AI Research (FAIR) and collaborators in 2021 and is a powerful prompting technique used by Bing search and other high-traffic sites to incorporate current data into their models. Using RAG in an AI-based application involves the following steps:

1. The user inputs a question.
2. The system searches for relevant documents that might answer the question (e.g. proprietary data stored in a document index).
3. The system creates an LLM prompt that combines the user input, the related documents, and instructions for the LLM to answer the user’s question using the documents provided.
4. The system sends the prompt to an LLM.
5. The LLM returns an answer to the user’s question, grounded by the context we provided.

“Fine-Tuning”, often referred as “Transfer Learning” or “Supervised Fine-Tuning”, involves taking the pre-trained foundational model, often referred to as a “base model”, to conduct further training using a smaller, task-specific dataset. During fine-tuning, the model's weights and parameters are updated based on the task-specific data, allowing the model to adapt its knowledge to the specific task (e.g. sentiment analysis) or domain (e.g. investments). Fine-tuning is usually a supervised learning process where the model is provided with labelled examples,and it learns to make predictions or generate text according to the task requirements. In this stage, techniques such as learning rate scheduling and regularisation are used to refrain the model from overfitting on the specific task while preserving its generalist knowledge.

The next table highlights the main differences between “Prompt Learning” and “Fine-Tuning” along with use cases and a simple example (source: author).

Overall, RAG has become relevant as a prompting technique based on research evidence as it’s more cost-efficient and straightforward to implement than model fine-tunning, especially when coupledwith services such as Azure Cognitive Search that facilitate the search of documents. In particular, using RAG with vector search allows to encode the input (user question and required documents) using a pre-trained embedding model such as OpenAI’s “text-embedding-ada-002” and a pre-trained LLM e.g GPT model. The next illustration displays this process very clearly (source: Microsoft):

NLP Models for Finance: Evolution, Opportunities and Challenges

The NLP revolution has also arrived in finance over the last decade, starting with very simplistic models that applied rule-based word-dictionary algorithms using 10-K and 10-Q information as training corpuses such as Lougrhan-McDonald . The advent of Transformer architecture came with the introduction in 2019 of FinBERT, a Finance-focused version of BERT models. Last but not least, LLMs have also started to be implemented in the investment industry e.g. introduction in 2023 of BloombergGPT, a GPT model with +50bn parameters and corpus data focused on finance-specific categories.

As discussed in the section “Generative AI Prospects”, there are many opportunities for investment organisations that are actively implementing LLMs tools within their organisation’s workflow. That said, the benefits of LLMs are more visible and easy to measure in some tasks such as customer service (e.g. customer service bots) or automation (e.g. summarised conference call transcripts) than in others where validation is far more complex and, sometimes, not feasible as it happens in stock news sentiment analysis.

For instance, Lopez-Lira, Tang (2023) conducted a first exploratory approach with LLMs applied to sentiment analysis by using ChatGPT conducting news sentiment analysis to predict daily stock returns using CRSP data and news headlines for the period September 2021 to December 2022. To avoid overfitting and look-ahead bias, the version of ChatGPT used by the authors had been trained only until September 2021. The authors use the following standardised prompt in their study  along with the next mapping applied to ChatGPT’s answers: “YES” is mapped to 1, “UNKNOWN” to 0, and “NO” to -1.:

ChatGPT scores were averaged on a given day and logged to assess stock returns predictability. Furthermore, the authors followed a conservative update approach as news released after the closing bell were incorporated in the opening of the next day. Using these signals, the authors create several long-short portfolios using the ChatGPT model (GPT in the table) and other models such as GPT2, GPT1, BERT and sentiment signals from a data vendor reaching the next conclusions: 

• ChatGPT (GPT using version 3.5) scores display statistical significance predicting daily stock returns. Although ChatGPT showcases a similar performance to other models with regards to ML model performance metrics (e.g. Accuracy, Recall, F1, etc); where it actually excels is in return predictability i.e. more basic models such as GPT-1, GPT-2, and BERT cannot accurately forecast returns, indicating return predictability is an emerging capacity of complex LLMs.
• ChatGPT outperforms traditional sentiment analysis methods from a leading data vendor. In other words, data vendors will be forced to enhance their sentiment models to pose a challenge for LLMs.
• Incorporating advanced LLMs into the investment decision-making process can yield more accurate predictions and enhance the performance of quantitative trading strategies.
• Predictability power is concentrated on smaller stocks and more prominent on firms with bad news, consistent with limits-to-arbitrage arguments rather than market inefficiencies.