Journey of Hallucination-minimized Generative AI Solutions for Financial Decision Makers

Generative AI-based chatbots have become increasingly popular over the last year with the launch of OpenAI’s ChatGPT in November 2022. Although language-based AI models have been built and researched for decades, now, the genesis of the chatGPT model can be traced back to the formation of OpenAI in 2015 followed by the launch of Generative Pre-trained Transformer (GPT-1) model in 2018 with 117 million parameters (Marr, 2023). The GPT-1 model was the first of its kind in the area of unsupervised learning models that could understand tasks and use books as training data to complete sentences or predict a few follow up sentences while maintaining context (Marr, 2023). The GPT-2 model released in 2019 was an upgrade with 1.5 billion parameters with significantly improved natural language generation (NLG) features with the capability of generating several paragraphs of contextual and sensible text. The launch of GPT3 in 2022 demonstrated significantly superior natural language comprehension and question answering capabilities owing to the 175 billion parameters model size. The faster turbo versions of GPT3.5 and the following evolved version of GPT4 estimated to have 1.76 trillion parameters (Prakash, 2023) showed significant enhancement in language translation and comprehension of large texts and multi modal data (OpenAI, 2023). This evolution in the class of LLMs that perform inferencing only, can be adapted to a variety of multi-modal data processing applications, such as audio, video, document processing and conversions through prompt engineering. In this work we present the challenges around prompt engineering process to ensure reliability and trustworthiness in such Generative AI-based systems and solutions.

The evolution in the current genre of LLMs with ethical and safety considerations (Thompson, 2023) has enabled its widespread usage and early adoption into products. LLMs significantly reduce the entry barrier to AI since that are easier to program using normal language instructions as opposed to the dependence on programming languages (Guo et al., 2023). Some noteworthy products that are completely language based and utilize generative AI are: the explain my answer feature in Duolingo (Duolingo, 2023), automated content creation with AIcontentfy (AIcontentfy, 2023) and job hunting and recruitment support with products like Occupop and SkillGPT (Doyle, 2023). However, LLMs pose a serious issue, also known as hallucinations, wherein inaccurate facts are presented to the user specifically in use cases that involve numerical or tabular data and non-language sources (Oviedo-Trespalacios et al., 2023). In this work, we present a novel framework that minimizes and controls hallucinations for such numerical and data table interactions to generate reliable and accurate answers for decision making tasks. Additionally, we present the development to launch journey for reliable and trustworthy LLM-based products for numerical and analytical domains such as finance and sales.

Building and deploying LLM-based systems and products have three major stages, namely, the prototyping stage, the scaling stage and the evolution stage as shown in Fig. 1.

In the first prototyping stage, business case value realization drives the build plan of the minimum viable product. The major development areas include the following: Data Science (prompt engineering, modular builds), UI/UX considerations and LLMOps setup for Infrastructure (Mcmahon, 2023). It is noteworthy that the prototyping stage may incur radical limitations with regards to reliability for certain user-queries. For instance, LLMs are safeguarded and limited against making predictions (Kumoai, 2023). Thus, for the analytical finance domain, queries such as “Which stocks in NYSE should I invest in?” will remain out of scope until a preferred prediction model is combined with the LLM prompts. In this prototyping stage, we build four novel components that monitor and control for hallucinations in the LLM responses and ensure repeatable and reliable answers.

The second stage for the hallucination-minimized solution is scaling the prototype for a variety of user questions, also known as intentions (such as why, what, where, how, trend, anomaly, what-if etc.) while benchmarking for the choice of LLM to ensure accuracy, reliability, repeatability, and optimal response times. The third and final stage of the solution is fine-tuning the LLMs based on an already curated set of user-queries and sample responses to ensure evolution in the question answering capabilities in accordance with reinforcement learning with human feedback (RLHF) criteria (Ouyang et al., 2022).

For a finance-question and answering system we propose a novel Langchain-based framework (Pandya and Holia, 2023) with custom modules to minimize hallucinations as shown in Fig. 2.

The novel components designed for this solution are as follows:

1. (1)

   Question intention classification: This module creates separate customizable prompts for each user query type. Thus, the instructions for Why, What, How, trends, anomalies What-if queries can be separately designed. For every user query, the first step is to categorize the intent to define the generic steps to process the specific user-request. Incorrect intent classifications can lead to hallucinations.

2. (2)

   Data chunk generation and filtering module: Since LLMs are trained on text data, converting tabular data to sentences and paragraphs is the optimal mechanism to pass data to LLMs. Each data table value is converted to sentences and stored as “data chunks”. Data chunks are further hierarchically categorized to support aggregated querying. Lack of granularity in data chunks can cause hallucinations.

3. (3)

   Custom prompt generation module: For each user query, the most pertinent data from the existing data chunks need to be selected to create a customized prompt that is then sent to the LLM. A customized prompt has four major components: the persona, key definitions, relevant data chunks and a sample question and answer. Filtering for the “most similar” data chunks per user query is necessary to minimize hallucinations. A customized data chunk ranking mechanism that is optimized for run-time is crucial to assimilate a customized prompt per user query that best represents the user’s intent and data requirements.

4. (4)

   Response quality scoring module: This module assesses each LLM response for hallucinations using standardized language-based libraries (such as nltk, spacy etc.). This novel component evaluates the question, the prompt sent to the LLM and the returned response together and evaluates the response for contextual, numeric, and uniqueness and grammatical accuracy. These four metric binary quality scoring modules categorize each response into Low/Medium/High confidence, that provides an automated estimate regarding any hallucinations to the user.

Hallucinations are an unwanted outcome of LLMs that need to be further studied and scored for non-language and multi-modal data use cases. While most hallucinations are caused by biased training data, abstract nature of questions/prompts and LLM parameters (Rawte et al., 2023), there are approaches such as advanced modular prompting that can minimize hallucinations. In this work we present a novel LMOps system design and the three stages of developing LLM-based products for analytical and finance domains, where hallucinations can have extremely detrimental impact for decision making tasks.

Accenture is a leading global professional services company of 738,000 people in 120 countries. They help businesses, governments and organizations build their digital core, optimize operations, accelerate revenue growth, and enhance citizen services. Accenture is one of the global leaders in helping drive change with technology at its core through strong ecosystem relationships, unmatched industry experience, functional expertise, and global delivery capability. In June 2023, Accenture announced that the company would invest $3 billion in its Data and AI practice to help clients across all industries rapidly and responsibly advance and use AI to achieve greater growth, efficiency and resilience.

Dr. Sohini Roychowdhury is the Global Head of AI/ML at the Corporate Data and Analytics Office, Accenture, USA. Her global team builds Generative AI solutions including a finance-chatbot and automated report generation for decision makers. Prior to this, she formed the Founding team and served as Director of Curriculum and Machine Learning at an Ed-Tech Startup called FourthBrain that provides specialized hands-on courses in the field of Machine Learning and AI. Prior to her entrepreneurial venture she was the Sr. Manager of Autonomous Drive and Head of University Relations at VolvoCars USA, and prior to that a tenure track Assistant Professor in Electrical and Computer Engineering at a University of Washington campus. Dr. Roychowdhury’s latest research directions benchmarking Large Language models for scalable product development. Till date she has over 60 academic research papers and 20 granted patents to her name and a Youtube channel, AI with Sohini.