Cross-Language Approach for Quranic QA

The Holy Quran is revered by Muslims worldwide as a Holy source of guidance and knowledge [1]. However, its accessibility is often limited by linguistic barriers, particularly for non-Arabic speakers and even for native Arabic speakers unfamiliar with its Classical Arabic form [2]. Readers and commentators have historically faced challenges in understanding the Quran due to its linguistic style and complex composition [3]. These complexities are compounded by the linguistic gap between modern standard Arabic (MSA), often used in questions, and classic Arabic (CA), the language of the Quran [4]. To bridge this gap, a cross-language approach is essential, taking advantage of advanced translations and language models.

Existing Quranic QA systems, such as those explored in the Quran QA 2023 shared task, have made significant strides to address these challenges [5]. These tasks focus on two primary subtasks: Passage Retrieval, which aims to identify the most relevant Quranic passages in response to a query, and Machine Reading Comprehension, which aims to retrieve accurate and contextually relevant passages. However, the development of Quranic QA systems faces critical challenges due to the small size of existing datasets and the linguistic disparity between MSA, used in questions, and Classical Arabic (CA), found in Quranic verses. These limitations cause a severe hinder to the model performance, emphasizing the need for innovative approaches to dataset expansion and fine-tuning LLMs and LMs to meet the unique requirements of Quranic QA.

This research adopts a cross-language approach to enhance Quranic QA systems. By translating both questions and answers, the linguistic barriers posed by MSA and CA are overcome. The dataset was significantly expanded through machine translation, paraphrasing, and the integration of additional questions from existing literature. Fine-tuned LLMs, such as BERT-Medium, RoBERTa-Base, DeBERTa-v3-Base, ELECTRA-Large, Flan-T5, Bloom, and Falcon, were trained to address these challenges effectively. This approach not only enhances the dataset but also aligns advanced language models with the unique linguistic and contextual complexities of Quranic texts. By upgrading and highlighting these advancements, the system provides more accurate and context-sensitive responses, allowing deeper engagement with the Quran[6].

Figure 1 illustrates the workflow adopted in this study, showcasing the process from dataset augmentation to fine-tuning and evaluation. This diagram highlights the cross-language approach and the transformations of the data set essential for improving the Quranic QA system.

The Quranic text is rich and contains many events, making it essential to efficiently retrieve passages and answer questions [7]. Its unique features, like complex syntax, multiple definitions, and deep linguistic elements, present challenges for question-answering systems [8]. One major challenge is bridging the gap between Modern Standard Arabic and Classical Arabic, as well as handling unanswerable queries using techniques like thresholding [9].

Models like AraBERT, CAMeLBERT, and AraELECTRA have shown great potential in handling Arabic text, effectively capturing its meaning and context [10]. However, limited or unbalanced datasets often hinder their performance, making it harder to handle new or unseen queries [11]. Sarhan and Elkomy [9] addressed these issues by using hybrid models that combine dual-encoder and cross-encoder approaches. They leveraged transfer learning with external datasets like TyDI QA and Tafseer, improving performance with a MAP score of 0.25. Despite this, there is still a need for more dataset expansion.

Mahmoudi et al. [12] introduced a multitask transfer learning method, combining both unsupervised and supervised fine-tuning using models like AraELECTRA and AraBERT. They used advanced embedding techniques like TSDAE and SimCSE to improve passage retrieval by producing high-quality sentences. However, they also stressed the need for more dataset expansion.

In another approach, Alawwad et al. [13] focused on Quranic passage retrieval by translating Arabic queries into English. This allowed them to use advanced English-based models, such as OpenAI embeddings and sentence transformers. Their system also included a paraphrasing module to create multiple query variations, significantly improving retrieval accuracy. This method demonstrated the power of translation-based techniques in overcoming linguistic barriers. The queries were applied to an English-translated version of the Quran, utilizing LLMs and other models that support English [14].

This work approach focuses on two main aspects: (1) Dataset Augmentation, which expands and enriches datasets by translating Arabic questions into English and paraphrasing to create more diverse queries. Answers are then retrieved from an English translation of the Quran; and (2) Language Model Fine-Tuning, where models like BERT-Medium, RoBERTa-Base, DeBERTa-v3-Base, ELECTRA-Large, Flan-T5, Bloom, and Falcon are fine-tuned to handle Quranic QA tasks. This approach effectively bridges the gap between Modern Standard Arabic and Classical Arabic, improving performance.

The methodology of this study comprises three key phases:

1. 1.

   Dataset Preparation: Expanding and diversifying the original dataset through the addition of new questions and paraphrasing to increase the linguistic variation.

2. 2.

   Cross-Language : Translating questions into English to leverage the capabilities of widely available pre-trained language models that perform better in English.

3. 3.

   Model Fine-Tuning: Fine-tuning state-of-the-art models, including BERT-Medium, RoBERTa-Base, DeBERTa-v3-Base, ELECTRA-Large, Flan-T5, Bloom, and Falcon, to optimize performance for the Quranic QA task.

The evaluation results show the impact of dataset expansion, cross language, and model fine-tuning on enhancing the performance of QA systems for the Holy Quran. Multiple models evaluated on metrics such as MAP10 (Mean Average Precision at 10), MRR (Mean Reciprocal Rank), Recall at top 5 and 10 passages (Rec5, Rec10), and Precision for the same ranks (Pre5, Pre10).

The table highlights the highest improvements achieved by fine-tuning . Noticed observations are:

• •

  Electra-Large achieved the highest improvements, with MAP10 improving from 0.04 to 0.31 and MRR from 0.11 to 0.43, Reflecting better ranking of relevant passages.

• •

  Roberta-Base showed strong baseline performance and achieved further improvements after fine-tuning, with MAP@10 increasing from 0.10 to 0.34 and MRR enhancing from 0.22 to 0.52.

  All models showed consistent improvements in Recall (Rec5, Rec10) and Precision (Pre5, Pre10), with DeBERTa-v3-Base achieving the best results, scoring 0.46 and 0.50 in Recall (Rec5, Rec10) and 0.25 and 0.24 in Precision (Pre5, Pre10), respectively.

This detailed comparison highlights the effectiveness of expanding the dataset,cross language and fine-tuning the models, showing how these improvements help QA systems better retrieve and rank Quranic passages.

This study addresses the unique challenges of Quranic QA systems by adopting a cross-language approach to reduce the linguistic difference between Modern Standard Arabic questions and Classical Arabic Quranic texts. By enhancing the dataset with techniques like machine translation, paraphrasing, and adding more varied question sets, and by fine-tuning both language models and large language models, this work achieved improvements in performance. Notably, RoBERTa-Base demonstrated the best MAP@10 (0.34) and MRR (0.52), while DeBERTa-v3-Base excelled in Recall@10 (0.50) and Precision@10 (0.24). These results underscore the effectiveness of cross-language strategies in overcoming linguistic barriers and enhancing Quranic QA systems. Future research will focus on further expanding dataset diversity, exploring alternative fine-tuning strategies, Fine-tuning multilingual models to improve accuracy and generalizability. This work lays a foundation for more accessible and robust Quranic QA systems, allowing users worldwide to engage more deeply with the Holy Quran.