Leveraging AI to Advance Science and Computing Education across Africa: Challenges, Progress and Opportunities

SuaCode ¹¹1http://suacode.ai/ is an AI-powered smartphone-based application that enables students across Africa to learn to code using smartphones (Figures 1 and 2). The app has monthly online coding courses with lesson notes, exercises, quizzes, and fun coding assignments in English and French which cover official languages across Africa. We use simplified lesson notes designed for offline reading to reduce Internet data usage since they are expensive across Africa [3]. Our courses adopt a project-based learning approach where learners build and interact with a game on their phones as assignments. Each section of the course ends with multiple-choice quizzes and a coding assignment. The assignments are graded by our automated grading software, AutoGrad [14] which also checks for plagiarism [15] and provides detailed individualized explanations for wrong answers with an option for learners to submit a complaint in cases where they disagree with the provided answers and feedback. Our course-specific, AI-powered, human-in-the-loop forum is designed to allow learners to ask questions (even anonymously) and get quick and accurate answers from their peers, facilitators, and our AI teaching assistant, Kwame [16]. Kwame enables learners to get individualized learning support so they do not quit when it gets tough. Each person’s name shows by the side an earned badge to encourage helpful engagement. Each of the 3 badges (bronze, silver, and gold) is received based on a “helpfulness score” that we calculate using various metrics such as upvotes on questions and answers contributed by each learner. Our leaderboard celebrates helpful learners and facilitators thereby encouraging others to be helpful. Students who complete receive certificates and mentoring by African tech professionals in companies such as Google and Amazon enabling our learners to receive advice from experienced individuals who have similar backgrounds as them. SuaCode has over 2.5K learners across 43 African countries and 117 globally.

We created SuaCode to address the problem that less than 1% of African children leave school with basic coding skills with several jobs struggling to fill IT-related roles despite Africa being home to the youngest workforce in the world [17]. In 2017, while running the 4th edition of our annual innovation bootcamp, Project iSWEST, in Ghana, we noticed from our pre-survey that out of our 27 students, 25% had laptops, yet 100% had smartphones. This situation of limited access to computers led us to modify our coding course and deliver it using smartphones, the first of its kind in Ghana. Our students built pong games on their phones with several students working on coding assignments while in traffic [18]. Realizing the potential of our smartphone-based course, in 2018, we created SuaCode, a smartphone-based online coding program aiming to teach millions across Africa how to code by exploiting the proliferation and untapped capabilities of smartphones [19].

Between 2018 and 2020, we ran 4 pilots of SuaCode while growing exponentially (Figure 3) that reached 3K learners across 69 countries (42 in Africa) [19, 20, 21]. The 2018 version focused on students in Ghana and had over 30 students enrolled with only 23% completing [19]. It was delivered using Google Classroom as the course management platform (forum and assignment submission), Google Docs for lesson notes and APDE for the coding app. We had 4 lessons and assignments, and an optional project at the end. The first version in 2019 also focused on Ghanaians. The second version in 2019 expanded beyond Ghana to all Africans and was dubbed SuaCode Africa [21]. It had 709 applicants across 37 African countries. We introduced an acceptance criterion (submit the first 2 assignments) to filter for highly motivated students. Out of that, 210 were accepted and 151 completed resulting in a 72% completion rate. The 2020 version was dubbed SuaCode Africa 2.0 with the course being offered in both English and French [20, 22]. We used Piazza for that cohort which had a better forum experience than Google Classroom. Over 2,300 students across 69 countries, 42 of which were in Africa applied. We accepted and trained 740 students and 62% completed. We then used the learnings to build our AI-powered smartphone app, SuaCode and then launched it in 2022 to scale the impact of SuaCode.

These courses were structured in cohorts that created a sense of community and fulfilled learners’ need for affiliation, support, and interaction and also optimized to encourage completion resulting in high completion rates (62%, n=1000) vs industry standards (less than 20%). Our 600+ past learners significantly improved their understanding of fundamental coding concepts despite using only smartphones to learn. We collected qualitative and quantitative feedback from our learners in various cohorts. Our quantitative analysis showed that students had an average of 17 out of 20 across all 4 assignments indicating mastery of the content [21]. Also, there was no statistically significant difference in assignment scores between males and females, and educational level (high school, university, and high school graduates) suggesting that our course might be adequate for different demographics [21]. Furthermore, there was a statistically significant improvement in students’ self-reported proficiency in the programming concepts [21]. In user surveys about the smartphone coding experience, 85% of our learners (n=457) rated the coding experience 4+ on a 5-point Likert scale [22] with feedback such as “It was really convenient, honestly. I didn’t have to necessarily sit behind a desk to do it so I could do it when I was on my bed, eating, even using the bathroom. So it was fun and convenient coding on my phone”. Overall, there were several positive feedbacks about the experience such as “Suacode has been a very great experience for me. I got to learn processing and actually code on my phone. I also had help from the tutors and my fellow course mates which made it easier. I learnt a lot and I’m glad I had the opportunity to be part of the first batch of suacode initiative” and “SuaCode helped improve my algorithmic thought process. I had lots of practice with thinking in a step by step process and working through challenges”.

An innovation like SuaCode is an example of ways to get around the previously mentioned challenge of limited access to resources such as computers and infrastructure like affordable Internet which could hamper efforts to deploy AIED tools. By leveraging tools that are much more accessible to Africans like smartphones, AIED tools could to made available to these students and eventually improve their learning outcomes like we have done with SuaCode. Furthermore, our use of lesson notes rather than videos for lesson delivery addresses the challenge of expensive Internet data for African students. Intentional design of educational experiences could help to address the effect of some of the infrastructural challenges in Africa.

AutoGrad is a novel cross-platform software for automatically grading and evaluating graphical and interactive programs written in the Processing programming language in SuaCode courses. [14]. It uses APIs to retrieve assignments from the course platform, conducts both static and dynamic analyses on these assignments to assess their graphical and interactive outputs, and subsequently furnishes students with grades and feedback (Figure 4). The AutoGrad system itself was written using Python and Processing. The Python component manages assignment retrieval and the dissemination of grades and feedback to students, while Processing oversees the grading module, ensuring the effective execution of checks on Processing code. AutoGrad has been successfully deployed in multiple iterations of SuaCode cohorts, servicing over 1,000 students across Africa and evaluating more than 3,000 code files. These deployments involved running AutoGrad as software on a computer. Notably, in the latest cohort, students were allowed to submit complaints in cases where they believed their assignments were inaccurately graded. This approach not only ensured fairness to all students but also provided valuable insights for addressing instances where AutoGrad’s performance fell short and facilitated ongoing enhancements to the software.

We assessed AutoGrad’s grading accuracy using both test assignment scripts and actual student scripts from previous cohorts of the SuaCode course. Using 10 student scripts for Assignment 1, AutoGrad yielded identical grades to those assigned by manually grading instructors in 8 scripts, resulting in a mean absolute error of 0.7. However, subsequent assignments exhibited a higher frequency of discrepancies than Assignment 1 [14]. We also collected feedback from students regarding their experiences with AutoGrad. Quantitative feedback was gathered by asking students to rate their agreement with the statement “I liked the feedback from AutoGrad” on a 5-point Likert scale, ranging from strongly disagree to strongly agree. Among the 457 students who completed the course and responded in the 2020 cohort, 75.9% agreed or strongly agreed with the statement, indicating that a majority of students found AutoGrad’s feedback beneficial, albeit with some room for improvement, as indicated by a mean rating of 4 out of 5. Qualitative feedback was also collected, focusing on suggestions for enhancing AutoGrad’s feedback mechanism. A common theme among responses was the request for more detailed explanations accompanying the feedback provided by AutoGrad, as well as the identification of specific areas within their code where improvements could be made [14].

We developed a tool that performs code plagiarism detection in students’ assignment submissions in SuaCode courses and also shows visual evidence of the detected plagiarism [15]. We built this tool to address the issues of code plagiarism in the SuaCode where we previously identified that 27% of the 431 students had cases of plagiarism via manual inspection. We trained machine learning models on three (3) cosine similarity-based scores extracted from the TF-IDF (with n-grams) feature vector of the code files to detect pairs of code files that had cases of plagiarism. We used as features the cosine similarity of (1) student 1 code and student 2 code, (2) student 1 code and example code template, and (3) student 2 code and example code template. Our evaluation using 431 code files showed a balanced accuracy of 84% using random forest with less than 1% false positive. Also, the system provides proof of plagiarism via a GUI tool that displays side-by-side pairs of code files while highlighting sections with overlapping code. This work is the first that builds a complete end-to-end system that uses (1) machine learning algorithms to detect plagiarized source code containing English and French texts while taking the code example provided by the instructors into consideration and (2) provides visual evidence of the plagiarism (Figure 5) [15].

Kwame is a bilingual AI teaching assistant that provides answers to students’ coding questions in English and French for SuaCode courses [16]. Kwame is a deep learning-based question-answering system that was trained using the SuaCode course materials (lesson notes and past questions and answers) and evaluated using accuracy and time to provide answers. It finds the paragraph most semantically similar to a question via cosine similarity using Sentence-BERT (SBERT) [23], a large language model that uses siamese and triplet network architecture with BERT to train models such that semantically similar sentences are closer in vector space (Figure 6). We compared Kwame with other approaches and performed a real-time implementation which showed fast response time and superior accuracy of top 1, 3, and 5 accuracies of 58.3% (58.3%), 83.3% (80%) and 100% (91.7%) for English (French) respectively. Kwame has been integrated into the SuaCode app and answers students’ questions by posting an answer on the course forum in reply to students’ questions. Kwame’s pipeline is a retrieval system that consists of an ElasticSearch vector store of the embeddings of passages from the lesson notes. When a question is asked, our system computes an embedding of the question using SBERT, computes cosine similarity scores with all saved embeddings, and retrieves the top 3 passages which are then posted on the forum as answers. Kwame is named after Dr. Kwame Nkrumah the first President of Ghana and a Pan-Africanist whose vision for a developed Africa motivates this work.

Kwame was developed to address the issue where our learners needed a lot of assistance given it was the first coding course for most learners. We relied on human facilitators to provide support and answer students’ questions. For example, in SuaCode Africa 2.0, facilitators contributed over 1,000 hours of assistance time for 8 weeks and helped to achieve an average response time of 6 minutes through the course [22]. This approach was however not scalable as the number of students applying to SuaCode increased exponentially year on year. Hence, in 2020, we built Kwame, an AI teaching assistant to provide accurate and quick answers to students’ questions which would reduce the burden on human teaching assistants, and provide an opportunity to scale learning support. Kwame addresses the diversity of languages used for learning in Africa by offering answers in both English and French. This caters to both Anglophone and Francophone regions, covering the official languages spoken across different parts of the continent.

Kwame for Science ²²2http://ed.kwame.ai/ is an AI-powered web application that offers two primary functionalities: (1) question answering and (2) viewing past national exam questions [24, 11]. These features are enabled by our curated knowledge base of content from textbooks and past national exams over the past 28 years (for Integrated Science at the Senior High School level) with answers from certified teachers. We built Kwame for Science to address the impact of the shortage of qualified teachers [4] across Africa which makes it difficult for students to have adequate learning support.

The question-answering (QA) feature allows students to pose science-related queries and receive three passages as responses, each accompanied by a confidence score (Figure 7). Additionally, this feature provides the top five related past national exam questions from the Integrated Science subject of the West African Secondary School Certificate Exam (WASSCE), along with their corresponding expert answers. Users also can review their question history. When a student asks a question, our system extracts an embedding from the text using SBERT, computes cosine similarity between the embedding and the embeddings of passages from textbooks stored in ElasticSearch on Google Cloud Platform, and then returns the top answers based on the cosine similarity scores. Students could rate the helpfulness of answers and related questions.

Moreover, the View Past Questions feature permits students to explore past national exam questions and answers from the Integrated Science subject. This feature includes filters for refining the displayed questions based on parameters such as examination year, specific exam, question type, and automatically categorized topics generated by a custom topic detection model (Figure 8). All these criteria could be inferred easily from the metadata of the original exam files except the topic for which we developed a model to automatically categorize each question according to one of the syllabus topics [11]. We trained a machine-learning model that used a support vector machine and SBERT embeddings of passages from the Integrated Science subject syllabus to classify each of the past exam questions into one of the 48 syllabus topics. We then used the model to automatically categorize questions into topics in the syllabus for all 28 years of exams.

We launched the web app in beta from 10th June 2022 to 19th February. During the 8-month deployment, we had 750 users across 32 countries (15 in Africa) asking 1.5K questions with Kwame’s helpfulness scores being top 1 and top 3 of 72.6% and 87.2% respectively. For the viewing past exam questions features, users most frequently used the filtering by year feature (237 times) [11]. Future work will assess how Kwame for Science can improve learning outcomes.

In building Kwame for Science, we faced challenges (previously highlighted) related to accessing and using educational content from Ghana. We were unable to get official partnerships with local textbook publishers to use their copyrighted Science textbooks due to trust issues in the ecosystem. We addressed this issue by using global open-source Science textbooks and hiring local experts to provide answers to past exam questions. Furthermore, past national exam questions were only available in hardcopy formats and open-source OCR technologies we experimented with proved inadequate in accurately extracting scientific and mathematical symbols and equations from scanned copies of the documents. We addressed this issue by hiring individuals to annotate the content and exploring commercial solutions with advanced AI technologies. Future work will explore the use of Generative AI to generate contextual, local content based on local syllabi and content from our experts, and also extract well-formatted content from scanned documents.

Brilla AI is an AI contestant that we developed and deployed to unofficially compete remotely and live in the Riddles round of the 2023 NSMQ Grand Finale, the first of its kind in the 30-year history of the competition [12, 25]. This work is motivated by the lack of enough qualified teachers in Africa [4] which hampers the provision of adequate learning support. An AI could potentially augment the efforts of the limited number of teachers, leading to better learning outcomes. Yet, there exist no robust, real-world benchmark to evaluate such an AI. Towards that end, we built Brilla AI as the first key output for the NSMQ AI Grand Challenge, which proposed a robust, real-world challenge in education for such an AI: “Build an AI to compete live in Ghana’s National Science and Maths Quiz (NSMQ) competition and win — performing better than the best contestants in all rounds and stages of the competition” [26]. The NSMQ is an annual live science and mathematics competition for senior secondary school students in Ghana in which 3 teams of 2 students compete by answering questions across biology, chemistry, physics, and math in 5 rounds over 5 progressive stages until a winning team is crowned for that year [27].

Brilla AI is currently available as an open-source web app (built with Streamlit [28]) that live streams the Riddles round of the contest, and runs 4 machine learning systems: (1) speech-to-text (using Whisper [29]) (2) question extraction (using BERT [5]) (3) question answering (using Mistral [30]) and (4) text-to-speech (using VITS [31]) that work together in real-time to transcribe Ghanaian accented speech, extract the question, provide an answer, and then say it with a Ghanaian accent (Figure 9) [25]. In its debut in October 2023, our AI answered one of the 4 riddles ahead of the 3 human contesting teams, unofficially placing second (tied) [32, 25]. Improvements and extensions of this AI could potentially be deployed to offer science tutoring to students and eventually enable millions across Africa to have one-on-one learning interactions, democratizing science education.

Brilla AI addresses the challenge of biased AI systems that do not work well in the African context. Brilla AI contains models such as speech-to-text systems that work for African accents, text-to-speech systems that speak with African accents, and question-answering systems that provide answers using African education materials ensuring that AIED tools address the African context.