From Automation to Cognition: Redefining the Roles of Educators and Generative AI in Computing Education

We teach at a large urban research-focused institution. Authors are primarily involved in teaching classes of approximately 1000 first-year Engineering students learning to program in C, and classes of approximately 300-500 first-year Computer Science students taught in Python. In both cases, students are engaged in weekly laboratory activities conducted in person and supported by graduate teaching assistants. The laboratory activities are submitted to CodeRunner, where the correctness of programming solutions is graded automatically using a test suite. Students also engage in other activities manually graded by the teaching assistants or involving other online tools (e.g., those described later in this paper). At the undergraduate level, authors also have experience teaching second-year systems, and third-year computer graphics courses.

Since the release of ChatGPT (OpenAI, 2024), we have seen a steady increase in student use of GenAI in CS classrooms. During the first few months after ChatGPT’s release, students were still wary of the adverse consequences of using GenAI in classrooms, as the majority of students were still uncertain of its performance and accuracy or were unaware of its existence, and those who knew its capabilities were concerned with potential violations of academic integrity through its use.

However, as GenAI grew in popularity, more students became aware of its capabilities, and the use of ChatGPT and other GenAI-powered tools became more prevalent. Despite the guidelines that prohibited using GenAI on course pages and assignment/assessment specifications, there was evidence that many students still used GenAI against the guidelines. For example, the average scores achieved by students for take-home assignments were much higher than those for in-person invigilated assessments. Furthermore, we observed in assessments of an introductory programming course that a few students struggled to write simple print statements despite having completed all prior take-home programming assignments, often using sophisticated code constructs and solution approaches. There were also a small number of cases of students accessing tools like ChatGPT during invigilated assessments such as tests and quizzes, in violation of the rules. We are aware that there are many reasons that students struggle and the difficulties observed are not all necessarily caused by GenAI. Even so, publicly available tools, such as ChatGPT, have made it possible to complete some take-home assignments without engaging in the learning process by providing answers to coursework at no cost or risk. This enables less motivated students to generate solutions and submit them as their own without engaging in learning. However, in contrast, we found that more motivated students were less likely to misuse GenAI and instead use it to their advantage to support their learning, such as by asking constructive questions and receiving immediate feedback to clarify confusion or misunderstanding that may arise during learning. Our observation aligns with the work of Prather et al., who found that while tools like ChatGPT and Copilot can help motivated students accelerate their work on programming tasks, struggling students often face persistent and compounded metacognitive difficulties when using these tools (Prather et al., 2024). As a result of our own observations, we saw significant importance in teaching students effective ways of using GenAI to support their learning.

As time passed and GenAI became ubiquitous, it became clear that policy changes had to be made to adapt, and simply prohibiting the use of GenAI was not the solution. In fact, the prohibition of GenAI during an academic programme may cause difficulty for graduates expected to transition to work environments where GenAI may be encouraged to boost productivity. Therefore, we began to allow, and even incorporate, GenAI in take-home assignments (several concrete examples are given in Subsection 2.3), so it became commonplace for students to use GenAI, even in the presence of teaching staff. But to ensure that students were engaging in learning and not reliant on GenAI, we enforced strict standards for some assessments, particularly secure (in-person, invigilated, and GenAI-prohibited) or online using secure assessment tools. Furthermore, we highlighted to students the nature of the assessments and, thus, the importance of learning and understanding the concepts even with the use of GenAI.

Anecdotally, over time we have noticed a reduction in student queries regarding concepts or questions related to programming exercises, and relatively higher percentages of student queries related to course administration. This is presumably because many students are making use of GenAI for receiving fast and often detailed answers.

As students become increasingly familiar with using commercial AI tools, like ChatGPT, we recognise the importance of integrating GenAI into classroom activities to align with their expectations. Over the last few years, we have explored several activities that leverage large language models (LLMs) to provide unique and engaging ways for students to interact with computing concepts. These activities aim to develop students’ problem-solving, debugging, and conceptual understanding skills through meaningful interaction with LLMs. To develop critical thinking skills, we have also explored scaffolded tasks where students seek help from LLM-based teaching assistants and evaluate their outputs. Below, we outline six key activities explored in our first-year introductory programming courses.

GenAI use is still not widely accepted in education, and using GenAI tools to assist with writing assignments is typically disallowed (Perkins, 2023). This is a sensible approach where the assignments have not been adapted for a GenAI context — most assignments currently used in courses are designed to be solved individually and not with the assistance of GenAI. However, since GenAI tools are becoming more widely used, we believe that restricting GenAI use in all assignments is unrealistic and does not reflect real-world conditions. Hence, our view is that at least some assignments should be designed to support learning with GenAI.

For example, the Prompt Problem exercises described in Subsection 2.3.1 require students to engage with problem specifications, an authentic task required by programmers writing programs to solve problems outside the highly constrained educational context. This helps students develop important dispositions outlined in Computing Curriculum 2023 (Kumar et al., 2024), such as being meticulous and persistent while raising awareness of the capabilities and limitations of GenAI tools.

Activities such as peer code review have been shown to benefit students producing reviews (Indriasari et al., 2020). These activities require students to understand and critique the code produced by others. One potential barrier to such activities is the difficulty of sharing peer code in classrooms — an administrative burden that typically requires a dedicated peer review tool. The same activity (and indeed any activity that requires students to engage with work produced by others) can use GenAI to create the code to be reviewed. Students can be asked to reflect on the quality of the code and comment on differences between the generated code and their solutions. This approach retains the focus on traditional learning outcomes for a given assessment, yet it raises student awareness of GenAI capability (and potentially the limitations of such tools where the quality of the generated code has room for improvement).

As a general example, any traditional assignment involving code writing could be redesigned to focus on code comprehension and encourage students to link concrete examples more explicitly to more abstract concepts. Figure 3 is an example of a typical code writing question used in a CS1 course. Some students may query GenAI for the solution and submit it as their work without processing the information. This can occur whenever we focus assessment on the product, since GenAI is excellent at solving programming problems. It is possible to redesign this question to focus on more abstract levels of understanding programming concepts and encourage students to use GenAI tools for scaffolding rather then simply producing solutions. Figure 4 guides students to use the capabilities of GenAI tools as a scaffolding tool to support learning rather than replace learning. Although this does not guarantee productive learning, it lowers the likelihood of students copying and pasting without reading and understanding the responses. Assignments like this also align with the common suggestion of shifting towards assessing students on the learning and completion process rather than the final product (Cao and Dede, 2023; Prather et al., 2023).

The focus of the redesigned assignments is to assist with learning rather than to evaluate learning. Although we believe that GenAI should be used in assignments, we also believe it is beneficial for courses to include components of secure assessments (in-person, invigilated, and GenAI-prohibited) to ensure that we measure student capability for accreditation purposes and to focus attention on that capability rather than relying on an outside source (such as GenAI) to produce outputs that students are expected to complete themselves. We enforce secure assessments by having students submit their code on a web-based automated marking tool, such as Coderunner (Lobb and Harlow, 2016), and a firewall that only allows Internet access to the said marking tool. This eliminates the possibility of students using GenAI during secure assessment components.

A potential challenge arising from more qualitative assignments, or focusing on process rather than product, is the increased workload for educators to grade student submissions. Using the traditional grading method of manually reading submitted material may be ineffective, and innovative methods may need to be devised to speed up the grading process. One possibility is to design a proxy for students to access GenAI tools that store interactions that may be used to automate grading purposes. The requirements of the grading process may vary between assignments, but they should be considered when designing the assignments.

The redesign of assignments also raises concerns regarding the correctness of responses generated by GenAI since students risk counterproductive learning if the generated information is incorrect or misleading. However, instead of perceiving this as counterproductive learning, educators must teach students to identify incorrect information by fact-checking using trusted non-AI-generated resources such as textbooks and teaching staff, which is an essential skill in a discipline where much of the recent content knowledge is acquired through community sources (such as forums, blog posts, and online documentation). This can strengthen student understanding of topics and simultaneously improve their metacognitive skills.

With increasing online resources, students are becoming less reliant on educators, as their first source of study assistance is usually the Internet (Hou et al., 2024a) because Internet searches supply answers instantly. One main advantage of educators’ responses to student queries is that they provide personalised feedback that answer specific details of the questions that online searches may not supply. However, now GenAI tools are easily accessible and students can receive unlimited, immediate, and personalised feedback. Consequentially, we have observed fewer help requests from students in our computing laboratories, as described in Subsection 2.2. This indicates that we may better support student learning by increasing attention to skills students cannot learn simply with assistance from GenAI, such as metacognitive skills crucial when learning CS (Prather et al., 2020).

In Subsection 2.2, we outline the differences in the approaches of using GenAI between more motivated students and those less motivated, which may also be attributed to the differences in metacognitive skills, where more motivated students understand the harmful effects of using GenAI without engaging in learning. In contrast, those less motivated may not consider the future effects of copying from GenAI, such as the lack of understanding leading to difficulties during assessments. We believe that it should be the educators’ responsibility to highlight the risks of copying from GenAI and the correct and most effective ways of using GenAI when learning, such as asking GenAI to refrain from providing full solutions and instead provide explanations and examples.

Kazemitabaar et al. (2024a) present a variety of GenAI-supported tasks that align with different levels of Bloom’s taxonomy. These tasks require tools designed to scaffold learning with GenAI and could be introduced into CS classrooms with guidance that raises awareness of how the activities impact learning and the role of GenAI in that process.

More generally available tools, such as ChatGPT, are more likely to be used in practice by students working independently on programming tasks. The way students assemble their prompts significantly influences the content, thus usefulness, of the responses, so educators should teach students about the characteristics of effective prompts for students. Teachers can easily demonstrate how to prompt ChatGPT and the impact of different prompts. The Prompt Problems exercise described in Subsection 2.3.1 illustrates how guidance about prompts could subsequently be assessed as part of a course activity.

For example, in the scenario of a CS1 student attempting to understand the process of writing a prime checking function with the assistance of ChatGPT, Figure 5 shows an example of a general prompt, where the explanation lacks details and may be difficult to understand for a novice programmer. Additionally, it provides the full model solution, which reduces the learning that occurs as students explore the solution space. In contrast, Figure 6 shows an example of a prompt requesting no code and only explanations. This is more valuable for learning than the general prompt, as the students can then attempt to write the code themselves from the information generated by ChatGPT. If students fail to write the code themselves, they can use the prompt shown in Figure 7, which expects a full code solution but with further comments and details. As expected, the corresponding explanation and code are more comprehensive and explicit than that shown in Figure 5, even providing explanations for the use of the if-statements and for-loops. Using the prompts shown in Figure 6 and Figure 7, and observing the corresponding annotated outputs, may help a student develop a deeper understanding of the problem and the computational steps involved, compared with only using the prompt shown in Figure 5.

The characteristics of effective prompts can vary across courses and disciplines. Therefore, educators should provide course-specific guidance on crafting good prompts for queries within the context of their course. Additionally, students need to recognize that GenAI responses are not always accurate, and they must develop the ability to identify misinformation. This process can serve as both a valuable learning experience and an important metacognitive skill.

AI-generated content comes in various forms, each potentially requiring different fact-checking methods. Educators should demonstrate the fact-checking techniques most relevant to their courses to teach students how to identify misinformation effectively. For example, educators can present snippets of AI-generated content and walk students through the process of verifying this information using course materials, textbooks, documentation, and trusted websites. Similarly, educators can demonstrate how to test AI-generated code for correctness by running it. If the code produces incorrect results, educators can showcase the debugging process, which is an essential skill, particularly as code generation becomes more prevalent.

The CE landscape will inevitably undergo further significant change due to GenAI. While many new teaching practices have emerged in response to GenAI, such as those outlined in this work, we argue that further research is essential to evaluate their validity and effectiveness. This includes trialling innovative teaching methods, gathering student feedback, and assessing student performance. It is critical to thoroughly investigate the concrete impacts of GenAI and proactively prepare for these changes before the research gap becomes too wide. Neglecting this need could result in reduced student productivity due to ineffective GenAI usage and leave graduates ill-prepared because of outdated teaching practices.

We believe that future graduates must be proficient and experienced in leveraging GenAI to support their work. Therefore, methods for effectively using GenAI should be integrated into teaching curricula. We urge the CE community to prioritize research that examines the tangible impacts of GenAI, including its influence on course structure, content, and teaching methods. Such research will enable educators to make well-informed decisions as we adapt our teaching practices to the evolving role of GenAI in education.