Assessment Modeling: Fundamental Pre-training Tasks for Interactive Educational Systems

The first layer in the assessment model maps each interaction to an embedding vector. First, we embed the following attributes:

• •

  exercise_id: We assign a latent vector unique to each exercise id.

• •

  exercise_category: Each exercise has its own category tag that represents the type of the exercise. We assign a latent vector to each tag.

• •

  position: The relative position $t$ of the interaction $I_{t}$ in the input sequence. We use the sinusoidal positional encoding that is used in (Vaswani et al., 2017).

As shown in Example 3.2, IES collects diverse interactive features that can potentially be used for Assessment Modeling. However, not only using all possible interactive features for Assessment Modeling is computationally intractable, there is no guarantee that the best results on label-scarce educational tasks will be achieved when all the features are used. For experimental studies, we narrow down the scope of interactive features to the ones available from an exercise-response pair, the simplest and widely-considered interaction in Knowledge Tracing. In particular, we embed the following interactive features:

• •

  correctness: The value is 1 if a student response is correct and 0 otherwise. We assign a latent vector corresponding to each possible value 0 and 1.

• •

  elapsed_time: The time taken for a student to respond is recorded in seconds. We cap any time exceeding 300 seconds to 300 seconds and normalize it by dividing by 300 to have a value between 0 and 1. The elapsed time embedding vector is calculated by multiplying the normalized time by a single latent embedding vector.

• •

  inactive_time: The time interval between adjacent interactions is recorded in seconds. We set maximum inactive time as 86400 seconds (24 hours) and any time more than that is capped off to 86400 seconds. Also, the inactive time is normalized to have a value between 0 and 1 by dividing the value by 86400. Similar to the elapsed time embedding vector, we calculate the inactive time embedding vector by multiplying the time by a single latent embedding vector.

Let $e_{t}$ be the sum of the embedding vectors of exercise_id, exercise_category and position. Likewise, let $c_{t}$, $et_{t}$ and $it_{t}$ be the embedding vectors of correctness, elapsed_time and inactive_time, respectively. Then, the representation of interaction $I_{t}$ is $e_{t}+c_{t}+et_{t}+it_{t}$.

Inspired by the masked language model proposed in (Devlin et al., 2018), we use the following method to mask the assessments in a student interaction sequence. First, we mask a fraction $M$ of interactions chosen uniformly at random. If the $t$-th interaction is chosen, we replace embedding vectors of interactive features that are being predicted as targets with $mask$, a trainable vector that represents masking. For instance, if the correctness and elapsed_time are prediction targets for Assessment Modeling, $c_{t}+et_{t}$ is replaced with $mask$ and the embedding vector for interaction $I_{t}$ becomes $e_{t}+it_{t}+mask$.

After the interactions are embedded and masked accordingly, they enter a series of Transformer encoder blocks, each consisting of a multi-headed self-attention layer followed by position-wise feed-forward layer. Every layer has input dimension $d_{\mathrm{model}}$. The first encoder block takes the sequence of interactions $I_{1},\dots,I_{T}$ embedded in latent space and returns a series of vectors of the same length and dimension. For all $2\leq i\leq N$, the $i$’th block takes the output of the $(i-1)$’th block as input and returns the series of vectors accordingly. We describe the architecture of each block as the following.

The multi-headed self-attention layer takes a series of vectors, $X_{1},\dots,X_{T}$. Each vector is projected to latent space by projection matrices $W^{Q},W^{K}\in\mathbb{R}^{d_{\mathrm{model}}\times d_{K}}$ and $W^{V}\in\mathbb{R}^{d_{\mathrm{model}}\times d_{V}}$:

Here $X=[X_{1},\dots,X_{T}]^{T}$, and each $q_{i}$, $k_{i}$ and $v_{i}$ are the query, key and value of $X_{i}$, respectively. The output of the self-attention is then obtained as a weighted sum of values with coefficients determined by the dot products between queries and keys:

Models with self-attention layers often use multiple heads to jointly attend information from different representative subspaces. Following this, we apply attention $h$ times to the same query-key-value entries with different projection matrices for output:

Here, each $\mathrm{head}_{i}$ is equal to the output of self-attention in Equation 4 with corresponding projection matrices $W_{i}^{Q}$, $W_{i}^{K}$ and $W_{i}^{V}$ in Equation 3. We use the linear map $W^{O}$ to aggregate each attention result. After we compute the resulting value in Equation 5, we apply point-wise feed-forward layer to add non-linearity to the model. Also, we apply the skip connection (He et al., 2016) and layer normalization (Ba et al., 2016) to the output of the feed-forward layer.

Assume that the last encoder block returns the sequence $H=[H_{1},\dots,H_{T}]^{T}$ of vectors. For pre-training, the predictions to features in $i$’th timestep are made by applying a linear layer with the proper activation function to $H_{i}$. We consider four interactive features as prediction targets: correctness, timeliness, elapsed_time and inactive_time. If the prediction target is correctness or timeliness, the sigmoid activation function is applied to the linear layer output. If elapsed_time or inactive_time is the prediction target, the final output of the model is the output of the linear layer. The overall loss is defined to be

where $\mathcal{L}_{t}$ is the sum of binary-cross-entropy (resp. mean-squared-error) losses if the prediction target is correctness or timeliness (resp. elapsed_time or inactive_time). The value $m_{t}$ is a flag that represents whether the $t$-th exercise is masked ($m_{t}=1$) or not ($m_{t}=0$). The input embedding layer and encoder blocks are shared over pre-training and fine-tuning. For fine-tuning, we replace the linear layers applied to each $H_{i}$ in pre-training with a single linear layer that combines all the entries of $H$ to fit the output to label-scarce educational downstream tasks.

Exam score prediction is the estimation of student’s scores in standardized exams, such as the TOEIC and the SAT, based on the student’s interaction history with IES. Exam score prediction is one of the most important tasks of AIEd, as standardized assessment is crucial for both the students and IES. Because a substantial amount of human effort is required to develop or take the tests, the number of data points available for exam score prediction is considerably fewer than that of student interactions automatically collected by IES. By developing a reliable exam score prediction model, a student’s universally accepted score can be estimated by IES with considerably less effort. Exam score prediction differs from response correctness prediction because standardized tests are taken in a controlled environment with specific methods independent of IES.

Assume that a student incorrectly responds to an exercise $e_{\mathrm{rev}}$ and receives corresponding feedback. The goal of review correctness prediction is to predict whether a student will be able to respond to the exercise $e_{\mathrm{rev}}$ correctly if they encounter the exercise again. The significance of this AIEd task is that it can assess the educational effect of an exercise to a particular student in a specific situation. In particular, the correctness probability estimated by this task represents the student’s expected marginal gain in knowledge as they go through some learning process. For example, if the correctness probability is high, it is likely that the student will obtain relevant knowledge in the future even if their initial response was incorrect.

For pre-training, we first reconstruct the interaction timeline of each student by gathering the responses of a specific student in increasing chronological order. For each interaction $I_{t}$, $\emph{correctness}_{t}$ is recorded as 1 if the student’s answer is correct and 0 otherwise. $\emph{timeliness}_{t}$ is recorded as 1 if the student responded under the time limits recommended by TOEIC experts (Table 1) and 0 otherwise. $\emph{elapsed\_time}_{t}$ is the time taken for the student to respond recorded in seconds, and $\emph{inactive\_time}_{t}$ is the time interval between the current and previous interactions recorded in seconds. We exclude the interactions of students involved in any of the label-scarce educational downstream tasks for pre-training to preemptively avoid data leakage. After processing, the data consists of 414,375 students with a total of 93,121,528 interactions.

For exam score prediction, we aggregate the real TOEIC scores reported by students of Santa. The reports are scarce in number because a student has to register, take the exam and report the score at their own expense. To collect this data, Santa offered a small reward to students in exchange for reporting their score. A total of 2,594 score reports were obtained over a period of 6 months, which is considerably fewer than the number of exercise responses. For our experiment, we divide the data into five splits, and use 3/5, 1/5 and 1/5 of the data as training, validation and test set, respectively.

For review correctness prediction, we look over each student’s timeline and find exercises that have been solved at least twice. That is, if an exercise $e$ appears more than once in a student interaction sequence $I_{1}=(e_{1},r_{1}),I_{2}=(e_{2},r_{2}),\dots,I_{T}=(e_{T},r_{T})$, we find the first two interactions $I_{i}$ and $I_{j}$ ($i<j$) with the same exercise $e$. The sequence of interactions $I_{i+1},\cdots,I_{j-1}$ and $e_{j}$ are taken as input, and $\emph{correctness}_{j}$ is taken as the label. The total of 4,540 labeled sequences which are not appeared in the pre-training dataset are generated after pre-processing. Similar to the case in exam score prediction, we divide the data into five splits, and use 3/5, 1/5 and 1/5 of the data as training, validation and test set, respectively.

Our assessment model consists of two encoder blocks ($N=2$) with a latent space dimension of 256 ($d_{model}=256$). The model takes 100 interactions as input. For pre-training, the model parameters are first randomly initialized and trained with 0.6 making rate $M$. Then we replace the last layer of the pre-trained model with a linear layer appropriate for each label-scarce educational downstream task, and fine-tune the whole parameters of the model. In fine-tuning, to alleviate label-scarce problem, we apply the following data augmentation strategy. Given the original interaction sequence with a label, we select each entry in the sequence with 50% probability to generate a subsequence with the same label. The model is fine-tuned on these subsequences. In both pre-training and fine-tuning, dropout rate and batch size are set to 0.2 and 128, respectively. We use the Adam optimizer (Kingma and Ba, 2014) with $lr=0.001,\beta_{1}=0.9,\beta_{2}=0.999,epsilon=1e-8$, and Noam scheme (Vaswani et al., 2017) to schedule the learning rate with 4000 warmup-steps. We conduct 5-fold cross validation to select the model with the best result on the validation set, and report the evaluation result of the model on the test set.

We compare the effectiveness of the assessment model with the following content-based pre-training methods. Since the existing content-based pre-training methods learn embedding vectors of each exercise, we replace the embedding of exercise_id in our model with respective exercise embedding for fine-tuning.

• •

  Word2Vec (Mikolov et al., 2013) is a standard word embedding model used in many tasks. (Huang et al., 2019) used word embedding vectors obtained from Word2Vec model to generate exercise embedding vectors. In our experiment, we obtain 256 dimension word embedding vectors from the continuous bag-of-words Word2Vec model trained on text corpus consists of exercise descriptions. The embedding vector assigned to each exercise is the average of embedding vectors of all words appearing in the exercise description.

• •

  BERT (Devlin et al., 2018) is a Transformer encoder-based bidirectional representation learning model used in various domains. The original BERT was pre-trained by masked language modeling and next sentence prediction objectives. However, several subsequent works (Liu et al., 2019; Lan et al., 2019) questioned the necessity of the next sentence prediction objective. Accordingly, we train 2 layers and 256 dimension BERT model using only masked language modeling objective on exercise descriptions. Similar to the case in Word2Vec, we obtain each exercise embedding vector by averaging the embedding vectors of words in the exercise description.

• •

  QuesNet (Yin et al., 2019) is a content-based pre-training method learning unified representation for each exercise comprises of text, images and side information. Following the method suggested in the original paper, we train bi-directional LSTM followed by multi-headed self-attention model with holed language modeling and domain oriented objectives. The embedding vector of each exercise is computed as the sentence layer output of the model when the exercise is taken as input.

We use the following evaluation metrics to evaluate model performance on each label-scarce educational downstream task. For exam score prediction, we compute the Mean Absolute Error (MAE), the average of differences between the predicted exam scores and the true values. For review correctness prediction, we use Area Under the receiving operating characteristic Curve (AUC).

We demonstrate the importance of choosing appropriate pre-training tasks by comparing models pre-trained to predict one of correctness, correctness + elapsed_time, correctness + timeliness, and correctness + inactive_time. Since correctness is the feature with the most pedagogical aspect, we include correctness in the prediction targets of all the pre-training tasks. The results are shown in Table 2. For both exam score and review correctness prediction, the results show that it is the best to pre-train the model to predict correctness and timeliness. This is because the exam score and review correctness distribution are related to not only correctness but also timeliness of the student’s answers. This shows the importance of choosing appropriate pre-training tasks relevant to the downstream tasks.

Experimental results comparing Assessment Modeling with three different content-based pre-training methods and the model without pre-training are shown in the Table 3. In all downstream tasks, Assessment Modeling outperforms the other methods. Compared to the model without pre-training, MAE for exam score prediction is reduced by 21.96% and AUC for review correctness prediction is increased by 6.99%. Also, Assessment Modeling improves 13.34% MAE for exam score and 4.26% AUC for review correctness prediction, respectively, from the previous state-of-the-art content-based pre-training methods. These results support our claim that Assessment Modeling is more suitable for label-scarce educational downstream tasks than content-based pre-training methods.