Learnings from LTAG - Dr Adrian Hunter

University teaching across Australia and internationally is shifting away from traditional lectures toward more interactive forms of learning. At Adelaide University, the focus is on “rich digital learning activities”, online resources that students can access at their own time and pace which complement and strengthen the time spent with educators. For example, short, captioned videos fit the attention spans typical of modern online learning and are an efficient way for instructors to reach large cohorts. However, watching a video online is intrinsically less engaging than being in the presence of a passionate educator. Consider that anyone can watch free recordings online of concerts that would have cost hundreds of dollars to attend in person – yet hardly anyone does. The question, then, which Dr Michelle Coulson and I tackled in this project, is how to design online videos to help students learn as effectively as they would in live classes.

How can we improve online videos?
One approach is to embed questions that ask students about what they’ve just learned. This changes video watching from a purely passive activity into an active one, and students listen more carefully in anticipation of the imminent questions. Actively recalling information just heard helps consolidate that information into long-term memory.

These benefits are well established in psychology and education research, but we wanted to see whether they hold in an authentic university environment. So we showed two matched sets of videos to students from two large first-year biology cohorts but switched which set of videos included embedded questions between cohorts. We then compared each cohort’s subsequent performance in low-stakes open-book quizzes held shortly after the videos, and later in the relevant sections of the end-of-semester exams:

The likely explanation is that each year, more students use generative AI browser plug-ins for “help” in answering online quizzes.  This inflated the quiz scores of the later cohort. The invigilated end-of-semester exam then exposed that cohort’s lesser knowledge, as fewer of them had benefited from a genuine attempt at the quiz.

Did you give up and conclude the project was a failure?
No!  We realised we could analyse our data another way.  By pooling results from both cohorts, we could compare performance on topics where the videos did include embedded questions against those where they didn’t:

And what did you find?
Students performed significantly better when the online videos contained embedded questions.

Quiz scores for those topics were 5% higher (p < 0.001), and exam scores were 3% higher (p < 0.05) – even though the exam took place about eight weeks later: