Educational Data Mining Conference -- Part Two

Exploratory learning environments, by their nature, allow students to freely explore a target domain. Nonetheless, the art of teaching compels us to provide appropriate, timely supports for students, especially for those who don’t learn well in such independent situations. While intelligent tutoring systems are designed to identify and deliver pedagogical and cognitive supports adaptively for students during learning, they have largely been geared to more structured learning environments, where it is fairly straightforward to identify student behaviors that likely are or aren’t indicative of learning. But what about providing adaptive supports during exploratory learning?

One of the key note lectures at the Third International Conference on Educational Data Mining conference in Pittsburgh in June (discussed in a previous blog posting) covered this topic. Cristina Conati, Associate Professor of Computer Science at the University of British Columbia, described her research using data-based approaches to identify student interaction behaviors that are conducive to learning vs. behaviors that indicate student confusion while using online exploratory learning environments (ELEs). The long term goal of her work is to enable ELEs to monitor a student’s progress and provide adaptive support when needed, while maintaining the student’s sense of freedom and control. The big challenge here is that there are no objective definitions of either correct or effective student behaviors. Thus the initial effort of her team’s work is to uncover student interaction patterns that correlate with, and thus can be used to distinguish, effective vs. ineffective learning.

Core to this “bootstrapping” process is the technique of k-means clustering employed by Conati and her team. K-means clustering is a cluster analysis method to define groups (e.g., students) that exhibit similar characteristics (e.g., interaction behaviors), and is commonly used in educational data mining research. Data from student use of two different college-level ELEs were used in the study: AIspace, a tool for teaching and learning artificial intelligence (AI), and the Adaptive Coach for Exploration (ACE), an interactive open learning environment for exploration of math functions. The data sets consisted of both interface action only or interface action with student eye-tracking. Identification of groups as high learners (and therefore presumably exhibiting largely effective behaviors) vs. low learners (and therefore presumably exhibiting largely ineffective behaviors) was determined either by comparing students’ pre- and post-test scores or through expert judgment. Formal statistical tests were used to compare clusters in terms of learning and feature similarity.

In the end, the data permitted distinction of two (one high learner and one low learner) and three (one high learner and two low learner) groups (i.e., k=2 and k=3) as a function of student behaviors. Differential behavior patterns include:

Low learners moved more quickly through exercises, apparently allowing less time for understanding to emerge.
High learners paused longer with more eye-gaze movements during some activities.
Low learners paused longer after navigating to a help page.
Low learners chose to ignore coaches’ suggestion to continue exploring current exercise more frequently than high learners.
Low learners appeared to move impulsively back and forth through the curriculum.

In summary, the research shows promise for k-means clustering as a technique for distinguishing effective from ineffective learning behaviors, even during unstructured, exploratory learning. Of course, this work is just a start. For example, additional research with larger numbers of students (24 and 36 students were used in the current studies) might support distinguishing of additional groups — should such additional groups exists. In the end, the hope is that by identifying patterns of behaviors that can serve as indicators of effective vs. ineffective learning, targeted, adaptive interventions can be applied in real-time to students to support their productive learning while maintaining the freedom that defines ELE learning.


Bob Dolan, Ph.D.
Senior Research Scientist, Assessment & Information
Pearson