"When training an AI model, students' emotions, motivation, metacognitive data and cognitive abilities must all be taken into account," says Danial Hooshyar, professor of artificial intelligence at Tallinn University. Together with his research group, Hooshyar is seeking to develop and analyze an AI-based algorithm that incorporates these factors.

The five-year project will enable digital learning environments to make use of AI's potential beyond large language models alone. Initially, the researchers are focusing on eighth-grade mathematics, testing various tools and metrics in a digital learning environment. "The entire goal of the project is to help students regulate their own learning, rather than simply giving them AI tools to look up answers," the professor explained.

In a recent study, the researchers tested the predictive capabilities of their AI-based algorithm using a dataset collected from U.S. students, comparing the new algorithm with an LLM. "Our model made significantly fewer errors than the LLM, which is very good. At the same time, the LLM made illogical decisions when predicting knowledge levels — these models have not been trained on datasets based on student learning," Hooshyar recalled.

From Estonian scientists to Estonian students

Earlier this year, Danial Hooshyar's research group studied the impact of tools designed to support the regulation of emotion, motivation and cognition on learning. Nearly 700 Estonian eighth-grade students tested three versions of the Opiq platform, none of which yet use AI. One version included all of the new tools, another contained only cognition-related additions, while the third was a traditional digital textbook.

"The aim of the study was to examine the effect of self-regulated learning tools. We collected log data on how students behave while learning in a digital environment," the professor explained. The research group will next use the data to train an AI model capable of predicting skill levels while also taking into account students' own assessments of their learning goals, as well as emotional and motivational factors that may affect learning.

The model's predictive capabilities will be used to support self-regulated learning, with three different modes currently under development. The first relies entirely on recommendations and guidance generated by AI. The second combines AI and learner input, offering recommendations while leaving the final decision to the student. In the third mode, the learner independently makes all decisions related to studying. The entire process is supported by an open learner model that provides real-time information about the user's learning progress.

According to Hooshyar, the model under development gains additional value from being trained specifically on data from Estonian students and from not relying on an existing large language model (LLM). Over the next two years, the researchers plan to test the algorithm in schools. "We will test an upgraded smart digital learning environment in schools to see its impact without an LLM," the professor said, outlining the team's future plans.

AI needs transparency and cooperation

According to Danial Hooshyar, great caution is needed when developing AI for use in schools. "It can be compared to a new smart car that is faster, smarter and more comfortable, but once it enters real traffic, the system begins behaving unpredictably and problems emerge that developers try to fix one by one. Although the car improves after each fix, new issues may also arise," he illustrated.

Using the car analogy, he stressed that LLMs should not be rushed to market for widespread use unless they have been responsibly developed with safety in mind from the outset. Education, he added, is not simply another market, meaning responsibility cannot be postponed to later fixes. Hooshyar identified the so-called "black box" nature of LLMs as one of the main risks, since they do not explain how their decisions are made. His research group currently has no plans to incorporate LLMs into its development work. "The so-called white-box, or interpretable, models we have developed are able to explain their predictions based on data," he noted.

The professor also emphasized the importance of involving social scientists and school practitioners throughout the entire development process of AI solutions intended for education. "If we do not work together with them, that is where mistakes will emerge. Only through cooperation can we be sure that models are developed and used responsibly and correctly," Hooshyar said.

--

Follow ERR News on Facebook and X and never miss an update!