According to research, students have approximately 30% of the time when engaged in learning on a digital platform where they are distracted and lose concentration.
Another recent study found only 7% of educational technology organizations have done any form of randomized controlled testing to validate their offerings.
This guide will identify what exactly occurs technically when digital technologies change the behaviour of learners. It will also explore how does ed-tech improve engagement and provide detailed criteria for measuring cognitive engagement versus superficial interaction.
Understanding the Dimensions of Learner Engagement
In order to assess educational technology, we first need to define what we mean by engagement.
The way commercial enterprises report massive increases in engagement levels is based on statistics that reflect the total number of daily active users, click-throughs, and similar metrics. However, these data sets simply show what a user has done, not their level of cognitive engagement with the content they are accessing.
If we are to assess engagement properly, we need to break engagement down into specific, measurable dimensions.
Behavioral vs. Cognitive Measures
Behavioral engagement provides the most basic form of measure for engagement. It includes measures of participation, persistence, attendance, and task completion. For example, if a learner logs on to a learning management system (LMS) and clicks through a series of modules, the LMS will show that the learner has been positively behaviorally engaged.
However, behavioral data alone are not adequate for evaluating the quality of engagement. A learner can spend three hours on a learning platform for one of two reasons: they are very engaged with the content, or they are struggling to navigate the interface and find their assignments.
Cognitive engagement represents actual cognitive processes. Cognitive engagement includes elaborate reasoning, logical reasoning, and strategy development. When using an educational technology platform, users are required to actively retrieve previously-stored knowledge, apply previously-learnt information, or provide a rationale for their choice of strategy when completing the activity.
When a student rapidly clicks through a video, they are exhibiting rapid click behavior, but they are not engaged cognitively. However, when a student pauses the video to respond to a highly detailed diagnostic question, they demonstrate cognitive engagement.
Emotional and Social Engagement
Emotional (affective) engagement encompasses interest, confidence, belonging, and enjoyment, as well as boredom and academic results. Digital elements that minimize frustration and provide visible progress may enhance emotional engagement.
Social engagement reflects how often a learner interacts with another individual to enhance understanding. For example, when other learners ask questions, debate responses, or share resources, those learners are exhibiting high social engagement. The best educational technology will not isolate the learner behind a computer screen. Rather, it will leverage the screen as a tool for facilitating better interaction among individuals.
How Does EdTech Improve Engagement?
Educational technology will not inherently improve learning merely because it is expressed digitally. Educational technology will improve learning where the educational technology is designed to do the following: make content relevant to the learner, create an environment that encourages active participation, rapidly provide feedback, and facilitate social connections among learners.
Adaptive Learning Pathways and Task Difficulty
When a task is less challenging than one can easily accomplish, then the learner will typically lose interest in the activity. Conversely, if the difficulty of the task is very high, the learner will tend to feel overwhelmed and stop trying to complete the task. When using the traditional classroom model, a single educator must typically attempt to deliver an identical lesson to 30 different learners simultaneously.
Adaptive learning software was specifically designed to address this issue. The software manufacturers collect and analyze every response provided by a learner. Based on those analyses of responses, they create a data model of a learner’s current knowledge; therefore, the software can end up allowing each learner to efficiently achieve mastery of all learning objectives.
When a student answers 3 consecutive algebra problems correctly, the system immediately increases the difficulty level of the next question. If the student answers the 4th question incorrectly, the system will provide them with foundational materials and review the lower-level question items to help them succeed.
This continual fluctuation of difficulty keeps the student engaged in their learning experience. Using this technology keeps a student from becoming bored or overly frustrated while they learn in an inclusive classroom where multiple ability levels exist. Teachers can then use the information collected regarding errors to create focused, small-group interventions.
Active Participation and Interactive Quizzes
To truly engage with materials in a way that leads to true understanding, a student must shift from passively reading material (on a digital device or paper) to actively using the material to create their own understanding of the material.
By forcing a learner to be active, technology enhances engagement. A completely self-paced online course using only 1-hour video lectures will see high abandonment rates. To counter this, instructors can add short, required quizzes for retrieval every 8 minutes.
Retrieving facts, explaining concepts, or categorizing information on a quiz will make the learner's experience active. The best indicator of success doesn't depend on how many videos the learner has viewed; it depends on how often he or she has participated in the retrieval quizzes and the quality of the answers provided during those activities.
Immediate Feedback and Error Correction
A learner may complete a paper assessment on Tuesday and receive their grade on Friday. By the time they receive their grade, they will have forgotten the way in which they originally reasoned through their answers. Learning loops are destroyed by long delays.
Digital platforms solve this dilemma by providing immediate feedback when a student enters an incorrect answer. This immediate response causes students to change the way they learn. They can see why they were wrong while the original logic is still fresh in their brain, process the information immediately, and try again.
The key to the operation of this system is the amount of time between an incorrect answer and another attempt. A fast feedback loop creates a cycle of continuous effort, which is what sustained engagement means.
Gamification Driven by Mastery
Gamification is the most misunderstood system in education. Badges, points, streaks, and leaderboards do not contribute to sustained engagement when they are connected to meaningless tasks. When a student logs into a program and receives a badge, they learn to value that login instead of the learning content.
Gamification is effective when tied to the mastery of an academic concept. Students may feel as if they are at the end of the line with a complex language issue, but a well-designed digital game will provide visual and incremental goals. This allows the student to break down a large subject into many small steps, all of which can be accomplished.
When students see the progress bar on the screen and see that they are moving forward only as a result of true understanding, the game mechanics encourage cognitive effort. The visual progress creates tenacity in students, encouraging them to return to the software many times to advance to the next level of actual mastery.
Social Accountability Systems
Some designers of digital learning assume that the digital learning experience must be done alone. However, digital collaborative tools often demonstrate the highest levels of sustained effort.
Digital collaboration, such as peer-reviewed assignments and group problem-solving, creates social responsibility among peers. As students recognize their peers depend on their contributions for virtual assignments, they will be less inclined to abandon a given task. The technology serves as a central hub where humans stay connected.
These features are especially important in online education. Mobile solutions with low-bandwidth discussion forums provide a mechanism for solitary students to communicate to each other, compare and contrast notes, pose questions, and continue participating in the course despite the geographical separation between them and their peers.
Using Learning Analytics to Be Proactive
In the absence of assistance from an instructor, a student who is struggling will be unable to receive help until someone brings it to the instructor's attention.
An absence of information regarding a struggling student could mean the instructor is unaware that the student has ceased to attend class, stopped reading, and is failing before the student has received notice of a failed test.
Learning analytics modify the traditional mode of providing the instructor with information regarding the performance of students after the fact. They provide the instructor with a means to offer an early warning to students regarding their academic performance.
The instructor will be able to view the performance of each student through the use of an electronic dashboard.
This monitors the student's performance in terms of assessment attempt statistics, the time it has taken for each student to complete their required modules, and the frequency with which the student has logged into the system. Whenever a student's performance deviates from the baseline trends established for the previous history of the student, the student's account will be flagged for attention.
Learning analytics do not re-engage students but will notify the instructor when an issue is present. This allows the instructor to contact the student with a targeted message or invite them to attend office hours at a time that is relevant for the student. The instructor can provide help to the student at the moment the student requires it, rather than waiting until the end of the course.
Experiential Simulations
Many areas of study are difficult, expensive, or too risky to practice in a traditional environment, such as vocational education and medical training. Educators and trainers will frequently utilize technology in order to replicate the operational environment. They use technology in conjunction with experiential simulations to provide a mechanism to facilitate the transition from the theoretical to the practical application of knowledge and skills.
Virtual simulations give learners an opportunity to practice making choices and to observe and learn from the consequences of their actions without risk of real-world consequences. Students can engage in a simulation where they can safely fail with an electric grid while learning how to troubleshoot.
The engagement value of this type of learning strategy stems from how quickly students make decisions and receive feedback on their decisions. This type of engaged practice is much more effective at teaching electrical safety, compared to simply reading a manual.
Measuring the Effectiveness of Digital Tools
The first step an institution should take when purchasing or implementing a digital tool to increase student engagement is to have an objective measurement system in place to track results and evaluate how does ed-tech improve engagement objectively. Accepting vendor assertions about the effectiveness of their products without conducting internal verification is a mistake.
The second step is to isolate the metrics associated with activity versus mastery. A national education data set that included more than 69,000 students demonstrated that raw activity statistics do not predict student learning outcomes as effectively as some other factors. Therefore, if an institution only tracks total hours logged in the system, they will mistake confusion for engagement.
To assess whether or not a digital tool is effective, administrators must separate the basic activity measures from the mastery measures. They need to examine the activity of the student in the system before and after they received corrective automatic feedback.
The final step in measuring the effectiveness of a digital simulation tool for student learning is creating valid baselines. An institution must measure the completion rates, attendance rates, and assessment scores of students for four to eight weeks prior to the release of the new software so as to create baseline data against which to assess the success of the new application.
Once the institution has created a valid baseline and is ready to release the application out to the entire institution, they should first run a pilot program. This should occur within a selected sample of students (the experimental group) versus a sample of students who do not receive the application (the control group). The data collected from the pilot program will need to be analyzed very closely to identify any patterns that emerge related to the experimental group's retry and persistence after receiving corrective automated feedback, including:
Days between a first incorrect response and a second response.
Mapping the progression to where a student would have historically performed on a mastery score from the baseline.
Recovery and reactivation of students previously at risk.
Where those numbers show improvement because of the use of the technology versus the baseline, this indicates that the user is "engaged." If screen time increases but those numbers do not increase, the user is being distracted through the technology.
Why Educational Technology Sometimes Fails
Technology amplifies what works well with good pedagogy, but it also amplifies what is not working with poor pedagogy. It is imperative that any evaluation be a true audit to identify the points where educational technology does not deliver better engagement for learners.
Distraction and Cognitive Load
The cognitive load of learners is limited and overwhelmed when software is poorly designed. The poor design of software creates excessive amounts of visual clutter or noise. Learners cannot concentrate on learning, but instead, they spend their energy determining how to navigate through all of the things that distract them.
If there are constant pop-up notifications, lengthy menus, and auto-playing videos, the cognitive load is created to such an extent that learners' mental energy is expended on figuring out how to use the system instead of engaging with the content.
As a direct result, the tool becomes an active disincentive to engagement. In fact, educational researchers have found that more than a third of all teachers believe that devices primarily serve as a distraction in a typical classroom setting.
Bandwidth and Hardware Barriers
Even the best artificial intelligence tutor in the world cannot help learners if they lack a steady internet connection.
In many instances, designers of educational technology design for high-end technology and high-speed connections.
However, in reality, there are millions of students who access and use the content through shared mobile phones or that cannot afford expensive data plans. If a platform requires learners to download huge amounts of video, then those learners will not access it, which equates to zero engagement for those users.
A data-driven evaluation must look to identify common examples of system failure prior to embarking upon a digital strategy:
Replacing direct human instruction with unguided, open-ended device time.
Deploying superficial rewards (or reward systems) without clear academic objectives.
Taking automated algorithm-generated predictions to be "absolute" or final assessments of ability.
Ignoring the hardware needs of the student.
Aligning Technology with Teaching
Educational technology, in and of itself, does not produce engagement. The application of digital code applied to a learning environment leads only to increased costs, hardware, and screen time.
Grasping how does ed-tech improve engagement requires recognizing that true learner engagement is the result of the instructional design process.
The reason that digital tools have proven highly successful is that they fundamentally change how learning occurs. Digital tools provide a means of adjusting the level of difficulty in a learning experience so that learners are not frustrated by learning. Digital tools also provide means of generating immediate feedback compared to a traditional delayed assessment process.
Digital tools also require that passive readers become active problem solvers and provide teachers with the data necessary to intervene and help learners before they become completely disconnected. Institutions that view technology as a substitute for human instruction will see a consistently decreasing level of engagement.
Conversely, an institution that uses digital technology as an operational method of eliminating barriers to access, accelerating the rate of feedback, and providing opportunities for repeated successful implementations will see measurable and verifiable improvements in student outcomes. The ultimate value of developing educational technology is determined entirely by the strategic logic of its implementation.