Skip to main content

ECoach pilot: Can messages and nudges influence student success?

Image
By
This is the first post in a two-part series on effective messaging in support of student learning.

Do students benefit from prompts that offer study tips and reminders? Can these prompts lead to student success? These are questions the University of Minnesota has been exploring through piloting ECoach, a tailored messaging system developed by the University of Michigan that supports students by facilitating one-on-one communication and coaching. Leslie Schiff, Associate Dean in the Office of Undergraduate Education, was motivated to bring ECoach to the University of Minnesota to explore the tool’s potential to personalize the education experience in large gateway classes.

In Spring 2022, the University of Minnesota completed the 3rd semester of a pilot. To date, 3,141 students enrolled in intro-level Chemistry, Biology, Economics, and Philosophy classes have participated in the ECoach pilot.

How does ECoach work?

Instructors work with the ECoach support team to identify goals for their students. This includes:
  • creating a student intake survey on individual student’s goals, interests, motivation, and help-seeking behaviors;
  • determining which ECoach features to use in a course (to-do list, Exam Playbook, reflections, messages); and
  • drafting messaging to help students meet their unique course goals.
These messages are sent based on predetermined triggers (e.g. timing or performance on an exam) to provide students with the right message at the right time.

In addition, ECoach can tailor and customize support and coaching messages using:
  • Student ECoach survey data, which gathers information about student motivation, grade goals, and likelihood to use certain services
  • Canvas class data, which could include grades, attendance, or engagement
  • Demographic data, such as academic standing, GPA, or transfer status

How our approach evolved over 3 semesters

Semester 1: Getting started

The primary goal of the pilot’s first semester was to understand the platform’s tools and functionality in order to create a model for support. Each class focused on implementing a feature of ECoach that instructors felt aligned with their course goals. The messaging and features focused on providing timely messages to help students meet course goals, and provided academic study skills from resources such as Effective U.
The Exam Playbook really helped me map out how to study for my exams. It asks for specifics on what you're going to do when you’re going to do it and where. Being specific makes you more prone to do it.

Semester 2: Refining our approach

The second semester of the pilot focused on customizing the use of ECoach features. Instructors participating in the pilot utilized more of ECoach’s functionality to provide nuanced and course-specific messaging to their students. Notably, one course added the Reflection activity within ECoach that prompts students to apply course content to their daily life.
ECoach gives good advice on how to get through challenging classes. It helps you reflect on past assignments and change things for the future.

Semester 3: Enhanced content delivery and tailored messages

In spring 2022 we expanded the modalities of content delivery by adding video content. We continued to build course-specific tailoring. Our focus was on more customization for student messaging about general academic skills based on student goals and class performance. For example, if a student's current class grade is less than their goal grade, they might receive a 'To-do' list item that prompts them to "Consider mixing-up your study approach: Making a few changes can really help with study performance. Effective U’s Boost Your Study Habits is a good starting point for finding options to try!"
ECoach is reassuring, like having a support group or support system so I don’t feel alone.

Student feedback

In focus groups and surveys, students tell us:
  • ECoach makes large asynchronous courses feel more personal.
  • Students appreciate and benefit from timely, context-specific messages.
  • ECoach would be especially helpful early in a student’s academic career to develop basic study skills and habits.
  • Using the Reflection activity is impactful and helps students personalize course goals.
  • Communication is good; more communication is better. Students would like ECoach to provide even more communication throughout the term.
  • Students were given a Desirability Matrix composed of 44 words, positive and negative, and asked to choose words that describe the tone of ECoach messages. The top three words selected were helpful, encouraging, and informative.
  • Students find the messages repetitive but acknowledge that this isn’t a bad thing: hearing things multiple times positively impacts their learning.
  • Students are motivated to use ECoach because of extra credit points.

Conclusion

Students benefit from ECoach because it provides timely support and resources to students in a non-judgemental tone. They appreciate guidance in making decisions about time management and having access to resources that they may not be aware of. They also report that ECoach makes large courses feel more personal, and that targeted reflections can deepen learning.

The pilot is made possible with funding from the Provost's Office. This pilot is being facilitated collaboratively by:
  • Center for Educational Innovation
  • Academic Success Centers
  • Academic Technology Services - Office of Information Technology
  • Office of Undergraduate Education
  • Participating Faculty

Acknowledgments

This post was co-written with:

Labels: