Co-authored with Mariam Tahir and Jeff Weber
This is the first in a series of posts that describe how the Academic Technology Services team is partnering with the Carlson School of Management in order to learn more about student perspectives on learning data and accelerate the adoption of learning analytics. We invite you to follow our journey and find out what happens next!
In early 2023, the Academic Technology Services team sponsored a capstone project with the Carlson School of Management for the undergraduate Information and Decision Sciences Department. The capstone project, called “Learning Analytics: The Student Perspective,” offered a unique opportunity to have students engage their peers on the topic of learning analytics and then submit recommendations based on their findings. This post describes the goals of the capstone project and the recommendations offered by the participating student teams. We will also describe some next steps that we are actively exploring.
What is a Capstone Project?
A capstone is a student-led research project where students use their cumulative academic experience to deeply engage with a topic and solve a real-world problem. For the Carlson Capstone, companies outside of the University and units within the University submit projects for students to research. If a project is approved, then students can choose to opt-in. Capstone projects are an opportunity for students to develop skills, gain real-world experience, and receive credit for their work. Their findings also provide a significant return to the sponsoring organizations and help these organizations meet their goals.The Process
The Academic Technology Services team (OIT-AT) submitted the capstone project “Learning Analytics: The Student Perspective” in the Fall of 2022. The project was approved and officially kicked off on January 19, 2023. Twelve students opted for the project; organized into three teams, they were tasked with delivering their final presentation on February 28, 2023. Jeff Weber, Lauren Marsh, and Mariam Tahir served as consultants for this project. We held weekly office hours to support the students; we answered their questions and helped them understand the learning data environment at the University of Minnesota. They also had the support of their course instructor, Kevin Kuhn.While we initially posed a very broad set of questions for students to explore, the fast pace of the capstone resulted in a sharper focus, captured by these questions:
- How much do students know about data captured in the context of teaching and learning?
- How comfortable are students with being measured and monitored?
- What data do students want instructors to have access to in order to support teaching and learning?
- What data do students NOT want instructors to have access to in order to support teaching and learning?
Student Findings
Across the three teams, over 40 students' perspectives were captured and represented in the results. Teams converged on the following findings:- Students are NOT aware of learning data being collected.
- Certain types of notifications and nudges are helpful, but students are resistant to other nudges (even when research suggests these are good practices for student success).
- Students are uncomfortable with instructors having access to personal learning activity data as it might cause bias.
Recommendations
Learning Data Statement
All three teams found that their peers are not aware of what learning data is being collected and how it is currently being used. Before we can introduce any learning analytics tool to students, they must be aware of how their data is being used and trust their instructors and the institution with their data.One recommendation that a team shared was to implement a public-facing statement guiding the use of data. This would build student trust and awareness of the data being tracked, and create a culture of transparency at the University, which might increase the usage of tools created for students.
Built-in Canvas Nudges & Notifications
In the surveys, students were asked questions about the usefulness of various nudges or notifications. The results showed that students consider as most useful nudges about upcoming assignment due dates or missing assignments. Students found less useful nudges notifying them of when to start an assignment or what resources might be helpful in their preparation for an upcoming exam. Note that student preferences might run contrary to research that suggests these kinds of nudges are actually good practices contributing to student success.Capstone teams recommended implementing nudges on assignment due dates and missing assignments in the LMS as a first step to introducing analytics to students. As described by one of the teams, “This will get the foot in the door to start implementing other kinds of nudges, such as when to begin studying or when to start an assignment.”
Canvas Learning Analytics Dashboard
One team recommended building a learning analytics dashboard directly in Canvas. The dashboard would display relevant Canvas data: here, students would be able to see course percentage grades and a summary of exam and assignment scores. As the team envisioned it, students could also interact with the dashboard: they could create course improvement goals and activate tracking capabilities.
The team also encouraged the use of Artificial Intelligence in the form of a Canvas Chatbot. The chatbot would provide insight into upcoming deadlines, overall progress, highest-priority tasks, and other items that would contribute to student success.
Instructor use of Learning Analytics
The teams acknowledged faculty are important stakeholders in the learning analytics journey and can gain valuable insights to support teaching. The caveat is that instructors must receive adequate training to reduce conscious and unconscious biases when dealing with analytics.What’s Next at the University of Minnesota?
As a result of this project, we have built connections with CSOM faculty who are interested in leveraging the Unizin Data Platform (UDP), a data source optimized for higher education because it combines data from Canvas, PeopleSoft and third-party tools such as Kaltura. CSOM faculty envision using the UDP to assist with learning analytics research and further development of an existing student-facing analytics tool. The tool is currently used in a few courses, with greater adoption planned across CSOM.To make the UDP more widely available to faculty and staff, we are working to finalize the UDP request access form and are exploring solutions to implement row-level security (RLS) to the data. The end goal of RLS is to allow users to access only the data they are authorized to view. This will ensure that we are complying with the University of Minnesota’s data security model while making this data resource more available.
This CSOM Capstone experience has been essential in understanding what learning analytics could look like at the University of Minnesota. We look forward to keeping the momentum going and hope that you will follow the journey!