Innovation Insights: Understanding the Effect of a MOOC

When examining the effectiveness of a massive open online course, or MOOC, standard metrics paint an incomplete picture. 

Test scores, completion rates, and grade-point averages are often used to measure the success of a traditional academic program. These markers don’t translate as easily to MOOCs, where enrollment and completion rates diverge. Instead, research shows students enrolled in open online courses are seeking something beyond top marks.

These learners intrigued Dr. Caren Stalburg, whose popular “Instructional Methods in Health Professions Education” MOOC has continued to attract interest since its launch in 2013, with nearly 25,000 enrollees and counting. Stalburg, an associate professor of obstetrics and gynecology and learning health sciences at the University of Michigan Medical School, wanted to find out how her learners were using the course, and whether it was impacting their professional and personal goals.

Teaching the Teachers

Stalburg’s career path was shaped early on by her curiosity around teaching and learning. After completing her residency at U-M, Stalburg was a new faculty member, adjusting to her role as an educator after years of being a student. She joined a committee that included experts from the Medical School and the Center for Research on Learning and Teaching, and swiftly realized there was more to instruction than just passing along knowledge. 

“I got involved in some of our curricular designs and realized that there is a science to education, and nobody teaches it to us,” Stalburg said.

She decided to learn the science. She earned a master’s degree in 2006 from what is now called the Marsal Family School of Education. Then in 2012, Stalburg again followed her curiosity, becoming one of the earliest faculty members to embrace the university’s new partnership with Coursera and launching a MOOC in collaboration with the center.

Stalburg’s course tapped into the lessons she learned early on about the importance of teaching the teachers, designing modules dedicated to informing and improving students’ instructional skills.

“I wanted the content to be broad enough so that it was applicable to any and all professionals who were teaching others about health,” she said.

Measuring the Impact

As the course’s enrollment grew, Stalburg was curious about its effectiveness. She decided to explore that question after seeing a report by the center’s senior research scientist Nate Cradit and former director Cait Hayward, which examined how students evaluated a MOOC’s quality based on the course’s affordances and attributes. 

Partnering with the center, Stalburg developed a survey specific to her course to evaluate if students’ needs were being met. In addition to gathering demographic data on the learners, the survey examined how they interacted with and benefitted from the content.

“We designed this study to look at understanding the participants’ goals for the course and how completing the course has impacted their professional goals,” Stalburg said. 

Surveys were sent to 278 learners, with 40 of those participants completing the survey. The respondents represented were diverse and educated, hailing from 18 different nations with 75% of them holding advanced degrees. Many were entering the middle of their careers, and they worked in a diverse range of fields, including medicine, nursing, dentistry, physical therapy, and more.

A vast majority of the learners said they took the course to increase knowledge and skill development, with some seeking professional development or meeting a requirement for their current job.

Interestingly, Stalburg and her team also discovered that the learners were using the material for their own instruction. Most of the participants downloaded the course material and used it to improve their teaching methods or create their own lessons. 

It was a satisfying finding for Stalburg, affirming her pursuit of sharing the science behind the instruction with fellow educators.

“I really believe that it’s about increasing human capacity, and meeting people where they are and in the needs that they identify,” she said. “And so, to me, this is like that old parable about teaching people to fish.” 

Local and Global Lessons

The impact of the Health Professions Education MOOC can be found here at the university, as well as on campuses across the globe.

In 2021, Stalburg was tapped to help design and launch the Health Infrastructures and Learning Systems Online Master of Science Degree, again in collaboration with the center. It is the first and only online degree program offered by the Medical School and Rackham Graduate School. 

Stalburg’s MOOC course was selected by universities and hospitals around the world to help train their medical communities. The University of Guyana partnered with Coursera and enrolled 86 students between 2018 and 2021. 

There was also a cohort of surgeons from the SSR Medical College in Mauritius who utilized the course, and praised the focus on teaching the instructors.

“For most of us MOOCs are a novelty, and the fact that the course content was so relevant to our professional activities made the experience so much more enriching,” wrote one faculty member.

More to Learn

Stalburg now wants to know more about those original respondents to the MOOC query. 

She plans to contact the participants for further interviews to gain a deeper understanding of the course’s impact on their career trajectories or personal lives, how their professional roles may have changed, and how they are applying the course content with their colleagues. 

Looking at the data she has collected so far, Stalburg believes the responses will reveal this MOOC’s success, and how it can’t be measured solely in numbers, but instead through its impact on learners’ lives. 

“I am hoping that people will sort of say I got better at teaching, I’ve become more recognized for my teaching, or my job opportunities have increased,” Stalburg said. “You know, just a flourish, a boost in whatever direction they’re looking to go.” 

References

Cradit, N., Hayward, C. (2023, October). What is a Successful MOOC? Lessons from Global Learner Narratives [Paper presentation]. IEEE Learning with MOOCS, Cambridge, MA, United States.