The Right Tool for the Job: Metacognitive Processes and AI

A person is sitting on outdoor steps, wearing headphones and holding a smartphone. They have on a dark blue shirt, light blue jeans, and black slip-on shoes. Next to them is a black backpack with visible zipper compartments and orange string loops. The background shows green foliage with sunlight filtering through the leaves, and there is a blue section of a building partially visible behind the trees.

How this will help

Students can use generative AI to build their learning process

Providing structured guidance is key to responsible AI use

A current challenge facing university instructors is the use of generative AI tools by students to achieve the goals and objectives of a course. Students commonly interpret course goals and objectives as things they need to produce – the outputs of learning. But what is missing in this view is how they need to think, learn, and build their knowledge – the process of learning.

Students are commonly asked to produce written output as evidence of their knowledge and thinking, such as essays and assignments. Given that, it’s not surprising they would turn to generative AI tools, such as ChatGPT, that are programmed to generate and produce contextually appropriate written output. And because of the relative ease with which quality-looking output can be generated with these tools, it’s also not surprising that instructors are placing even greater attention on verifying that assignments are created by students and not produced by a generative AI tool.

With so much concern surrounding students’ use of generative AI tools in classroom learning settings, specifically with the outputs of learning, could AI promote other important learning processes that are not focused on outputs? Could the focus be shifted to strengthening students’ thinking processes instead?

Expert Thinking

At the center of an effective learning experience are problems to be solved. The goal is for students to come away from the learning experience having acquired a certain amount of content knowledge so they can engage with and solve problems that are presented. However, teachers and educational researchers know that content knowledge alone isn’t sufficient to solve problems.

When presented with a problem, experts not only use content knowledge, they also actively weigh the possible strategies and options they may want to use to tackle the problem – using metacognition in the problem-solving process. Metacognition is thinking about your own thought processes. Metacognitive strategies are ways to encourage and actively engage in the “whys” around thought processes.

For example, the problem-solving process may involve asking or reminding themselves:

Have I considered all the other options?
Am I jumping to conclusions too quickly?
Have I seen this before?
Remember what happened the last time.

And while the use and benefits of metacognitive knowledge may have been developed over the course of a professional career, educational researchers know that metacognitive strategies can not only be of great benefit for novices learning in the classroom, but that it can be explicitly taught alongside content.

Using AI to Support Metacognition for Learning

Let’s say you’ve assigned an analysis of a local art installation. Some students might find themselves struggling to get started even after attending lectures, passing quizzes, and attending small group discussions. By introducing metacognitive strategies, you could help get their thinking started.

Providing them questions to ask themselves and explicitly walking them through this thinking process is an example. By demonstrating this question strategy with students, they not only begin the process of generating output for their analysis, but they also strengthen their metacognitive processes for similar analysis in the future.

Your students are now feeling confident that they can tackle this analysis and will be able to implement these metacognitive strategies at 11 pm when they do their work, right? Of course, we know learning doesn’t work this way.

This is where generative AI tools can come in to support students’ use of metacognitive strategies and their writing process – at the time when they need it. This shifts the use of generative AI tools from “prompting for output” to “prompting for learning.”

For example, if the student is stuck and doesn’t know how to get started, they could be provided a set of prompts that ask the generative AI tool to provide questions designed to elicit written responses from the student.

Prompt: Act as a writing coach to help me write a critical analysis of a local art installation. Your goal is to model metacognitive strategies to help me unpack, organize, and compose my analysis.

The requirements for the analysis are as follows:

The rubric that will be used to determine the quality of the analysis is as follows: [provide rubric information].

As a writing coach, you will ask me questions that will help me gather and compose my observations and insights about the art installation I visited.

You will then evaluate my response against the requirements and rubric, and ask me to consider what other information I could provide to clarify my analysis. Continue to ask me questions until I have written at least 200 words.

Do not provide any examples for how to improve my analysis. Your job is to help guide my approach toward composing my analysis.

Do not ask all the questions at once. Instead, ask one question at a time, expect a response, evaluate the response, then ask another question.

In creating a writing coach with a generative AI tool, students are not only prompted to start writing, but the prompts themselves model a process of what a metacognitive strategy looks like for students, and what they can begin to use for future writing assignments.

If we want to help students fully realize the goals and objectives of our courses – to be able to apply what they’ve learned and create solutions in the real world – it’s important that we provide them with instruction and tools that emphasize strengthening their thinking skills and building their knowledge.

Resources

U-M Generative AI Tools

U-M Generative AI Use Cases

References

Dennis, J.L., Somerville, M.P. Supporting thinking about thinking: examining the metacognition theory-practice gap in higher education. High Educ 86, 99–117 (2023).

Krathwohl, D. R. (2002). A Revision of Bloom’s Taxonomy: An Overview. Theory Into Practice, 41(4), 212–218.

National Research Council. 2000. How People Learn: Brain, Mind, Experience, and School: Expanded Edition. Washington, DC: The National Academies Press.

McCormick, C. B., Dimmitt, C., & Sullivan, F. R. (2012). Metacognition, Learning, and Instruction. Handbook of Psychology, Second Edition, 7.

Merrill, M. D. (2013). First Principles of Instruction: Identifying and Designing Effective, Efficient, and Engaging Instruction. San Francisco, CA: Pfeiffer.

Pintrich, P. R. (2002). The Role of Metacognitive Knowledge in Learning, Teaching, and Assessing. Theory Into Practice, 41(4), 219–225.

Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A., Sarkar, A., Sellen, A., & Rintel, S. (2024). The Metacognitive Demands and Opportunities of Generative AI. CHI ‘24: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems

Finding ways to support every student is a fundamental challenge for instructors. When the learning occurs online, ensuring an equitable experience can seem daunting, especially when students are part of teams that meet outside a professor’s purview.

According to researcher Yiwen Lin, interventions aimed at boosting student engagement and experience are effective, and the strategic use of generative AI could ensure group learning benefits every team member.

Local Inspiration

As an undergraduate student at the University of Michigan, Lin got a glimpse into her future research while attending a talk on the student support tool ECoach. Developed by the Center for Academic Innovation, ECoach software provides students personalized feedback and tailored strategies for success.

Lin recalled attending the presentation given by ECoach founder Tim McKay, Arthur F. Thurnau Professor of Physics, Astronomy, Education. She was struck by McKay’s finding that while female physics students did not frequently speak in class, they did engage and contribute in other meaningful and important ways.

“What he found was that women like to back channel,” Lin said. “I thought, well women engage, but oftentimes they just engage differently, and it’s hard for an assessment that only looks at the frequency of participation.”

Lin, now a postdoctoral associate at the University of Pittsburgh, researches this deeper data with an eye on gender differences. She examines how psychological factors impact the persistence of online STEM learners, the quality of participation in team settings, and what interventions can be used to encourage more equity among students. Lin shared her research in an Innovation Insights talk titled “Charting Equity in Online Learning Teams: Opportunities and Challenges,” presented by the center.

Male vs. Female Motivation

Examining gender differences in STEM learning has traditionally evaluated how students’ psychological experiences impact outcomes. Lin’s research delves deeper into the learning process, revealing some surprising findings.

In one study, Lin and her team replicated a previous research project that looked at how a sense of belonging and STEM identity impacted female students’ desire to continue in STEM. But unlike the former study, Lin’s research used a pool of international online learners, many of them graduate students.

The results corroborated the importance of belonging and identity for women. However, when they examined the same connection for male learners, Lin’s team found that belonging and identity were also strong motivators for men. In fact, identity and belonging showed a slightly stronger link to STEM persistence for men compared to their female peers. This was the opposite of the previous findings.

Lin believes the pool of students (international and online) may have been a factor in the divergence from past research. Either way, interventions designed to increase female learners’ belonging and identity also clearly impacted male learners. Subsequent polling showed that a positive group dynamic impacted both male and female retention in STEM.

“We found that facilitating effective group dynamics can be potentially quite important for cultivating a more inclusive psychological experience,” Lin said.

Beyond Quantifying Participation

It can be challenging for instructors of online courses to incorporate those interventions, especially for small groups meeting outside the virtual classroom.

Lin outlined those challenges and the importance of diving deeper into the data in a study monitoring 88 small teams (three students per team) who were given a series of challenges to complete in a short period of time. Examining the gender differences in participation, Lin’s team confirmed that women spoke less in mixed-gender groups as well as male-majority teams, using fewer words and speaking less often compared to their peers.

The team then ran a language analysis on the transcripts of the students’ collaboration and found the female students actually provided a higher quality of participation than their male peers.

“Female students were better at responding to their teammates, building onto their contributions, and also being more cohesive with their own participation,” Lin said.

It affirmed her assertion that research can help look beyond the initial observations about frequency. Lin hopes that assessing the quality of contributions will be key to developing effective tools that encourage student participation in online courses and bring more equity to small groups.

AI for an Equitable Learning Experience

What those tools may look like is an exciting proposition to Lin, especially generative AI tools that can be applied to what she describes as the “in between,” the learning experience of students as they work through their course and team assignments.

“We sort of conceptualize that it is useful for AI to help us assess and model collaborative processes, rather than only collaborative outcomes,” Lin said.

Generative AI tools could provide personalized support for students, identifying learning patterns that may require intervention, like an intelligent tutoring system. Lin also sees potential in creating a similar generative AI program for teams, encouraging more equity in their collaboration and helping students from varied backgrounds and diverse perspectives interact in constructive and respectful ways. She referred to the center tool Tandem as an example of how well-designed support tools can reveal more about team dynamics and help instructors better support and guide students. Tandem coaches students working on team projects and allows instructors the chance to intervene when they see a group needs assistance.

Lin acknowledges that integrating generative AI with student support comes with challenges. That is why, Lin says, instructor input is key to ensuring tools are built using careful consideration of privacy and bias, and are extensively tested before launch. When done correctly, they could be powerful tools for building a more inclusive and equitable online learning environment.

“We wanted to think more deeply about how we can leverage AI as a tool for equity,” Lin said. “And this would perhaps be always a constant discussion in the community as we move forward with it.”

References

Lin, Y. & Nixon, N. (2024) STEM pathways in a global online course: Are male and female learners motivated the same?, L@S 2024: Proceedings of the Eleventh ACM Conference on Learning @ Scale, 243-249.

Lin, Y., Dowell, N., Godfrey, A., Choi, H., & Brooks, C. (2019). Modeling gender dynamics in intra and interpersonal interactions during online collaborative learning. LAK19: Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 431–435.

Nixon, N., Lin, Y., & Snow, L. (2024). Catalyzing equity in STEM teams: Harnessing generative AI for inclusion and diversity. Behavioral and Brain Sciences, 11(1), 85-92.

Lewis, N.A. , Sekaquaptewa, D. , & Meadows, L.A. (2019). Modeling gender counter-stereotypic group behavior: A brief video intervention reduces participation gender gaps on STEM teams. Social Psychology of Education, 22(3), 557–77.

Dowell, N., Lin, Y., Godfrey, A., & Brooks, C. (2019, June 25-29). Promoting inclusivity through time-dynamic discourse analysis in digitally-mediated collaborative learning. [Proceedings] In Artificial Intelligence in Education: 20th International Conference, AIED 2019, Chicago, IL, USA. Springer International Publishing AG, Part 1(20), 207–19.

Generative AI (GenAI) tools are becoming increasingly popular for a wide variety of uses, including in classrooms. Whether you’re generating images, building slides, or creating summaries of readings, it’s important to be thoughtful about the tools you’re using and the impact they can have on both your students and our world as a whole.

Bias

A GenAI tool is only as good as its training data; if that data contains content that is racist or sexist, we shouldn’t be surprised when the GenAI tool develops the same kind of bias. Bias can come in a variety of different types: stereotypical, gender, and political. All of these biases can lead to certain groups being inaccurately featured more or less in outputs.

Bloomberg tested the biases present in the Stable Diffusion text-to-image generator in 2023. When they prompted the model to create representations for jobs that were considered “high-paying” and “low-paying,” the images generated of high-paying jobs were typically of people with lighter skin tones. People with darker skin tones featured more prominently in images of low-paying jobs. Bloomberg found similar results when they looked at the gender of the people in the images. Stable Diffusion generated three images of men for one image of a woman. When women did appear in the generated images, they were typically in lower-paying and more traditional roles, like housekeeper. Prompts for jobs like “politician,” “lawyer,” “judge,” and “CEO” led to images that were almost entirely light-skinned men.

Harmful Content

Besides being biased, GenAI can produce content that is harmful in a variety of ways. GenAI can hallucinate content that is not based on actual data, and is instead fictitious or unrealistic. It can be used to produce artificial video or audio content impersonating a person’s likeness. When this kind of video and audio content is done with permission of the person, it’s commonly called “synthetic media.” When people create artificial video or audio content of someone without their permission, it’s referred to as a “deep-fake.” Deep-fakes are often used to harass, humiliate, and spread hate-speech. GenAI has made the creation of deep-fakes easy and cheap, and there have been several high-profile cases in the US and Europe of children and women being abused through their creation.

Policymaking efforts to combat the proliferation of and harm caused by deep fakes have become common both in the U.S. and abroad, with proposals often including disclosure requirements for the use of synthetic media, at least for certain activities. While educational uses of these technologies are unlikely to be restricted or banned, users should strongly consider disclosing the use of these technologies by default in the interest of transparency and in anticipation of any future requirements to do so that may apply. It may also be worthwhile to consider whether companies offering these products are well positioned to comply with this quickly evolving regulatory landscape as well as whether they are making reasonable efforts to help prevent the misuse of their products.

Data

The collection of data used to train GenAI models can raise a variety of privacy concerns, particularly around personal and proprietary data. Some personal data collection can be declined, although the methods of how to do so are often buried in lengthy terms of service that most users don’t read. Those terms of service also cover how the GenAI tool can use the data that you put into the tool via prompting, so you should be cognizant of the kind of information you’re feeding it.

Recently, the Cisco 2024 Data Privacy Benchmark Study revealed that most organizations are limiting the use of GenAI, with some banning it entirely, because of data privacy and security issues. This is likely because 48% of employees surveyed admitted to entering non-public company information into GenAI tools. There’s also a general lack of transparency around what kinds of data sets have been used to train GenAI tools. Although some explicitly state where their training data comes from, many are vague about what the training data was and how they accessed it.

Copyright

Right now, many believe that using content, like books, images, and videos, to train GenAI falls under fair use in the U.S., but there are currently multiple lawsuits challenging this notion. If companies are unable to leverage fair use to acquire training data, the effectiveness and availability of GenAI is likely to decrease dramatically. The cost of obtaining licenses for the incredible amount of data needed will likely drive all but the biggest companies out of the market.

The outputs created by GenAI can have their own copyright issues, depending on how much they pull from the training data. If the image generated by GenAI, for example, is substantially similar to an image in the training data, there could potentially be some liability for copyright infringement if or when the image is used. Many GenAI tools are attempting to avoid this by refusing to generate content that is similar to copyrighted material, but there are ways for creative prompters to get around these restrictions.

Although many GenAI tools claim to be trained on openly licensed content, studies show that when asked about licensing requirements, 70% of the tools didn’t specify what license requirements were for the generated work, and if they did, the tool often provided a more permissive license than what the original creator intended.

The use of GenAI brings up ethical issues around authorship that are often related to copyright but are separate. For example, when using information gathered from GenAI, there may be an ethical obligation to cite the original source to avoid claims of plagiarism. GenAI doesn’t typically provide citations, and when it does, those citations are frequently incorrect. There are also concerns about the displacement of human authors and artists by GenAI; this frequently comes up when GenAI is used to create works in the style of certain artists or authors.

Environmental Impact

GenAI has a huge environmental impact. Research has shown that training the early chatbots, such as GPT-3, produced as much greenhouse gas as a gasoline powered vehicle driving for 1 million miles. Generating one image using GenAI uses as much energy as fully charging your phone. ChatGPT alone consumes the same amount of energy as a small town every day. On top of that, the data centers needed to house the training data and infrastructure for these tools require large amounts of electricity and water to keep them from overheating. Right now, it’s nearly impossible to accurately evaluate or know the full extent of the environmental impacts of GenAI.

Equity

There are a variety of different types of equity concerns when it comes to GenAI. Most GenAI tools are trained on data from data rich languages and are less likely to include non-standard dialects or languages. There are also access and efficacy disparities. Not everyone will have access to GenAI tools, whether it’s because of the cost, a lack of internet access, or because there are accessibility issues with the tool. Underrepresented or underserved groups may find their experiences missing from the training data, which is only optimized for some groups, not all, limiting the efficacy of the outputs.

Finally, it’s important to remember that all of the legal and ethical issues discussed so far have a disproportionate effect on marginalized groups. For example, negative environmental effects tend to be felt the worst in more vulnerable communities. Considering the major impact GenAI has on the environment, how are we going to work with these groups to help ensure they’re not further harmed?

Conclusion

Overall, there are pretty significant legal and ethical issues we should consider before using GenAI tools. This doesn’t mean that we shouldn’t use GenAI tools; it means that we should be thoughtful about when, how, and why we’re using them. And we should know that the way we use them might change in the not so distant future. The current lawsuits will take years to work their way through the legal system, and depending on how they shake out, GenAI tools may have to go through some major changes when it comes to their training data.

Practical Tips

Here are five tips for navigating through these complex issues:

Investigate the reputation of the GenAI tool and the company that created it. Perform an online search for any potential legal or ethical issues. Add search terms like “complaint,” “violation,” or “lawsuit” with the company’s name, and be sure to read product reviews.
Check the terms of service. Review the terms of service and privacy policies before using GenAl. Caution should be taken before publishing materials created through GenAI.
Protect sensitive data. In addition to data shared for training purposes, it should be assumed, unless otherwise stated, that data shared when using GenAI tools will be accessible by the third party tool provider and affiliates. Data sharing must adhere to U-M policies.
Consider the ethics/limitations. Continue to remember, and remind your students, that GenAI tools are often biased, as the technology is designed to output common results based on its learning model. GenAI can also “hallucinate,” so specific claims should always be verified before sharing.
Consult resources and ask for help. We are still swimming in uncharted waters. Utilize resources available here at U-M, including training and workshops on GenAI that are hosted across U-M. There is also a new GenAI as a Learning Design Partner series led by U-M instructors that is freely available via Coursera.

Jenni Patterson is a Design Manager Senior at the Center for Academic Innovation. In this interview, Jenni speaks with us about her role at CAI, the GenAI short courses, and how to get started with a short course.

Tell us a little bit about your role at the Center for Academic Innovation

I’m a Design Manager Senior here at CAI, and in this role I oversee the project management of various online learning initiatives in our non-credit and MOOC space. My job is to keep each project team – the faculty member, Learning Experience Designer, and Media Designer – aligned and on track as we move toward the project’s launch date. I initially focused on developing Teach-Outs, but in Fall 2023 I shifted to focusing on the project management of our new short courses.

You mention the short courses – tell us a little bit about what a MOOC is, and how the short courses are different from a regular MOOC?

A MOOC, or Massive Open Online Course, is a free, online course that is open to unlimited learners from all around the world. Here at CAI, our MOOCs typically take a learner 3-8 weeks to complete, expecting around 4-6 hours of effort each week. Short courses, as you can imagine by their name, are much briefer learning experiences! We expect a short course to take a learner just 1-5 total hours to complete. The goal of a short course is to provide learners the opportunity to dive into a very focused, yet still high quality, course. Where a traditional MOOC may have several learning objectives, a short course may have only two or three, or in some cases just one. This focus lends to a shorter learning experience.

Why are you so excited about these short courses? Why should faculty want to consider working on a short course?

I’m excited about these short courses because they are a great opportunity for faculty to share their expertise with our community of learners in a short, digestible way. Whether faculty have created an online course before or are new to the process, I hope they’ll consider working on a short course because it’s a great opportunity to develop an online learning experience, start to finish, in just a few months.

The current round of short courses are all focused on Generative AI. Why do you think GenAI lends itself to this format? Who will be most interested in these courses?

I think GenAI lends itself really well to the short course format. During the ChatGPT and Generative AI Teach-Outs, we heard from our community of learners that they wanted to learn more about GenAI as it related to their specific industries and personal contexts. The GenAI short courses allow U-M to provide this specialized content to our learners.

We think learners will be interested in these courses because they offer GenAI content beyond the many foundational GenAI courses we’re seeing. We’ll be offering GenAI short courses from a variety of perspectives – from programming to business and law to social work – allowing learners from different backgrounds to learn about GenAI in a context specific to them.

If a faculty member was interested in creating a short course, what would they need to do?

Faculty who are interested in creating a short course (or any type of project with CAI!) should contact our Partnership Development team at [email protected]. When you connect with them, they’ll want to know a bit about your idea and who you believe may be most interested in taking the course. From there, they’ll work with you to help imagine the course and to discover which product type (short course, regular MOOC, etc.) is best suited to meet the goals for the learning experience.

What should people expect from working with the Center for Academic Innovation?

The staff here at CAI is eager to help bring your project idea to life! On a typical design team, you’ll be working with a Design Manager (project manager), Learning Experience Designer (who supports course development and design), and Media Designer (who manages filming, production, and media assets). We work together, both synchronously and asynchronously, meeting weekly to review course development progress while also completing individual tasks between meetings. We are committed to working collaboratively with our faculty partners to create exemplary online learning experiences!

COVID-19 caught everyone off guard in 2020. Suddenly, all classes had to be held online and instructors and students had to react quickly with minimal help. With time to reflect on these experiences, faculty ask themselves what methods are available to keep students engaged and motivated in an online or virtual environment.

At the Center for Academic Innovation, gameful pedagogy is one approach to increasing student engagement. This method of course design takes inspiration from how good games function and applies that to the design of learning environments.

One key goal of gameful pedagogy, as one might guess, is leveraging student motivation. To achieve that, course designers draw on elements of Self-Determination Theory, or SDT for short. This theory centers the power of intrinsic motivation as a driver of behavior. It sits on three primary pillars: autonomy (the power of choice a learner can have in their learning experience), competency (a feeling of accomplishment derived from completing a challenge), and belongingness (a feeling of being included and heard by the environment one is in or the people around them) (Deci & Ryan, 2000).

Yet, gameful pedagogy isn’t just about SDT. Practitioners also believe in an additive point-based grading system instead of traditional grading. In traditional deductive percentage-paced grading, learners start at 100% and have their points deducted as they learn, which does not align with what learning is about.

In a gameful course, learners are treated as novices when they first start a learning journey, so they start from zero and then work their way up to their goals. It also provides learners the freedom to fail. From a gameful point of view, it is unfair to expect learners to be “perfect” in learning environments because mistakes are common in learning, and they are great growth opportunities. Therefore, in gameful, learning environments that leave space for learners to explore and offer chances to make up for mistakes are preferred. It is important, however, to acknowledge that this freedom does not mean creating an out-of-control environment. Educators can still apply limitations by assigning different point values, requiring the completion of certain tasks to unlock others, etc. to ensure that students are working toward the learning goals. All of these approaches and more boil down to gameful pedagogy, and this course design method has been used in a wide range of classes, from higher education down to K-12. However, most use cases occurred in person before the 2020 COVID outbreak. Does gameful also work in online environments?

That turns out to be a great question for Pete Bodary, clinical associate professor of applied exercise science and movement science in the School of Kinesiology. He has taught gameful courses for several years, including MOVESCI 241. This course teaches body mass regulation assessments, principles, and strategies. It is constructed with an additive point-based grading scheme, all-optional assignments (a student has the autonomy to complete any combination of assignments to get to their desired grade/goal), a strong supportive network, and real-world relevant topics (diabetes, disordered eating, weight control, supplements and safety, etc.).

To maintain all assignments as optional while ensuring that students are on track to the learning objectives, Bodary assigns significantly more points to certain assignments to encourage completion. Some assignments include personal dietary intake and physical activity tracking, case studies, participation and reflections on dietary and physical challenges, and more.

In Winter 2023, he decided to give students more freedom to engage with the class lectures on top of the existing setup. Students could choose from three distinct sections: the in-person section, the synchronous virtual section, or the asynchronous virtual section. In the in-person section, students were required to attend lectures in person. In the synchronous virtual section, students could participate in lectures online while being live-streamed. The asynchronous virtual section allowed students the freedom to watch lecture recordings at their convenience without the obligation to attend lectures in real-time.

Did students in different sections perform differently in this course? The short answer is no, not significantly.

“Those who are remote do not have the ease of popping out a question, [meaning the ability to raise their hand and spontaneously ask questions], so that is one difference to consider. However, we maintain a pretty active [asynchronous] Q/A space. I don’t believe that they ‘performed’ differently,” Bodary said.

Students engage with the course content differently, but they are all motivated and learning in their own way.

In fact, to find out students’ motivations in this course, Bodary deployed a U-M Maizey project. U-M Maizey is a generative AI customization tool that allows faculty, staff and students to build their a U-M GPT chatbot trained on a custom dataset. Bodary set up Maizey in the Fall 2023 term for the same course with a similar structure and prompted Maizey: What is the primary motivation of students?

By evaluating students’ activity data, Maizey summarized that students are primarily motivated by finding course materials relatable and beneficial to improving their personal and loved ones’ health and well-being, connecting knowledge and issues they garnered in their daily lives to class content, and implementing course content in real-world problems.

Looking at this example, three key characteristics emerge: controlled freedom for students to choose how to engage with the course, opportunities for students to make personal connections with course content, and possibilities for students to apply course content in real-world situations.

Tying these characteristics back to gameful pedagogy, there is alignment between them and the three components of SDT – autonomy, belongingness, and competency. Furthermore, the additive grading system and all-optional assignment design support student exploration and agency to choose assignments and coursework. The course format, whether in-person or online, didn’t impact students’ motivation. Instead, the fact that students can choose their own way to participate in the class may motivate them even more.

What’s important here isn’t modality (online, in-person, or asynchronously) but rather the content and design of the course. The success of MOVESCI 241 hinges on a carefully designed course where students can successfully meet the learning goals regardless of how they engage. The design of MOVESCI 241 is gameful, but not all gameful courses are designed this way. If you want to use gameful pedagogy to increase engagement in your course, you can start with these steps. You can also check out GradeCraft, a learning management system (LMS) built at the center to support gameful courses. Some key features of GradeCraft that make it a perfect companion for gameful courses are the additive grading system, mechanisms for tracking tangible progress (points planner, levels, unlocks, and badges), and functions for flexibility (highly tailorable for both instructors and students). Finally, if you want to learn more about gameful pedagogy or GradeCraft, please email us at [email protected], and staff would be happy to set up a conversation with you.

References

Deci, E. L., & Ryan, R. M. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227-268.

Educators can use generative AI to transform dense, technical material into clear, easily understandable content. This improves students’ comprehension and makes the learning experience more inclusive to a wider audience. While students are growing in their knowledge of complex academic topics, sometimes academic terminology can be a barrier. Particularly early in the course, students may not yet be familiar with the jargon and language of your subject matter. In addition, you may have learners in your course with a wide range of educational and cultural backgrounds. Some of your students may be from countries outside of the United States, and English may not be their first language. By demystifying complex concepts, jargon, and metaphors with generative AI, educators are empowered to create more equitable and effective learning environments for our diverse array of learners.

For example, you can use the following example prompt to get started:

“Could you please rewrite the following text to an 8-10th-grade reading level (using Flesch-Kincaid Grade Level) and clarify any complex jargon: “[Your text]” Thank you!”

In this prompt, we are asking ChatGPT to rewrite text to an 8-10th-grade reading level on the Flesch-Kincaid Grade Scale. This is the reading level recommended for a general adult lay audience. Feel free to adjust this to fit your target audience.

Example: An Online Course on Neuroscience

Drafting

Now imagine that you are a renowned neuroscientist and a highly regarded faculty member at Michigan Medicine. You are interested in developing an online course that will bring neuroscience concepts to a lay audience. You are excited to get started, but as you begin to develop content, you quickly realize that your typical content is aimed at seasoned medical students and filled with jargon that may be daunting to those without prior knowledge. You realize that generative AI may be able to assist you in breaking down concepts into simpler terms.

You fill in the example prompt with some of the text from one of your old in-person presentations with key concepts that you would like to include in this online course:

“Could you please rewrite the following text to an 8-10th-grade reading level (using Flesch-Kincaid Grade Level) and clarify any complex jargon:

In our introductory exploration of neuroscience, we delve into the intricate workings of the central nervous system (CNS), focusing on the complex interplay between neurons and glial cells. We examine the fundamental principles of neuronal communication, emphasizing the role of neurotransmitters in synaptic transmission and the significance of action potentials in neural signaling. A detailed analysis of the brain’s functional anatomy is conducted, where we dissect the various lobes of the cerebral cortex, the limbic system, and the intricate circuitry of the basal ganglia, highlighting their contributions to cognitive and motor functions. Additionally, we explore the mechanisms underlying neuroplasticity, emphasizing the synaptic and non-synaptic changes that facilitate learning and memory. The course also delves into the pathophysiology of neurological disorders, exploring the etiology and progression of conditions like Alzheimer’s disease, Parkinson’s disease, and multiple sclerosis, and discusses the latest advancements in neuropharmacology and neuroimaging techniques.

Thank you!”

In response to your input, ChatGPT gives you the following output:

“In our first look at neuroscience, we study how the central nervous system (CNS) works. This includes understanding how neurons (brain cells) and glial cells (supporting cells) interact. We learn about how neurons talk to each other, focusing on neurotransmitters, which are chemicals that help send messages in the brain, and action potentials, which are electrical signals in neurons.

We also look closely at the brain’s structure. This includes studying the different parts of the brain, like the lobes of the cerebral cortex, the limbic system (which deals with emotions), and the basal ganglia (important for movement), and how they help us think and move.

The course also covers neuroplasticity, which is how the brain changes to help us learn and remember things. We look at both synaptic changes (changes at the connections between neurons) and non-synaptic changes (changes within the neurons themselves).

We also learn about neurological disorders like Alzheimer’s disease, Parkinson’s disease, and multiple sclerosis. We study what causes these diseases and how they progress. Finally, we discuss new developments in treating brain disorders and ways to look at the brain, like neuropharmacology (study of how drugs affect the brain) and neuroimaging (brain scanning techniques).”

In this example, ChatGPT keeps all of the main concepts intact while using simpler language, providing definitions of terminology used (rather than removing it entirely), and breaking the large paragraph into more digestible, smaller paragraphs or chunks.

Refining

As a content expert, it is important to read through the output and ensure that all key concepts remain intact. It is also up to you to determine whether the revisions are sufficient and appropriate for your audience. You may choose to ask for stylistic revisions as well. For example, ChatGPT wrote the text as though the course is currently happening. However, you plan on delivering this information at the beginning of the course to talk about what the learner will learn. This is your preference.

You can ask ChatGPT to revise with the following:

“The course hasn’t happened yet. Please rewrite this to account for that. For example, using the phrase “we will” throughout.”

ChatGPT will then go through and make the requested revisions to the text using the appropriate tense that you indicated in your input:

“In our upcoming exploration of neuroscience, we will delve into how the central nervous system (CNS) functions. This will involve understanding the interaction between neurons (brain cells) and glial cells (support cells). We will learn about neuronal communication, focusing on neurotransmitters, which are chemicals that help send messages in the brain, and action potentials, the electrical signals in neurons.

We will also closely examine the brain’s structure, studying different parts such as the lobes of the cerebral cortex, the limbic system (which deals with emotions), and the basal ganglia (important for movement), and understanding how they influence our thinking and movement.

The course will cover neuroplasticity, which is how the brain changes to facilitate learning and memory. We will observe both synaptic changes (alterations at the connections between neurons) and non-synaptic changes (alterations within the neurons themselves).

Additionally, we will explore neurological disorders, including Alzheimer’s disease, Parkinson’s disease, and multiple sclerosis. We will investigate the causes of these diseases and their progression. Finally, we will discuss the latest developments in treating brain disorders and methods for examining the brain, such as neuropharmacology (the study of how drugs affect the brain) and neuroimaging (brain scanning techniques).”

Continue to refine as needed. Consider feeding into the chat examples of your tone of voice so that the content is not only accessible for learners but also contains a human element. In addition, you can increase your expectation of language understanding as your students grow in their knowledge and your expectations of understanding increase.

Generative AI can be a valuable asset to instructors looking for assistance with creating various aspects of course design. For example, generative AI, such as ChatGPT, can be a valuable tool for educators in drafting learning objectives. Using GenAI in any setting is usually a process of drafting and then refining prompts until the desired result is achieved. In this article, we will outline some ways to generate and refine learning objectives for a course.

Learning objectives are concise statements that articulate what students are expected to learn or achieve in a course. They play a crucial role in guiding both teaching strategies and assessment methods, ensuring that educational experiences are focused and effective. Clear and well-defined learning objectives are essential for aligning educational activities with desired learning outcomes. By analyzing a vast array of educational content and pedagogical methods in its training data, AI can offer a wide range of learning objective recommendations, which educators can then build off of, using their knowledge as experts in the field.

Using your preferred GenAI tool, here is an example prompt that you can use to get started:

Please create learning objectives for a course on [topic], focusing on [big idea 1], [big idea 2], and [big idea 3]. The learning objectives are concise one word sentences beginning with an action verb. Please refer to Bloom’s Taxonomy if you get stuck. Thank you!

This example prompt can be modified to fit your needs. For example, you may choose to add more ideas and give additional context about the course. The more detail and context you provide in your input, the better the AI output will be. So please feel free to add in outlines, syllabi, or any other materials that may help your GenAI assistant better understand your vision.

Example: An Online Course on the Cold War

Drafting Objectives

Now that we have our example prompt, let’s see an example of it in action. Imagine you are an instructor for an introductory online course on the Cold War. You plan to use ChatGPT to generate some ideas on potential learning objectives to get you started and guide your curriculum creation. You already have some general ideas on what you want to cover: causes, major events, and overall impact. You fill in the prompt as so:

Please create learning objectives for an online introductory course on the Cold War, focusing on its causes, major events, and impact on international relations. The learning objectives are concise one word sentences beginning with an action verb. Please refer to Bloom’s Taxonomy if you get stuck. Thank you!

You press enter and ChatGPT provides you with the following learning objectives:

“Identify the primary causes of the Cold War.

Describe major events and turning points during the Cold War.

Explain the ideologies of the United States and the Soviet Union during the Cold War.

Analyze the impact of the Cold War on international relations.

Compare and contrast different perspectives on key Cold War incidents.

Evaluate the effectiveness of diplomatic strategies used during the Cold War.

Summarize the global consequences of the Cold War in the post-war era.

Discuss the role of nuclear weapons in Cold War policies and strategies.

Interpret historical documents and sources related to the Cold War.

Assess the influence of the Cold War on contemporary international politics.”

Refining

It is now up to you as the expert to determine which learning objectives are the most relevant and how you should go about revising them. For example, you may look at the list and notice that there are no learning objectives that ask the learners to create something with the knowledge they’ve acquired throughout the course (e.g., a final project). You return to ChatGPT and ask the following:

Please craft a learning objective that has learners create something relevant to the topic at hand.

In response, ChatGPT provides you with the following:

Design a multimedia presentation that illustrates the timeline of the Cold War, integrating key events, major political figures, and their impacts on international relations.

If you disagree with this suggestion, you can reply with “More?” to get additional ideas. ChatGPT will then provide you with a longer list:

“Develop a documentary script focusing on a specific aspect of the Cold War, such as the Cuban Missile Crisis or the Space Race, incorporating primary sources and historical analysis.

Construct a series of infographics that visually represent the economic, political, and social changes in different countries during the Cold War era.

Compose a series of fictional letters or diary entries from the perspective of individuals living in different countries during key moments of the Cold War, reflecting their experiences and viewpoints.

Create a podcast series that discusses the influence of the Cold War on modern international relations, including interviews with experts or simulations of discussions between historical figures.

Produce an interactive map or timeline that allows users to explore the progression of the Cold War, highlighting significant events, treaties, and shifts in alliances.

Formulate a role-playing game where players assume the roles of world leaders during the Cold War, making decisions that could change the course of history.

Design a virtual museum exhibit showcasing artifacts, propaganda, and media from the Cold War, with descriptive and analytical captions explaining their significance.”

You can repeat this process as often as you’d like – adjusting the prompt and adding additional context (e.g., outlines, key ideas, information about your teaching style) to get better responses. When formulating responses for you, ChatGPT looks at the entire chat log so it is recommended that you continue to add to the same chat for best results.

In our next article, we’ll explore how to use Generative AI to improve accessible language in your course.

Education is undergoing a significant transformation as generative artificial intelligence continues to develop at a rapid pace. It is now easier than ever for educators to experiment with generative AI in their practice and see for themselves how generative AI can be leveraged during the course development process to brainstorm, synthesize, and draft everything from communications to students to learning objectives.

Generative AI: The Basics

Before experimenting with Generative AI (GenAI), it is helpful to have some high level foundational knowledge of how GenAI works. Essentially, GenAI functions using advanced machine learning algorithms, specifically neural networks, which emulate human brain processing. These networks are trained with large datasets, enabling them to learn language patterns, nuances, and structures. As a result, GenAI can produce contextually relevant and coherent content, a capability exemplified in tools like ChatGPT.

To better understand how GenAI tools like ChatGPT work, let’s look at a breakdown of the acronym “GPT”:

GPT stands for “Generative Pre-trained Transformer.” It is a type of artificial intelligence model designed for natural language processing tasks. “Generative” refers to its ability to generate text based on a combination of the data it was trained on and your inputs. It can compose sentences, answer questions, and create coherent and contextually relevant paragraphs.

The term “Pre-trained” indicates that the model has undergone extensive training on a vast dataset of text before it is fine-tuned for specific tasks. This pre-training enables the model to understand and generate human-like text.

Finally, “Transformer” is the name of the underlying architecture used by GPT. Transformers are a type of neural network architecture that has proven especially effective for tasks involving understanding and generating human language due to their ability to handle sequences of data, such as sentences, and their capacity for parallel processing, which speeds up the learning process.

The GPT series, developed by OpenAI, has seen several iterations, with each new version showing significant improvements in language understanding and generation capabilities. Many of these improvements are due to the model continuously training on user inputs. OpenAI has made it transparent that your data is being used to improve model performance and you can choose to opt out by following the steps that will be outlined in the upcoming articles on how to use GenAI tools for course design, learning objectives and more.

Does It Matter Which GenAI Tool I Use?

Not really. Individuals may find preferences for one tool or another based on response speed or comfort with the interface. You may wish to use a tool that can opt out of using personal data for training purposes. Most of the GenAI tools are generally similar.

Next Steps and Considerations

In educational contexts, the incorporation of GenAI tools, such as ChatGPT, will potentially reshape our approach to content creation and improve efficiency for educators who often find themselves pressed for time. However, it is important to note the importance of acknowledging the technology’s limitations, such as potential biases, outdated information due to insufficient training data, and incorrect information – often referred to as “hallucinations.” It is vital that you always fact-check and revise GenAI outputs to maintain the integrity and high quality of your content.

In conclusion, by leveraging GenAI tools like ChatGPT, educators can navigate course design with greater ease and efficiency. From drafting learning objectives and engaging course titles to simplifying complex academic language and brainstorming assessments, GenAI has the potential to be an invaluable asset to your design work. However, it is critical to remember that these tools come with limitations, including potential biases and inaccuracies. By combining the strengths of GenAI with the expertise and critical oversight of educators, we can efficiently create new experiences for our learners.

How this will help

Examine the pros and cons surrounding ChatGPT

Navigate the concerns of LLM as well as the potential benefits for learners and instructors

It is safe to say that by now, you have seen many articles/posts, opinions, and stories about ChatGPT—and the larger AI-Language Learning Models (LLMs)—in relation to higher education and teaching/learning in particular. These writings included several perspectives ranging from raising concerns to celebrating new opportunities and a mix of the former and the latter. Also, these writings continue to evolve and grow rapidly in number as new AI-powered LLMs continue to emerge and evolve (e.g., Google’s new AI LLMs: Bard).

The intent of this piece is not to add another article sharing tips or concerns about ChatGPT. That being said, this article (1) summarizes the major concerns about ChatGPT and (2) the ideas about its positive implications based on what it is published to date.

Concerns about ChatGPT

Faculty, scholars, and higher education leaders have raised several concerns about ChatGPT. These concerns stem from possible ways it can be used.

Using ChatGPT to cheat by asking it to write essays/answer open-ended questions in exams/discussion forums and homework assignments (December 19th, 2022 NPR Story) (December 6th, 2022 Atlantic Story) (January 16 New York Times Story).
Using ChatGPT to author scholarly works which conflict with the ethical standards of scientific inquiry. Several high-impact/profile journals have already formulated principles to guide authors on how to use LLMs AI tools and why it is not allowed to credit such tool as an author—any attribution of authorship carries with it accountability for the scholarly work, and no AI tool can take such responsibility (January 24th, 2023 Nature Editorial).
ChatGPT can threaten the privacy of students/faculty (and any other user). Its privacy policy states that data can be shared with third-party vendors, law enforcement, affiliates, and other users. Also, while one can delete their ChatGPT account, the prompts they entered into ChatGPT cannot be deleted. This setup threatens sensitive or controversial topics as this data cannot be removed (January 2023 Publication by Dr. Torrey Trust).
ChatGPT is not always trustworthy, as it can fabricate quotes and references. In an experiment conducted by Dr. Daniel Hickey at Indiana University Bloomington, Instructional Systems Technology department, “ChatGPT was able to write a marginally acceptable literature review paper, but fabricated some quotes and references. With more work such as including paper abstracts in the prompts, GPT is scarily good at referencing research literature, perhaps as well as a first-year graduate student.” (January 6th, 2023, Article by Dr. Daniel Hickey)

Excitement about ChatGPT

At the other end of the spectrum, there have been several ideas that express interest and excitement about ChatGPT in higher education. These ideas stem from how they can be used ethically and in a controlled manner.

Using ChatGPT to speed up the writing of drafts for several outlets (reports, abstracts, emails, conference proposals, press releases, recommendation letters, etc.) ChatGPT can produce elaborated writing that must be edited to remove any possible inconsistencies or inaccuracies (December 7th, 2022 Social Science Space story)
Using ChatGPT in the process of brainstorming ideas for curriculum design, lesson planning, and learning activities. The tool can provide some novel ideas or remind educators of some instructional techniques and strategies that they had heard about in the past (January 23rd, 2023, Article by Dr. David Wiley).
Using ChatGPT to provide students tutoring/scaffolds. The tool can act like a virtual tutor who does not simply give the answer to the student but rather scaffold them to reach the correct answers by themselves. (Sal Khan, founder/CEO of Khan Academy, Spring 2023 TED Talk)
Teaching with ChatGPT to train students on using AI tools and models, provide opportunities to exercise critical thinking skills, and improve their technological literacy (January 12th New York Times story).

Concluding Thoughts

There are major concerns about ChatGPT and the larger AI-powered Language Learning Models (LLMs) phenomenon. These concerns are legitimate and are opposed by notable ideas about the positive implications of AI-powered LLMs in higher education classrooms. As we aspire to make evidence-based educational and learning design decisions, one should carefully review the research that has been done on AI in relation to higher education up to this point and engage with the gaps as opportunities to expand knowledge and find new opportunities and risks.

Our University’s newly formed advisory committee on the applications of generative AI is a good example of how higher education institutions ought to recommend the use, evaluation, and development of emergent AI tools and services. Additionally, discussions about generative AI and its implications on education happening in public venues are necessary to strengthen the public-facing mission of the University, where input from educators, students, and members of the community is welcome.