Custom Chatbots Successfully Save Time, Improve Submissions
U-M Maizey tool helps Kinesiology students refine project proposals – with impressive results
How this will help
Giselle Aronoff is a graduate of the U-M School of Kinesiology’s Movement Science Program and research assistant to Dr. Peter Bodary, associate dean of undergraduate education and clinical associate professor in the School of Kinesiology.
In the elective course MVS/AES 241, “Exercise, Nutrition, and Weight Control,” in the University of Michigan’s School of Kinesiology, students have the unique opportunity to pursue an eight-week dietary and/or physical activity challenge. Upon creating their own challenge, they submit their proposal to the instructional team for approval. But providing individualized feedback for every unapproved proposal has been very time-consuming for our instructional team, especially in a course with over 50 assignments.
In an attempt to streamline the approval process, our team created chatbots using U-M Maizey to act as “challenge screening tools” prior to the initial submission. The Maizey chatbots helped students refine their challenge proposals from the start, prompting them to make revisions even before submitting.
The results showed benefits for both students and instructors. Compared to past semesters, a higher percentage of proposals were approved without needing additional refinements. This freed up significant time for the instructional team, which we redirected toward giving feedback in later, more meaningful assignment stages. Most importantly, the majority of students felt that the chatbot enhanced the quality of their work.
Challenge Accepted
Students start by proposing dietary and/or physical activity challenges and variables for instructor approval. Upon confirmation that their proposal meets criteria for safe and realistic activities, they follow the challenges for eight weeks.
However, if a challenge is not approved in their first submission, students must post a revised version in Piazza, a Canvas-integrated Q&A platform, for instructor review. With over 200 students participating, the Piazza comment volume quickly escalates. Hence, in past semesters, this initial approval stage has been time-consuming for instructors, as they must review the many student revisions. It has also been tedious for the students, as they must wait for instructor feedback (at times through multiple revision cycles) before progressing with their challenges.
So, we created a Maizey chatbot to act as a challenge and variable approval assistant. Our system prompt outlined criteria for unacceptable challenges, provided a description of the chatbot’s intended role and demeanor, and explicitly prohibited generating challenges for the student. We uploaded documents to guide this, including a list of unacceptable challenges/variables, assignment instructions, and a few examples of acceptable challenges/variables.
One of our main concerns was that the chatbot may produce work for the student, detracting from their autonomous thinking. To safeguard against this, we clearly delineated in our system prompt that the chatbot was to act as a brainstorming assistant only and should never formulate challenges, variables, or a topic for the student. Instead, the chatbot’s purpose was to support the student in curating their original ideas to align with assignment guidelines.
If a challenge or variable does not meet assignment criteria, the chatbot explains the concern to the student and guides them toward appropriate improvements. If the challenge and associated variables do meet all assignment criteria, Maizey informs the student that their proposal is approved.
As of the fall 2024 semester, we introduced the Maizey chatbot as part of the initial challenge approval assignment. Students were instructed to screen their challenges and variables in Maizey before submitting their proposal to the instructional team in Canvas. To verify their use of the Maizey tool, we required students to include a screenshot of Maizey’s approval message in their Canvas submission.
Time Well Spent
That semester, we had 212 dietary and 225 physical activity challenge submissions. We found that compared to past semesters, a higher proportion of initial Canvas submissions were approved when implementing the Maizey screening tool. For the dietary challenge, nearly 90% of initial submissions were approved without need for additional revisions, compared to an average of only 66% approved in past semesters. For the physical activity challenge, 85% of initial submissions were approved, compared to an average of 46% in past semesters. Additionally, we observed significant reductions in Piazza comment volume, lessening the time commitment for instructors and enabling them to reallocate their attention to more meaningful teaching tasks.
While instructor time saving was one of our goals with the Maizey screening tool, our other objective was to streamline the approval process for students, offering prompt and useful feedback on their ideas. In a survey to participants, 81% reported having to revise their initial proposal multiple times to gain Maizey approval – revisions that previously would have occurred in Piazza, incurring additional time costs for the instructional team and delaying feedback to students. Approximately 90% of students felt that the Maizey tool helped to improve the quality of their initial submission. Furthermore, approximately 76% of students believed the Maizey tool to be beneficial in the challenge and variable brainstorming process.
Support From the Start
Maizey chatbots can be implemented in early stages of scaffolded assignments to provide students with immediate feedback on their work, helping them to better understand assignment expectations and setting them up for success in subsequent assignment stages
In large courses, using Maizey chatbots as a screening tool for papers, presentations, or project topic proposals enhances initial submission quality, reducing grading time and enabling instructors to reallocate time to evaluating higher-stakes, later-stage submissions.
Maizey chatbots do not replace instructor feedback, but can provide additional guidance to students throughout the brainstorming process, at a stage when they are typically on their own. Importantly, our chatbots were intended to support rather than detract from autonomous thinking, encouraging students to refine and strengthen their ideas.
Next Steps
In the 2024 – 2025 academic year, we used an early version of Maizey that lacked conversational capacity. This fall, we are piloting new, conversational Challenge Maizeys designed to actually guide students through the brainstorming process instead of only acting as a screening tool. The conversational chatbots will leverage motivational interviewing and reflective listening techniques to probe deeper into each student’s health and wellness habits, goals, and motivations.
Our hope is that the chatbots will make the assignment more personally meaningful to students and perhaps even encourage positive behavioral changes lasting beyond the eight-week challenges.
U-M Maizey
For any instructors interested in incorporating chatbots into their assignments, U-M Maizey is a very user-friendly tool thanks to helpful guides like this one from U-M Information and Technology Services. At a high level, creating your own Maizey involves three steps:
- Write a system prompt specifying how the chatbot will behave – in this case we instructed the chatbot to act as a challenge-and-variable approval assistant.
- Upload source documents (assignment instructions, example submissions, supplementary materials).
- Thoroughly test your chatbot before releasing it to students.
Important Tips:
- Anticipate potential deviations from assignment criteria. Provide specific responses to these violations in your system prompt. This will ensure that the chatbot appropriately redirects misguided students.
- Use clear, concise instructions and absolute language (where applicable).
- Have multiple team members testing the chatbots. Especially where primary instructor time may be limited, this is a suitable task to delegate to GSIs, TAs, and instructional aides.
Resources
Feature image by Emily Mathews / University of Michigan School of Kinesiology
