From A to Z: Exploring Decision Fatigue in Grading

About the Episode: 

In this episode, Jamie and Timothy dive into the fascinating world of decision fatigue and its impact on grading practices in higher education. Drawing from a study conducted at the University of Michigan, which analyzed over 30 million grading records, Jamie and Timothy discuss how individuals with last names lower in alphabetical order tend to receive worse grades due to decision fatigue among graders. They explore various causes of decision fatigue and its effects on grading outcomes, shedding light on biases such as anchoring bias and sequential grading biases. Furthermore, they examine potential solutions to mitigate decision fatigue, including randomization techniques and reducing class sizes. Join in the conversation to share valuable insights into the challenges of grading practices and the importance of designing equitable systems in education.

‍

Articles:

Link to Michigan study

Link to Journal of Athletic Training study on randomization

Key Takeaways

• Sequential Grading Bias Is Real:
  • Students graded later in a sequence (e.g., alphabetical order by last name) receive lower grades on average due to decision fatigue.
  • A University of Michigan study showed students with last names in the W-Z range scored an average of 0.6 points lower than those in the A-C range.
• Decision Fatigue Goes Beyond Grading:
  • The phenomenon impacts various fields, including medicine, sports officiating, and judicial rulings. For example:
    • Doctors are more likely to miss diagnoses later in the day.
    • Judges grant fewer paroles as the day progresses.
• Practical Solutions for Educators:
  • Randomization: Avoid alphabetical grading order; instead, randomize the sequence of review.
  • Smaller Class Sizes: Reducing the number of assignments per grader minimizes fatigue.
  • Rubric Simplification: Implement binary (yes/no) rubrics to make grading more objective and less mentally taxing.
  • Breaks and Self-Awareness: Encourage graders to take regular breaks to avoid the cumulative effects of fatigue.

‍

Episode Summary

The Science of Decision Fatigue in Grading

Jamie and Timothy explore the fascinating concept of sequential grading bias, as revealed by a large-scale study at the University of Michigan. By analyzing grading records from over a decade of learning management system (LMS) data, researchers found that students graded later in a sequence tend to receive lower scores and harsher comments from graders. This trend was especially pronounced in systems defaulting to alphabetical grading order, though the inverse occurred when grading began with Z and moved toward A.

The hosts relate this bias to broader research on decision fatigue, citing parallels in medicine, sports officiating, and even judicial rulings. For instance, doctors are less likely to maintain proper hygiene or catch medical conditions as their day progresses, and judges issue harsher sentences or deny parole more frequently in the afternoon.

‍

Addressing Grading Bias

To combat grading inequities, Jamie and Timothy propose several actionable strategies:

1. Adjusting LMS Defaults:
   • Many LMS platforms, like Canvas, sort submissions alphabetically by default. By shifting to randomized sorting as the default, institutions can distribute bias more equitably.
2. Enhanced Randomization Techniques:
   • Beyond simple randomization, block randomization (grouping students by similar characteristics) and covariate adaptive randomization (considering prior sorting history) offer more sophisticated ways to ensure fairness.
3. Structural Changes in Grading:
   • Reducing class sizes or distributing grading responsibilities across multiple reviewers can help alleviate individual fatigue.
   • Simplified rubrics and clearly defined criteria reduce cognitive load, fostering more consistent evaluations.
4. Grader Self-Awareness:
   • Encouraging graders to recognize signs of fatigue and take regular breaks can help mitigate its effects. Maintaining a comfortable grading environment is also key.

‍

The Bigger Picture: Decision Fatigue in Society

The conversation expands to consider how decision fatigue influences high-stakes decisions in various sectors. Whether it’s doctors diagnosing patients, judges granting parole, or sports referees officiating games, decision fatigue is a universal challenge. The hosts emphasize the importance of designing systems that account for human limitations, enabling more equitable and accurate outcomes.

‍

Final Thoughts

Grading bias caused by decision fatigue is not just a theoretical concern—it has tangible consequences for students and educators alike. By adopting evidence-based strategies like randomized grading orders, smaller class sizes, and simplified rubrics, institutions can create fairer, more equitable learning environments.

For educators and administrators seeking actionable change, the hosts recommend starting small: leverage LMS tools to randomize grading order, and encourage regular breaks during grading sessions. As Timothy notes, designing better systems isn’t just about improving grades—it’s about improving the entire educational experience.

‍

Connect With Our Co-Hosts:

Jamie Boggs
https://www.linkedin.com/in/jamiewboggs/

Timothy Davis
https://www.linkedin.com/in/davis-timothy/

About The Enrollify Podcast Network: The EduData Podcast is a part of the Enrollify Podcast Network. If you like this podcast, chances are you’ll like other Enrollify shows too!  

Some of our favorites include Generation AI and The Higher Ed Geek.

Enrollify is made possible by Element451 —  the next-generation AI student engagement platform helping institutions create meaningful and personalized interactions with students. Learn more at element451.com. 

Connect with Us at the Engage Summit: Exciting news — Jamie and Timothy will be at the 2024 Engage Summit in Raleigh, NC, on June 25 and 26, and we’d love to meet you there! Sessions will focus on cutting-edge AI applications that are reshaping student outreach, enhancing staff productivity, and offering deep insights into ROI. 

Use the discount code Enrollify50 at checkout, and you can register for just $200.

Learn more and register at engage.element451.com — we can’t wait to see you there!