Evaluating AI Tools for Your Classroom

A Step-by-Step Guide for Teachers

A-diverse-group-of-students-collaborating-around-a-tablet
Two teachers discussing electronic devices safety

Why should I evaluate?

Evaluating AI tools is crucial for effective teaching and learning because it ensures that the technology is accurate, safe, and beneficial for students. AI can enhance education, but if not carefully chosen, it may spread misinformation, invade privacy, or create unfair learning experiences. By assessing AI tools for reliability, security, bias, and educational value, teachers can integrate them effectively while maintaining student trust and promoting critical thinking, ethical use, and personalized learning. A well-evaluated AI tool supports both teachers and students in achieving meaningful, responsible, and engaging education.

Key Considerations

Steps to Evaluate AI Tools

How to evaluate an AI tool

1. Define the Purpose

Identify the specific learning objective or problem you want the AI tool to address.

Examples: Personalized learning, grading automation, language practice, or STEM exploration.

Define the purpose to use AI

2. Assess Alignment with Curriculum Standards

Ensure the tool supports state or national curriculum standards (e.g., Common Core, NGSS).

Check if it enhances subject-specific skills like math, reading, or science.

Assess Alignment with Curriculum Standards, Common Core, NGSS, math, reading, science

3. Evaluate Accessibility and Inclusivity

– Verify that the tool is accessible to all students, including those with disabilities (e.g., screen reader compatibility, text-to-speech features).

Ensure content is culturally responsive and free from bias.

Evaluate Accessibility and Inclusivity

4. Review Data Privacy and Security

Confirm compliance with laws such as COPPA (Children’s Online Privacy Protection Act) and FERPA (Family Educational Rights and Privacy Act).

Investigate how student data is collected, stored, and shared.

Illustration of data privacy and security with COPPA and FERPA compliance, showing how student data is collected, stored, and shared

5. Test Usability and Engagement

Try out the tool yourself to assess its ease of use for both teachers and students.

Look for engaging interfaces that motivate students without being overly distracting.

Test Usability and Engagement with engaging interfaces for teachers and students

6. Check for Bias and Ethical Concerns

Analyze whether the AI algorithms exhibit biases based on race, gender, socioeconomic status, etc.

Avoid tools that promote harmful stereotypes or misinformation.

Cartoon image showing Check for Bias and Ethical Concerns

7. Seek Feedback from Stakeholders

Consult colleagues, administrators, parents, and even students about the tool’s potential impact.

Conduct small pilot tests before full implementation.

Seek Feedback from Stakeholders, Consult colleagues, administrators, parents, and even students about the tools potential impact, Conduct small pilot tests before full implementation

8. Consider Cost and Sustainability

Determine if the tool fits within your budget and offers long-term value.

Explore alternatives if costs are prohibitive.

Consider Cost and Sustainability Determine if the tool fits within your budget and offers long-term value. Explore alternatives if costs are prohibitive.

Example 1: Appropriate AI Tool
Khan Academy Kids


Teacher’s Goal: Enhance early literacy and numeracy skills for K-2 students.

– Purpose:
The app focuses on foundational skills like reading, writing, and math, making it ideal for younger learners.

– Curriculum Alignment:
Aligned with Common Core State Standards for early education.

– Accessibility and Inclusivity:
Offers multilingual support and adapts activities to individual learning levels.
Includes audio instructions for non-readers.

– Data Privacy:
Compliant with COPPA; no personal information is collected without parental consent.

– Usability and Engagement:
Simple, colorful interface designed specifically for young children.
Incorporates gamification elements like rewards and interactive stories.

– Bias and Ethics:
Content is neutral and promotes inclusivity across diverse backgrounds.

– Stakeholder Feedback:
Teachers report high engagement and positive feedback from parents.

– Cost and Sustainability:
Free to use, ensuring equitable access for all students.

Conclusion:
Khan Academy Kids is highly appropriate for K-12 classrooms, particularly for younger grades.

Example 2: Inappropriate AI Tool
ChatGPT (Unsupervised Use)

Teacher’s Goal: Generate essays or answers for middle school assignments.

– Purpose:
While ChatGPT can generate text quickly, unsupervised use may encourage plagiarism rather than fostering critical thinking.

– Curriculum Alignment:
Not explicitly aligned with any curriculum standards; lacks pedagogical structure.

– Accessibility and Inclusivity:
Advanced vocabulary and complex responses may confuse younger students.
Potential for biased or inaccurate outputs.

– Data Privacy:
Raises concerns about sharing sensitive prompts or student work with third-party servers.

– Usability and Engagement:
Students might rely too heavily on AI-generated answers instead of developing their own ideas.

– Bias and Ethics:
Known instances of generating biased or inappropriate content when prompted incorrectly.

– Stakeholder Feedback:
Many educators express concerns about academic integrity and over-reliance on AI.

– Cost and Sustainability:
Free versions have limitations; premium plans may strain school budgets.

Conclusion:
Unsupervised use of ChatGPT is not recommended for K-12 classrooms due to ethical and practical concerns.

Evaluation Example

Evaluation Resources

The Ultimate Guide to Evaluating AI Tools in Your School

toddleLearn: Set the tone for AI tools evaluations in your school. Explore practical steps that guide you through this evaluation process.

The No. 1 Question to Ask When Evaluating AI Tools

MIT Sloan Management Review: Determining whether an AI solution is worth implementing requires looking past performance reports and finding the ground truth on which the AI has been trained and validated.

A Guide to AI in Education, Google Education

Applying Google’s AI Principles to our work in education.

Rubric for AI Tool Evaluation

by Kyle Mackie and Erin Aspenlieder, copyright 2024 Paul R MacPherson Institute for Leadership, Innovation and Excellence in Teaching, McMaster University is made
available under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License, http://creativecommons.org/licenses/by-nc-sa/4.0/.

A Comprehensive Guide to Evaluating AI Tools for Education

The role of AI in education has become increasingly prominent. AI tools for education can change the way we teach and learn, making it essential for us to understand how to evaluate these tools effectively. In this guide, we’ll outline what you need to evaluate these tools.

Rubric for Evaluating AI Tools for Schools

How can a school evaluate these tools to see if they really do provide safe, educationally sound, cost effective solutions?
To assist with this process I have created a “Rubric for Evaluating AI Tools in Education” which covers 18 essential criteria for assessing these products. The rubric is freely available for anyone to use or modify as needed.