Humans in the Loop: Insights from Codesigning AI in Real Classrooms

With the explosion of AI in education since ChatGPT’s release in November 2022, the edtech market has seen an influx of tools promising revolutionary changes to teaching and learning. But are these tools meeting real classroom needs? Leanlab Education set out to answer this question by working with five edtech companies, each developing AI-powered solutions, through codesign research involving over 40 teachers and their students across the U.S.

The Great Divide: Teachers' Expectations vs. AI Reality

Teachers bring three core expectations to AI-powered edtech. They expect:

1. Tools that save time.

2. Support for differentiated, engaging instruction.

3. Assistance with instructional design, delivery, and feedback.

These expectations aligned closely with the intentions of the edtech companies developing AI-powered tools involved in our studies. Their logic models and theories of change emphasized four primary goals:

• Personalize and differentiate learning to meet individual student needs.

• Enhance student engagement and increase agency in learning.

• Improve student outcomes by supporting educators and addressing diverse classroom needs.

• Support educators in designing, delivering, and providing feedback on instruction.

While teachers' expectations and edtech company intentions are well-aligned, Leanlab’s studies revealed significant gaps in practical application.

Across four Leanlab studies with teachers and students, several issues consistently impacted AI tools’ effectiveness:

• Technical issues like slow load times disrupted learning and wasted valuable classroom time.

• Differentiation shortcomings meant tools lacked flexibility to meet diverse student abilities, hindering personalized support.

• Trust in AI-generated content was low, with teachers questioning accuracy and reliability.

• Integration challenges limited how seamlessly tools fit within existing classroom workflows.

• Teacher familiarity and confidence with technology varied, affecting both adoption and effectiveness.

These barriers highlighted a core issue: while AI-powered tools aim to save time and enhance learning, their actual usability often falls short, leaving educators questioning the value of incorporating these tools into their classrooms.

Codesign Research: Bridging the Gap with "Humans in the Loop"

Leanlab’s codesign approach—facilitating collaboration between educators and developers—proved instrumental in addressing these gaps. Teachers provided ongoing feedback that informed real-time product changes, fostering improvements in both functionality and teacher trust. For instance, when teachers identified differentiation limitations, Leanlab recommended adding scaffolding to better support learners at various levels. This iterative process not only enhanced tool utility but also empowered teachers, who felt heard and valued in the product development process.

Building Trust and Usability in AI Edtech

Leanlab’s research demonstrates that embedding “humans in the loop” through codesign is essential to creating trustworthy, effective AI tools. This method helps companies validate that products address real classroom needs while evolving based on educator input. By collaborating with a diverse group of teachers and students early on, companies can ensure their tools are equitably designed and accessible across various educational contexts.

At Leanlab, we believe that codesign research is crucial for closing the gap between AI-powered edtech’s promise and reality in education. Interested in exploring how codesign research can shape your product? Discover Leanlab’s approach to creating trustworthy tools that align with teachers’ needs and ultimately transform education.

---

Codesign Product Research

Curious how codesign research can help bridge the gap for your AI-powered product? Learn more about Leanlab’s approach to creating trustworthy and effective tools by diving into Codesign Product Research. Let’s shape the future of AI in education, together.

---

A Year To Refocus on Community Wellbeing

When things got crazy in the early days of the pandemic, we started doing daily video calls with school leaders. While convening the education community has always been a core principle for us, we knew we needed to get in the trenches on a daily basis given the gravity of the situation that was unfolding.

The goal? To help them figure things out and plan. We pressed pause on our playbook and focused on core needs. 

Over the last 8 months, we’ve listened directly to our region’s schools and families, and our nation’s education innovators. As always, we’ve held steadfast to the belief that those closest to education-- parents, students, teachers--are the experts. 

True to our core values of human-centered design and boldness, we leaned in to understand their insights to provide direct support when our communities needed it most.

Even though it was a different direction for us, we prioritized basic needs and then we looked ahead at traditional school matters. While we’re not a direct service organization, we knew we had to pivot.  We did it the LEANLAB  way: 

We used research and data.

We served as a convener for our community schools.

We looked to the greatest needs to drive our actions.

Those early calls helped us understand the dynamic needs coming up for schools. Those calls moved us toward the connectivity work for which we might not have otherwise seen the need. 

We knew our community was counting on us to shift resources and reconfigure priorities so that’s what we did. It wasn’t perfect. It was messy. We didn’t get everything right.

But we listened. We responded. We took in the data and acted on it--arm-in-arm with our partners.

As a result, thousands of Kansas City kids got connected to the internet and were able to access educational services. 

I’m someone whose entire career depends on the ability to connect and communicate. I know first-hand how crucial those skills are for our kids’ future success. I’m honored we could deliver on that promise this year.

Sincerely,

---

---