This post originally appeared in Edtech Insiders.
The post-pandemic era has led to learning loss and an increased dependence on edtech tools—however, the majority of edtech tools still lack credible evidence supporting their efficacy. One 2023 report noted that only 39% of the most-accessed learning solutions support their usage with published research, while only 26% support their usage with studies aligned to federal standards. This is true despite the implementation of the 2015 Every Student Succeeds Act (ESSA), which clearly outlined four tiers of evidence to support edtech adoption in schools.
Why have so few edtech products engaged in efficacy research, and especially few in objective third-party research & development or evaluation studies?
Some attribute the lack of evidence to the lack of demand for evidence among school purchasers. Historically, many districts have relied on peer recommendations rather than a tools’ evidence base to make purchasing decisions.
Others note the noisy credentialing environment, in which organizations with no regulatory authority badge and credential products, makes it hard for decision makers to know which products are truly evidence-based.
Still others point to the cost of research—the painful, expensive, and time-intensive third-party evaluation processes that often don’t keep pace with the realities of a fast-moving tech sector.
However, as a flood of new AI-enabled edtech enters the market at the same moment that schools face the ESSER funding cliff, buyers are starting to cast a more discerning eye on which tools are truly moving the needle on outcomes.
Raising the Bar for Edtech Adoption
This June at ISTE Live, one group of education leaders sought to support the field by outlining some top considerations purchasers should consider when adopting new edtech solutions:
Does the tool have a credible evidence base?
Will the tool help us address a critical learning need?
Is tool adoption and implementation within our budget?
How much time will staff need to commit?
How will instructional time be impacted?
What data will be collected?
Is the tool safe for use with our students (e.g., will student data be collected)?
Is the tool just one more thing/initiative for our teachers to deal with (AKA: does the cost of getting buy-in outweigh the reward)?
What are the edtech company's intentions?
Simultaneously, a coalition of the sector’s leading edtech arbiters—1EdTech, Cast, CoSN, Digital Promise, InnovateEDU, ISTE, and SETDA— launched five Edtech Quality Indicators to bring more cohesion to the field. Consistent with education leaders' needs, the five indicators state that quality tools should be safe, evidence-based, inclusive, usable, and interoperable.
These parallel developments point to a potential shift in the market; nonprofit organizations are engaging in collective impact work to send cohesive and consistent signals into the market, while purchasers are seeking safe, vetted, streamlined, and cost-effective solutions.
Inclusive Innovation: Preventive Medicine for Edtech
While there is a need for more evidence measuring the potential of edtech, the evidence that does exist is promising… at least for some populations. Laurence Holt’s remarkable EdTech Next article “The Five Percent Problem” cites studies demonstrating considerable student learning gains from leading products Khan Academy, Dreambox, and IXL. The problem he points out is that these gains were realized only by “students who used the product as intended.” How many of the students “use the product as intended”? Often, as few as five percent of students studied, and often those with additional benefits. The obvious question: what do we do about the other 95%?
Enter inclusive innovation, a R&D approach that aims to increase the usability and effectiveness of emerging solutions by focusing on the process by which products are created and evolve. Inclusive innovation refers to a systematic process that elevates the insights of users representative of the diverse backgrounds, needs, and abilities in American classrooms.
The theory underlying inclusive innovation is that when products are:
Developed in close proximity to diverse, representative users
Designed to incorporate the feedback of diverse users
Built on a pre-existing foundation of evidence-based best practices
… the resultant products are more likely to be implemented as intended, and thus more likely to demonstrate impact on student outcomes.
We see inclusive innovation as ‘preventative medicine’ for edtech companies. By incorporating early touch points and feedback loops with diverse student populations while products are being developed, companies can expect reliable usage that drives long-term results.
Various organizations in the field describe such inclusive innovation practices, as ”inclusive R&D”, “co-design”, “co-creation”, “co-development”,” inclusive design”, and “participatory methods”. In practice, all variants of inclusive innovation involve engaging teachers and learners early and often during product development. Whether it’s the creation of user surveys, the development of research questions for focus groups, or the work to match an edtech company with a school to co-design an efficacy study that addresses school-identified goals, inclusive innovation practitioners are working directly with learner communities.
The Importance of Inclusive Innovation in AI-Powered Edtech
While AI has the potential to revolutionize education through personalized learning experiences and increased teacher efficiency, it also has the well-documented potential to perpetuate bias, threatening to exacerbate existing inequities in education.
The speed at which AI technologies are developing is rendering traditional research methods (which can take multiple years) unhelpful or obsolete. So, how might R&D help guide AI-powered edtech toward its full potential; to advance educational equity rather than threaten it?
Rather than eschewing research altogether, we believe that inclusive innovation practices—like rapid-cycle evaluations and iterative co-design—can be leveraged to produce timely and actionable findings for companies. But what exactly are these approaches and how do companies access them?
Many organizations are engaging in this work, often through collaborative efforts. Here are a few examples from the field.
Leanlab Education: Building Communities and Co-Designing Innovations
Leanlab Education is a nonprofit with a mission “to study and grow transformational education innovations co-designed with school communities”. Leanlab acts as the intermediary between edtech companies and school environments, matching partners on projects and leveraging its in-house research team to facilitate product feedback and research studies aligned to the ESSA tiers of evidence. By involving educators and students in the research design and product development process, emerging learning innovations address real-world challenges and meet the needs of diverse learners—intentionally designing for the margins.
By pairing enthusiastic education partners trained in inclusive innovation methods with companies seeking third-party research, Leanlab has increased access to inclusive R&D; to date, Leanlab has engaged over 30,000 students in nearly 80 studies. This approach ensures that emerging innovations are equitable, user-friendly, and a value-add to school communities, and thus more likely to demonstrate significant impact.
Central to this work is the newly-launched American Group of Innovative Learning Environments (AGILE) Network, a cohort of public schools, charter schools, and nontraditional learning communities ready to partner in R&D through a centralized (and co-designed) infrastructure. While managed by Leanlab, the AGILE Network is a collective impact initiative built on findings from the Global Edtech Testbed Network and supported by a growing number of partners—including founding partner AERDF, Digital Promise, and ISTE.
ISTE: Edtech Index
ISTE (along with acquired non-profit ASCD) is a global education community working to accelerate innovation in education through the smart use of technology. Well-known for their marquee conference, ISTE Live, that convenes more than 17,000 educators annually, ISTE is now using its reach to elevate the importance of evidence-based edtech and inclusive innovation.
ISTE’s Edtech Index consolidates information and validations on over 1,500 edtech products in one accessible platform, as well as a free teacher ready evaluation tool that helps educators determine the usability of edtech products for their contexts. The Edtech Index measures products aligned to the five domains of quality identified by the partnership of groups in the field: safety, evidence-base, inclusivity, usability, and interoperability.
Digital Promise: AI, Equity, and Inclusive Networks
Digital Promise’s efforts to advance equitable education systems through R&D takes various forms, including AI literacy and digital equity initiatives, work through their Center for Inclusive Innovation, and the Learner Variability Project. By conducting rigorous human-centered research, making findings accessible, and providing resources for educators, Digital Promise helps create an environment where all students can benefit from edtech advancements.
The Center for Inclusive Innovation reimagines education R&D by building the capacity for districts and communities across the country to engage in R&D, with a focus on supporting students furthest from opportunity. After their network partners co-design and engage in R&D, project details and solutions are shared with the larger field, highlighting R&D practices in diverse contexts. Digital Promise has also been an early leader in AI in education policy guidance, creating an AI Literacy Framework, leading workshops for school leaders, and advocating with companies, government, and schools for keeping humans in the loop.
AERDF: Finding Bold and Equitable Solutions
The Advanced Education Research & Development Fund (AERDF) maintains an extensive portfolio of inclusive R&D projects that incorporate learner and educator perspectives from start to finish. Inspired by federal Advanced Research Projects Agency (ARPA) initiatives, AERDF’s focus on equity-driven R&D ensures that breakthroughs are developed that address systemic inequalities in education, with a focus on Black and Latino students and students experiencing poverty.
AERDF’s advanced R&D projects involve sustained collaborations with educators, researchers, and communities to co-create solutions that are effective and inclusive. Their work creates bridges across the ecosystem, tackling challenges like assessments, math outcomes, and literacy, and showcases their findings for the larger field.
Chan Zuckerberg Initiative: Partnering to Co-Build
As a funder and collaborator in the edtech field, the Chan Zuckerberg Initiative (CZI) supports the development of evidence-based, co-created personalized learning experiences through the use of technology. CZI partners with organizations conducting and promoting inclusive R&D–including Leanlab Education, Digital Promise, and ISTE–as well as educators and students to co-build tools and reimagine education.
CZI engages in co-building through their edtech innovation studio, Render. With their collaborative product development approach, Render is creating AI-customized math materials, tools focused on elementary literacy, and administrative applications for classroom grouping. Reflecting their inclusive approach, Render’s co-building opportunities are posted online and matched to educator interests.
The Path Forward: Scaling Access to Inclusive Innovation
As AI continues to reshape the edtech landscape and schools look for breakthrough solutions that improve outcomes for all students,, there is an opportunity to establish inclusive innovation as an expected norm. Stakeholders across the edtech ecosystem—educators, researchers, solution providers, funders, and policymakers—can all engage in inclusive innovation and guide the transformation of K-12 education.
Through collective action, we can ensure the future of edtech, particularly AI-powered edtech, is not only innovative but also inclusive.
Here’s how each group can get started:
Educators, district leaders, and students can get involved in R&D projects, with opportunities aligned to their needs, interests, and capacity. Leanlab Education, In Tandem, Transcend, and Digital Promise are only a few examples of robust innovation communities.
Schools, when seeking out new edtech, can check validations in the Edtech Index, use Evidence for ESSA or other reviewing bodies, or include inclusive design criteria in RFPs.
Solution providers can co-build promising edtech with the communities they aim to serve, tapping third-party research organizations that use inclusive R&D methods or design internal R&D efforts on co-design principles.
Funders, policy makers, and solution providers can leverage open research, guidelines, and other resources to support inclusive R&D efforts, improve developing edtech, and collaborate more effectively.