Measuring Impact in Digital Youth and Adult Programs: Exploring the Three Levels of Evaluation
Image credit: PTBO Canada, Creative Commons
Canadian nonprofits have been investing more in digital learning opportunities and programs, and we've seen this trend grow firsthand. In just the last 3 years, we’ve witnessed nonprofit organizations at the forefront of the shift to digital learning, offering innovative digital programs that give people of all ages access to education and training. Nonprofit directors and program managers across Canada have made unprecedented decisions to take their learning programs and experiences online – and keep them there. In fact, research from Imagine Canada found that 54% of charities had moved some or all of their programs online, and 42% had developed completely new programs in response to need.
“nonprofit organizations [are] at the forefront of the shift to digital learning, offering innovative digital programs that give people of all ages access to education”
It's no secret that the COVID-19 pandemic forced many communities and organizations in Canada to switch to online learning. Non-profits responded by developing innovative digital programs and knowledge mobilization materials that provide access to education and skills training for people of all ages, including those who face barriers to traditional forms of education. With this rise and transformation to digital learning programs, non-profits are reaching more people than ever before.
Technovation Montréal's Digital Transformation Empowers Girls in Tech: A Case Study
Image credit: Technovation Montréal
Technovation Montréal is a Quebec nonprofit empowering young girls from marginalized communities to become tech entrepreneurs by supporting them in creating mobile applications or AI models that solve real problems in their community. Their 16-week bilingual program went online for the first time in 2021, with a simple synchronous instructor-led model, using Zoom and GoogleDrive.
The challenge with going digital: It quickly became apparent to Executive Director, Stéphanie Jecrois, that participants needed greater access and support online. A significant investment was needed to revise the program curriculum, and redesign the digital learning experience to include synchronous and asynchronous ways for participants. Technovation Montréal had an immense opportunity in front of them to evolve the program digitally to allow for greater creativity, collaboration, learning and connection between students, mentors, parents, the Montreal community and staff.
Technovation Montréal hired instructional designers and education technologists to support their team in transforming the way the program was delivered (i.e. platforms, tools, culture, processes) and experienced by the learners online. In 2022, the nonprofit opened its program and saw a spike in registrants coming from not only Montreal, but across Quebec, parts of Ontario and Guadeloupe. The program continues to be monitored and evaluated on a learner, program and community level to inform decisions for impact and improvement purposes. The Technovation Montréal team now supports over 150 young girls and family, 200 mentors, and intends on growing their educational impact and reach.
Measuring Digital Learning Impact
Canadian nonprofit directors and program managers have several reasons for investing in digital learning programs. One reason is to increase accessibility and flexibility for learners, as digital programs can often be accessed anytime and from anywhere. Additionally, digital learning programs can help lower the costs of program delivery and provide personalized learning experiences for participants. Another reason is the ability to collect valuable data and insights into learner engagement and progress, which can be used to improve the program and measure impact.
However, measuring and evaluating the impact of digital programs on learning has been a challenge for many organizations. As digital learning designers, we closely observe how nonprofits measure and evaluate the effectiveness of their digital programs. From our experience supporting nonprofits during their digital program transformations, we've realized that the standard program evaluation processes do not fully address the unique challenges of digital programs. Traditional evaluation methods often fall short in capturing the unique and complex ways that digital programs can impact learners.
Measuring and evaluating digital learning impact involves three levels: learner, program, and community. By measuring and evaluating at each of these levels, Canadian non-profits can gain a more holistic understanding of the impact of their programs and make data-driven decisions to improve program quality and outcomes. Digital tools and platforms, such as learning management systems, online video conferencing software, and social media, are frequently utilized by nonprofits to run and deliver digital programs. The transition to digital learning provides opportunities for more effective program evaluation, using online surveys, data analytics, and learner engagement metrics. By utilizing this data, nonprofits can enhance program delivery, make informed decisions, and showcase the impact of their programs on both learners and the community.
Measuring and evaluating the impact of digital learning programs is not just about improving program quality, it's about making a positive impact on Canadian society. As our country becomes more diverse, complex, and technically skilled, non-profits have a responsibility to keep up with the changing needs of their communities. But let's face it, building a culture of measuring and evaluating learning impact can be a challenge. It takes dedication and investment in staff and resources, as well as clear goals and objectives at a learner, program and community-level. However, by prioritizing continuous learning and improvement, non-profits can ensure that they are making a meaningful difference in the lives of those they serve. So let's embrace this challenge and work towards building a culture of learning that will benefit all Canadians.
Ready to take the next step towards measuring the impact of your digital learning programs?
Book a 30-minute consultation with one of our learning designers to learn more about how you can improve your program evaluation processes and make data-driven decisions.
