best books home library background. read vintage books in a row. college library table. vintage books publishing

December 2023

Technology in Program Evaluation: Embracing the Last Two Decades

When I entered the field of program evaluation in 1999, paper surveys were the norm and literature reviews meant a trip to the nearest university library. Today, online portals have replaced paper surveys, and literature reviews can now be conducted on any handheld device. Yet constraints like limited human and financial resources and complex social issues surrounding program participants remain. Addressing these challenges involves continuous learning, adaptation, and innovation in evaluation.


As a practitioner, I have noted some shifts in evaluation in the past two decades. In my view, these include:

  • Technology integration: The use of technology in data collection, analysis, and reporting has vastly expanded. Digital tools and software allow more efficient data gathering, sophisticated analysis, and interactive reporting, including AI and machine learning.
  • Increased stakeholder engagement: Evaluations have become more participatory, involving stakeholders in the design, implementation, and interpretation of evaluation findings, leading to more inclusive and insightful assessments.
  • Demand for real-time data: A greater demand for real-time or near real-time evaluation data to inform ongoing program adjustments and decision-making.
  • Rise of big data and analytics: Leveraging big data and advanced analytics techniques to derive insights and patterns that were not easily discernible before, providing a more nuanced understanding of program impacts.


For many months, I have kept a list of tools in my tattered notebook (not very tech-forward, I know). I'm sharing these resources with the hope they may contribute to addressing some vexing issues evaluation practitioners face. Resources have been grouped into three broad categories: data collection, analysis & visualization, and writing & research. The recommendations are made with an eye to every evaluator's challenges, including time and resource limitations and multiple competing priorities. Included in the list are platforms that are free or have a free version, though sometimes this means limited functionality.



best books home library background. read vintage books in a row. college library table. vintage books publishing

Data collection

  • Google Forms: Create surveys and collect data.
  • SurveyMonkey and Qualtrics: Create surveys and analyze responses.
  • Otter AI: Transcribe meetings, interviews, and focus groups
  • Zoom: Conduct online data collection, transcription feature allows transcription of video or audio recordings.


Analysis & visualization

  • Airtable: A flexible spreadsheet/database hybrid that helps organize and analyze data.
  • IBM SPSS: Statistical software for data analysis.
  • Microsoft Power BI: Data visualization and business intelligence.
  • NVivo: Qualitative data analysis tools.
  • Tableau Public: Data visualization tools for creating interactive dashboards.
  • QDA Miner Lite: Qualitative data analysis software for coding and analyzing text data.
  • R and RStudio: Open-source programming language and IDE for statistical analysis and data visualization.
  • Canva: Produce communications, data visualization, and infographics.


Writing & research papers

  • Google Scholar, JSTOR, Social Science Research Network (SSRN), Sci-Hub: Offers access to scholarly articles, books, and papers across various disciplines.
  • Mendeley and Zotero: Organize research papers, generate citations, and collaborate with team members.
  • ResearchGate: A platform for networking with researchers, accessing publications, and collaborating on projects.
  • SAGE Research Methods: Provides access to resources on research methods and statistics.


In the evaluation context, embracing technological advancements, accepting greater stakeholder involvement, and understanding complex social issues present the opportunity to have engaged, inclusive, and community-involved evaluations sensitive to a project's fiscal constraints.



(This article is the first in a three-part series. Part two covers the use of AI for evaluators and academics, and part three will speak to the benefits and pitfalls of social media in program evaluation.)