Did It Work? Program Evaluation in Data Science

One of the most critical questions in data science is: Did my program make a difference?

This course empowers you to bridge the gap between data science and program evaluation. It begins by helping data scientists understand why program evaluation is critical to your work. By mastering program evaluation, you can confidently describe how your model can have impact in the real world, in areas like marketing campaigns, risk modeling, comparing efficacies, and more. From defining program evaluation to effectively communicating findings, this course equips you with a shared language and methods to drive impactful decision-making.

Learning objectives
Evaluate data science programs by identifying a program’s needs, processes, effectiveness, equity, and efficiency evaluations; applying appropriate evaluation designs based on context; assessing program implementation through process evaluation; and creating and analyzing logic models for data science projects.
Analyze program effectiveness through understanding the differences between pre-experimental, quasi-experimental, and experimental designs, assessing the validity of evaluation measures, evaluating reliability of data collection methods, and interpreting interrupted time series and A/B testing results.
Design evaluation strategies that address stakeholder needs and contexts, incorporate appropriate measurement tools, account for equity considerations, and include well-designed questionnaires and data collection methods.
Communicate evaluation findings by developing targeted dissemination strategies for different stakeholders, adapting presentation methods to audience needs, articulating program impact and effectiveness, and presenting results in an accessible and equitable manner.
Apply program theory by constructing logic models for data science initiatives, implementing appropriate evaluation designs, measuring program outcomes and impacts, and assessing program implementation fidelity.