Using innovative technologies to measure behavior change in public health programs

Using innovative technologies to measure behavior change in public health programs – Mercy Corps Indonesia recently partnered with Portland State University’s SWEETLab to pilot remotely reporting sensors for measuring the use of water and sanitation facilities in poor urban neighborhoods. This paper compares findings from the sensors to traditional evaluation methods and suggests some powerful possibilities for understanding program effectiveness.

INSTRUMENTED MONITORING WITH TRADITIONAL PUBLIC HEALTH EVALUATION METHODS: An application to a Water, Sanitation and Hygiene Program in Jakarta, Indonesia, 2013.

• Full text, pdf

Evan A. Thomas, Kay Mattson.

This paper presents the findings from two complementary approaches to field evaluation of a water, sanitation and hygiene program conducted by Mercy Corps in Indonesia. The program under study installed hand washing stations and latrines at a number of sites across greater Jakarta, and conducted extensive behavior change programs to encourage use of the latrines and hand washing after utilization of latrines.

A public health evaluation was conducted at the end of the program, and found a high degree of self-reported compliance with the desired behaviors. A year later (and after the program’s end), instruments were installed that directly monitored hand washing water taps and latrine use, and correlated sanitation facility use against water use at hand washing taps. These results suggest a significantly lower behavior change than earlier estimated in the studied communities. This insight allowed further review of the survey results, and incorporation of the combined data sets into thoughtful review of behavior change efforts.

This paper illustrates that instrumented monitoring systems can provide organizations with a data collection method that can greatly enhance their capacity to measure behavior change in global health programs. Mercy Corps, like other development organizations, invests significant resources in water and sanitation infrastructure and behavior change programs, as well as in monitoring and evaluation staff time. Mercy Corps is also aware of the limitations of self-reported survey data, which can be biased, as beneficiaries may have a tendency to report desired results rather than actual practices.

Using instrumentation with remote data access may have the potential to enhance data collection and complement survey and observation methods as well as monitor specific program components over longer periods of time than surveys allow, at a lower per-sample cost, while more directly and objectively measuring behavior and reducing self-reporting bias.

Instrumentation, with its own limitations including difficulty in discriminating details of behavior, as well as cellular data costs and nontrivial data analysis methods, is not a replacement for survey tools. Rather, it may supplement traditional evaluation methods. Combining remote sensing evaluation tools with traditional evaluation methods may enable programs to have both objective quantifiable data, along with qualitative data from beneficiaries about their knowledge and practices associated with the desired behaviors that could serve to better inform and improve overall program outcomes.

Tagged as: behavior change, Indonesia, monitoring