Bad data is at best a waste of resources, but may also misinform policies or programs. Five major topics including electronic data collection, survey design, enumerator effects, how to measure difficult concepts, and behavioral responses to research. Anyone who has done research knows very well these topics can often impact data quality, the external validity of study findings, and our understanding of results. While there has been growing investment in research in international settings, there has not been a strong effort to ensure that the data collected are of high quality. Motivating researchers, donors, and implementers to focus on these measurement-related questions will result in more data-driven technology and strategies to ensure accuracy and efficiency in data collection that can be used for better policy or programmatic decision-making.
Technology for data collection: There has been widespread and increasing use of technology to collect data, but no clear consensus that what we get is always ‘better’. We can harness crowd-sourced data using cell phones or mobile platforms, but do we lose representativeness? Is it always better or worse to collect our survey data on tablets and cell phones? On paper, enumerators could move around the survey and change the order of modules and questions depending on who was home to take the survey. Now, surveys are more linear, and we can put ‘speed bumps’ or checks into place (For example, if someone says they are 25, and next says their birth year is 1940, the tablet can calculate this and show the enumerator an error to fix). Is this loss of flexibility better or worse? The answer is often that it depends.
Incentives: While the measurement and data quality were important, almost none had made this a specific focus of their work or projects. Without strong incentives from donors or other partners, however, it is difficult to prioritize this work. Everyone wants high quality data without taking the time to understand how to get it. Embedding a measurement experiment within a larger project might be one strategy, but there are concerns that the embedded measurement experiment could undermine larger survey results. This is compounded by the fact that many journals will not publish articles related purely to measurement or papers on studies that have negative findings. So, if you are a researcher about to embark on a large-scale data collection, carefully consider what you are measuring and how, and how you can optimize data quality, and if there are opportunities to share your data.
Therefore, it is extremely crucial that research partners, implementing partners, and donors need to invest effort and resources to begin to focus on quality and measurement. Although incentives are not currently in place to encourage this type of work, these are necessary steps to improve data quality and ultimately strengthen our research findings that ultimately drive global policy.