Dr. Johanna Choumert Nkolo and Nathan Sivewright
Full title: Views from the Field – No. 4 | Keeping your follow-up survey on track: How to successfully utilise tracking tools in a baseline survey
Household surveys have become a key source of data to inform and evaluate policy choices. Although most surveys are cross-sectional, panel data sets / longitudinal data sets offer great perspectives to monitor development indicators, evaluate development programmes, provide insights into the dynamics of certain phenomena such as poverty traps, and track behaviours over time (See Scott and Mariotti, 2014). Such multi-phase surveys, with the same respondents, can be successfully implemented only if the initial baseline survey is of high quality. The stakes are twofold: data quality and costs of tracking respondents.
What is a baseline survey? It not only provides a snapshot of a series of indicators at one particular time, but it should also collect information that will allow researchers/survey managers to track respondents in future data collection rounds. Collecting high quality baseline data is key in the context of impact evaluations and the construction of panel data sets. For example, the Kagera Health and Development Survey (KHDS) is a study into the long-run wealth dynamics of households and individuals within North West Tanzania. It entails the resurveying of a panel of households, originally interviewed for five rounds from 1991 to 1994. Resurveys were then organised in 2004 and 2010. At least 90 research articles have been published using this publicly available data set, on a wide range of topics including migration, family networks, healthcare, education, poverty, etc.
When a researcher designs a baseline survey, careful consideration must be given to several dimensions. For example, does the season matter? (e.g. in the case of agricultural surveys); the evolution of technology e.g. people changing mobile phone numbers or providing the “community” phone number. The key overarching questions that the researcher must ask themselves are: (1) Will the information I collect allow me to track respondents/households in a couple of months/years? (2) Will the content of the questionnaire allow the user to track some key indicators?
Drawing on our collective experience in designing baseline surveys and implementing various follow-up surveys, we provide some tools and tips to maximise data quality and to facilitate tracking of respondents.
COLLECTING INFORMATION TO TRACK RESPONDENTS IN FUTURE DATA COLLECTION ROUNDS USING COMPUTER-ASSISTED PERSONAL INTERVIEW (CAPI)
The emergence of CAPI as a research tool has both enabled the collection of new types of information and made the collection of other data far easier. Some of these key developments are explored in this section, with specific reference to the ease of tracking respondents.
GPS: The introduction of CAPI has facilitated the use of GPS coordinates being taken to aid future tracking efforts. Within surveybe Android is the functionality to capture GPS coordinates within an interview file. This avoids interviewers having to take GPS readings from an external device and copying over long numerical input and removing the possibility of human error.
Figure 1. Tracking screen
GPS coordinates can help to locate respondents before a follow up as they can be marked on a map for interviews and also during the fieldwork when directions are not easily attainable (see below). They can also play a role in quality control as a researcher can map the GPS taken in a follow up against the previous GPS readings.
Figure 2. GPS plotting (de-identified data)
Mobile phone numbers: One of the most effective forms of tracking a respondent is to collect mobile phone information. Mobile phone numbers can be particularly useful as the tracking efforts can be conducted remotely, can take place prior to the field team arriving in an enumeration area, and can still be valid even if the respondent migrates. A quick call during a follow up study can potentially confirm the respondents’ participation in the baseline, their current location and even be used to arrange an interview time. It is advised at baseline to collect as many relevant phone numbers as possible, not only the respondent’s telephone main number, but any other telephone numbers they use, as well as those of other household members or even neighbours. The more phone numbers available to a tracker, the more options remain if the main respondent number is invalid.
In surveybe there is a flexibility in capturing multiple telephone numbers within in interview. One option is to create a roster of telephone numbers where the interviewer can continue to add relevant phone numbers, so that they are not limited to just one or two. A text question can collect the name of the phone number owner and a drop-down menu can quickly indicate the relationship of this person to the respondent.
Figure 3. Telephone contact roster
Using surveybe’s validation functionality means that any telephone numbers that have too many digits or fall outside a country’s phone number structure is flagged up during the interview. This reduces the number of invalid phone numbers that are collected and used at a follow up.
Figure 4. Telephone number fails validation test
Interviewer’s comments: The most basic tool for tracking respondents when conducting a baseline survey is for interviewers to write comments within an interview file that can include information on the respondent such as a brief description of the household, nearby landmarks or noting down contact information. Although it is generally true that the more tracking information taken at baseline, the better, care should be given to ensuring that tracking information is taken in an organised and easy to understand manner. Having dedicated sections within a survey to collecting phone numbers and location information is ideal, both for researchers using data and guiding interviewers to asking for the most vital information. These sections can be bolstered by interviewers adding comments to variables, if they have any further information that may be useful for tracking respondents – this gives interviewers the freedom to go above and beyond in their tracking efforts.
Figure 5. Interviewer comment for clarification
Photographs: One option to aid the tracking process is to take photographs of respondents during the baseline interview. In the follow-up survey, the image is presented via a unique identifier. Having images of respondents to refer to whilst undertaking tracking can be extremely useful in verifying whether the tracked individual is indeed the desired respondent from baseline. Verifying that the respondents are the same as baseline in follow up can be particularly difficult when tracking common names in urban areas, when there is a large gap between baseline and follow up as the respondent may not be able to recall, or confuse the survey with another. Doubts over whether an individual is the respondent desired can lead to frustration and delays in the data collection process as interviewers are unsure over whether to proceed with the questionnaire. Worse still, it may lead to incorrect respondents being interviewed which has consequences on the quality of data for analysis. One benefit of CAPI is the ease at which photographs can be taken and stored; surveybe has the functionality to take a photograph using the electronic device’s camera that is stored within an individual interview file – eliminating the need to ask respondents to hold up a piece of paper with an ID on.
Figure 6. Respondent photograph built into survey
Barcodes and QR codes: Depending on the context, taking photographs of respondents or houses in a baseline survey may not be appropriate (for example, a survey with highly sensitive questions). An alternative is to use non-identifiable information to track respondents such as barcodes and QR codes. Printed barcodes can be distributed to respondents at the end of an interview and asked to be held in a safe place. On return to the respondent in follow ups, the barcode can be scanned to ensure that it matches with the reference data collected at baseline.
EXPERIENCES FROM THE FIELD
The first step to a baseline survey that successfully collects tracking information for follow up interviews is to consider the full range of tools at your disposal and understand the context of your project, be it the country, industry, or nature of the research. A young respondent answering highly sensitive questions about their sexual activity is unlikely to want an interviewer asking for their parents’ phone numbers, nor will a respondent answering questions about political beliefs necessarily want to have their photograph taken. However with careful consideration, it is possible to develop tracking tools that are effective and respect the privacy of the respondent.
It is vital that interviewers are trained in the methods and importance of collecting tracking information before going to field. Trainers should explain why certain information needs to be captured, the benefits for future rounds of high quality tracking data and the consequences of low quality data. Training should also ensure that interviewers have the confidence to explain to respondents the necessity for collecting tracking information and put them at ease that providing this information will do them, nor others, no harm nor sold for commercial purposes.
Quality control activities on interviewers should not neglect tracking information. Checks for invalid telephone numbers or GPS coordinates can be incorporated as a real time validation check. Back-checks can be made to ensure that information provided is correct and direct observations by supervisors can assess an interviewer’s performance in collecting as much information as possible (i.e. did they probe for further telephone contacts?). Interviewers that consistently collect fewer telephone numbers than others or have high rates of respondents refusing to take a photo should be flagged up and checked in field. The audio audit functionality can be used in such quality control activities.
When conducting a baseline survey with a follow-up in mind, researchers should always enter the field with mantra that just five extra minutes spent collecting organised and meaningful tracking data now can save hours of time, money and, most importantly preserve their sample further on down the line.
FURTHER READING
A list of available micro panel datasets (as of June 2014) for almost 20 developing and emerging economies provided by the Chronic Poverty Advisory Network (CPAN) can be found here.
Bamberger M., Rugh J., Mabry L. (2012) RealWorld evaluation: working under budget, time, data, and political constraints, 2nd ed. ed. SAGE, Thousand Oaks, Calif.
Scott L., Mariotti C. (2014) A place for panel data in the ‘data revolution’. ODI Briefing 92.
Witoelar F. (2011) Tracking in Longitudinal Household Surveys. Washington, DC: The World Bank, Development Research Group, LSMS-ISA Integrated Surveys on Agriculture.
