Callum Taylor
| “Data is only as reliable as the people collecting it”
Collecting primary data through face-to-face interviews is one of the most challenging forms of data measurement. Seeing an interview as a simple act of asking questions and recording responses is missing the wider picture. It is a far more nuanced and personal process than people often recognise. When using this method, the data is only as reliable as the people collecting it. Interviewers are expected to be sensitive and adaptable, while at the same time adhering to the boundaries laid out by the survey protocol.
Data collected through surveys is open to interviewer bias, which is exacerbated when there are multiple people conducting the interviews. It is also potentially open to manipulation or falsification if the correct structures, such as in-software validations and supervisor checks, are not in place. Our Surveybe software is an advanced surveying tool that helps us to reduce the potential for errors in the field. But, even the most powerful tool in the world is useless without people who know how to use it. EDI therefore also requires, among other things, a capable and knowledgeable workforce who can both programme the tool and use it in the field.
Figure 1 – EDI Training in Action
EDI is committed to providing our clients with the highest-quality data and, through our longstanding experience of collecting data from large-scale surveys, we have in place a number of rigorous processes to help achieve this.
One of the most fundamental steps we take is to ensure that all our interviewers are given in-depth training, both in interview style/methodology and on survey-specific issues. EDI invests a lot in our staff members and so we believe that much of our real value-added comes from our people.The interviewers are obviously a vital part of survey research, but also contribute to wider factors such as quality control (QC).
Their role is a complex one and goes far beyond the simple reading of questions and recording of answers. It also requires them to perform a number of wider tasks including: (1) Locating respondents, (2) Engaging respondents, (3) Clarifying any misunderstanding/issues, (4) Checking the quality/validity of responses, (5) Reporting to their supervisor.
| “Interviewer training helps lay important foundations for everything that happens in fieldwork”
This could place a lot of expectation and pressure on the interviewers, however our bespoke interviewer training sessions are designed to prepare our staff for these roles and provide numerous tangible benefits including:
- Familiarity with the survey and the software
Our interviewers are given plenty of time to familiarize themselves with the survey during training. Specific activities in this regard include survey walk-through’s, tips for using Tablets/PCs, practice interviews, outdoor/in-field practices, and survey-specific exams. At each stage of training the interviewers are given thorough feedback to help improve their understanding. We also aim to improve the understanding of our interviewers by giving them wider exposure to the research being conducted, for example by talking to them about the wider importance and aims of the project so they can understand the impact their work will have. A better understanding of the wider survey context helps to further engage our interviewers in their work and enables them to value their efforts in the project, which in turn leads them to maximise their contribution. Having good familiarity with the survey and the software helps the interviewers conduct the survey in a smooth and efficient manner and helps to reduce the chance of errors (e.g. through recording comments and solving errors).
- Identifying & addressing issues in the survey
As part of the training process, the interviewers are guided through the survey question by question. This is a particularly interactive stage of the training as each question is discussed at length to give interviewers the opportunity to clarify or raise any issues they might have. One of the most important aspects in this regard is the local expertise the interviewers bring (For example, knowledge about local food items, availability of resources). This helps us to reduce inconsistency errors within the survey and make any necessary changes before we go to field. Typically, the suggestions from our interviewers and local teams lead to a lot of changes, resulting in a much improved survey.
- Better consistency and efficiency
As a result of the discussions that take place around each question, we have greater certainty that all our interviewers interpret the question in the same way and know how to deal with all possible responses. Additionally, Surveybe can be programmed by our research team to display on-screen documentation to remind our interviewers about important questionnaire related issues. This helps to generate consistent data across all interviewers (see figure 2). The on-screen Surveybe manual gives our interviewers important guidance and clarifications about the questionnaire.
Figure 2 – Surveybe Documentation (Tablet)
- Improve skills and getting familiarized with the survey – high quality staff
As part of the training process, all interviewers will receive training on the hardware they will be using in field. They are also given the opportunity to conduct several practice interviews. At each stage they will both be given feedback on their interview and also have the opportunity to give feedback on the survey. This helps to improve their interviewing skills and increase their familiarity with the questionnaire, both ultimately leading to significantly shorter interview times. We also use paradata (e.g. interview start and end times, movement between questionnaire screens) to track our interviewers and check they are conducting the interviews in the appropriate way.
- Better Quality Control
The interviewer training also helps lay important foundations for everything that happens in fieldwork. It is an excellent opportunity to bring together everyone involved in the project – EDI staff from the UK, clients, coordinators, data processors, supervisors, interviewers, consultants, and other stakeholders. One of the key relationships is between the supervisors and interviewers. Our supervisors directly observe field interviews, so the training sessions are a good opportunity for supervisors and interviewers to get to know each other, for interviewers to know what to expect during field and what QC is required, and for supervisors to help improve the skills of their interviewing teams. Supervisors also go through their own training sessions, which help to reinforce the QC strategies that they are responsible for employing.
Additionally, all our interviewer teams are given training in research methods as part of their induction in order to make them aware of potential interviewer biases and how to reduce them. This is refreshed during the main training phase for each of our projects.
The working culture and team structures at EDI encourage all staff members, permanent or temporary, in the UK or in Tanzania, to input into every stage of the data collection process. In recognition of their importance to data quality, we aim to make our training sessions as inclusive as possible for our interviewers. We encourage them to give as many observations and ask as many questions as possible, particularly after initial testing. We believe this shows our interviewers that they are highly valued members of our workforce and are expected to contribute as much to the success of the project as anyone else.
|“A tool is only as good as people’s ability to use it”
In all our work, EDI strives to create a positive impact on peoples’ lives through the provision of world class research services, tools, and innovations that enable better decision making from household to global level. The structures and strategies outlined in this blog help to provide EDI with a highly skilled and trustworthy workforce. This in turn helps to improve the quality and reliability of our data and help generate our high client satisfaction. Our Surveybe software can be programmed to identify a wide range of potential errors, but a tool is only as good as people’s ability to use it. EDI’s combination of our advanced, in-house software with our well trained and experienced staff gives us an edge over many of our competitors and helps us to reliably provide our clients with the highest quality data.