Dr. Johanna Choumert-Nkolo, Julius Josephat, Marie Mallet, Luca Privinzano, Callum Taylor
Sustainable Development Goal 4 calls for universal quality education. Education is central to unlock opportunities, provide better lives, reduce inequalities, and promote social inclusion. Significant progress remains to be achieved, especially in Sub-Saharan Africa, as evidenced in the graph below. The challenges are twofold: improving children’s access to quality education and improving their learning outcomes. To address these challenges, we need reliable education statistics and robust evidence to inform decisions about education policies, to understand how teachers teach, how students learn, and the contribution of parents to learning outcomes.
Source: UNESCO, Fact Sheet No. 48 February 2018 UIS/FS/2018/ED/48
EDI has wide ranging data collection expertise including the design and implementation of surveys of schools; conducting interviews with head teachers, teachers, students, parents, officials, and also conducting classroom observations. For the project “Experiments to Improve Learning Outcomes in Tanzania”, we conducted over 10 rounds of data collection on behalf of Twaweza with 350 public schools, 350 head teachers, 1,500 teachers, and 10,500 students. The data is used in: Mbiti, I., Muralidharan, K., Romero, M., Schipper, Y., Manda, C., & Rajani, R. (2018). Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania (No. w24876). National Bureau of Economic Research. EDI is also in the process of conducting education-related data collection activities for NORC at the University of Chicago. The “Impact Evaluation and Performance Evaluation for the Waache Wasome Program” has involved a series of surveys of schools in Tanzania with more than 350 parents, 2,400 adolescent girls, and of attendance records from 61 schools. EDI will be conducting follow up surveys as part of this project throughout 2019 and 2020. Last, EDI is currently involved in the data collection of the project “Research on Improving Systems of Education”, on behalf of Georgetown University Initiatives on Innovation, Development and Evaluation. This complex survey, with eight different tools, involves interviews with head teachers, teachers, students, Ward Education Officers, District Education Officers as well as classroom observations and assessments with both students and teachers. For this survey, we are visiting 397 schools, reaching a target sample size of 397 head teachers, 3,970 teachers and 7,900 students.
Over the years, we also have implemented numerous surveys focused on the learning outcomes of children and adolescents, and related topics such as youth empowerment. Below we provide a few tips and recommendations to researchers wishing to conduct school surveys.
1. Timing and logistics
Informing all stakeholders
During the preparation phase, it is essential to inform all key stakeholders of the survey activities. These can be the local Ministry of Education, regional and local authorities, and relevant Institutional Review Boards. They can assist in building the field plan around school examinations and holidays. They can also provide support letters to facilitate the introduction of field teams to schools.
Avoiding exam periods
It is usually hard, if not impossible, to conduct a school survey during examination periods, when teachers and students are busy preparing for and sitting examinations. Furthermore, some schools may not allow surveys in the weeks leading up to exams as they do not want any unnecessary disruption. When preparing the field plan, it is thus recommended to contact schools ahead to iron out this point. The same applies to holidays, elections or any event likely to disrupt the day-to-day functioning of schools.
Locating schools
In field, distances between schools can be large, especially in rural areas. It is essential to prepare field teams to plan for these remote schools. Dropping the schools from the sample simply because they are difficult to reach would introduce a bias to the sample.
When and where can interviews take place in a school?
It is advisable to only conduct data collection activities during school hours. Not only will this prevent unwanted tension with the local community from seeing strangers on school grounds during non-school hours, but our experience tells us that teachers are not always willing to stay after their typical working hours to participate in surveys.
Where possible, especially when working with students, conducting assessments, or asking sensitive questions, the activities should take place in a private space. Depending on the space available, some creativity may be required. Often this may mean conducting interviews outside, though teachers may also be able to find a quiet space within the school. Again, this is an important point to discuss with the head teacher upon arriving at the school.
2. Interviewing head teachers and teachers
Getting cooperation and consent
When field teams arrive at a school for the first time it is vital that the first step they take is to meet with the head teacher to explain the aims of the project, the activities that will take place, and to gain their consent. Having the head teacher on side with all research activities will make the job of the field teams much easier since they have the power to prevent any and all activities from taking place.
Availability and knowledge of head teacher
The head teacher may be needed for listing and sampling of teachers in the school according to specific eligibility criteria. It should be noted that the head teacher may not be the preferred respondent for this activity especially if he or she is new to the school. Besides, head teachers are usually very busy, therefore conducting a full listing with them without being interrupted by parents, teachers or students is almost impossible. Thus, a short interview is preferred for the head teacher and the listing of teachers can be conducted with another respondent in the school such as a senior teacher or the deputy head teacher who may be more available for this activity.
Planning for revisits
Since some teachers may be busy during the school day and may not be willing to stay after school hours to participate in the interviews, one should plan back-up days to revisit schools in the field schedule.
Conducting teacher assessments
Some school surveys include a teacher assessment component that aims to learn and assess the teacher’s teaching skills in order to improve student learning outcomes. The test may consist of a short exercise replicating a student homework/exam that the teacher has to correct and appreciate as if it was a real test completed by a student. The necessity to anonymise the test with unique IDs and not names is required to also ensure acceptance from the teacher to participate in this activity. It may be that some teachers are not willing to complete some parts of the tests, if unrelated to the subjects they currently teach (e.g. a Math teacher may not want to answer English related questions as part of an assessment). It is advised to implement all the teacher assessments jointly as much as possible (i.e. all teachers are assessed in a room at the same time). Finally, the timing for organising the session with all teachers should take into account the specific characteristics of the surveyed population. In some countries, like Tanzania, the teachers in first grade of primary schools are in majority women who have an important role in the household and are less likely to be available during lunch break or after class hours in order to fulfill these tasks. The best timing to maximise the availability of the teachers is at the start of the school day.
Tips for conducting classroom observation with teacher
Before the start of the lesson, it is important to make sure how long the lesson is going to be. This is essential in order to make sure that the enumerator has enough time to complete all the required number of segments for observation and segment for scoring within the allocated time of the lesson. Moreover, it is also essential for the enumerator not to interact with the classroom or the teacher, in order to keep the dynamics of the classroom unaltered and as close as possible to any ordinary school day.
3. Interviewing students
Consent from teachers or parents?
Seeking the consent from young respondents and the approval from a capable adult who is responsible for the young respondents is essential. Depending on the content of the questionnaire, either consent from the parent(s)/guardian(s) or the head teacher is required. For instance, if the student questionnaire is strictly related to the school environment (i.e. no questions about personal information outside the school are asked) then the consent of the head teacher is sufficient. But, in other cases where personal or sensitive questions are being asked the consent of parents should be sought.
Taking students out of lessons
Based on EDI experience, teachers and head teachers are willing to allow researchers to take students out of class to conduct student interviews or assessments when the teachers have been informed of the purpose of the research and the contribution of the survey to the education system. A short student interview is preferable to avoid them being out the class for too long. It is very difficult to reach the student during a school break, lunch time or after school hours.
Assessing students
One of the crucial aspects to keep in mind while designing student assessments is the time of the year in which the survey is planned to take place. In order to have homogeneous and comparable outcomes, it is fundamental to assess students that are supposed to have reached a common level of knowledge. Indeed, the content of the test should be targeted to measure knowledge and skills that all the students of the same assessed level have already acquired. The risk of assessing students on skills to be learnt during the same school year in which the survey is taking place would lead to uncomparable results, since there could be differences in the timing that given topics are covered across classes of the same level.
4. Interviewing parents
How to contact parents?
Contacting parents without contact details may be challenging and time consuming. A successful strategy used by EDI consists of giving an invitation letter addressed to parents to the students during class with the help of the head teacher. The letter should explain the study and purpose of the survey and invite the parent(s)/guardian(s) to come to school for the survey at a specific time. The letter may contain a tear-off slip to indicate the name and contact details of parents and to be returned to the head teacher via the student, so that researchers have an idea of the expected number of parents on the interview day and potentially follow-up with them about the survey date.
Getting parents to come to school and on time
Getting parents to visit schools to participate in research activities can prove to be a problem. Attendance varies and often parents show up late or not at all. If this happens regularly it could be costly, or mean that field teams are not able to complete the intended number of interviews. By sending letters to parents or otherwise contacting them in advance for example by text message, researchers can help to ensure that parents are encouraged to attend at the scheduled time and understand the importance of their input.
Getting parents interested in the project
When conducting face-to-face household surveys, a vital part of the questionnaire introduction is an explanation of the nature and aims of the research, which can help to pique the interest of the respondent. With education surveys, the same is true of the parents. From our experience we know that parents are keen to learn about research which could impact on their children’s education and wellbeing. However, they are also keen to see the proposed improvements take place. Follow-up research needs to be aware of this so that parents feel like their contributions are valued. Explaining the value of the research during the first contact with parents can also help to encourage them to partake in the survey.
