Dr. Johanna Choumert-Nkolo
Part 1 of ‘Building a questionnaire for a household survey: a check-list’ can be accessed here.
[6] Do you have good measures of socio-economic status? Socio-economic status of respondents is a key variable of economic models, and yet perhaps the most difficult to measure. Money-metric measures, i.e. income and expenditures, can be interpreted as a measure of material well-being. However, it is difficult to measure accurately households’ income and expenditure levels in developing economies, especially in rural areas. Individuals may not have a precise knowledge of their income, may refuse to disclose it to interviewers, income may vary according to seasons, and there may be difficulties in taking domestic or unpaid production into account, etc. Using consumption expenditures as a proxy to socio-economic status raises similar challenges. Household expenditure is often preferred to income with the underlying hypothesis that consumption choices are determined by household’s estimation of long-term income and not by their current income. Hence, expenditures are expected to be more stable over time due to intertemporal consumption smoothing. Nevertheless, expenditures appear to vary significantly over time, which calls for caution when using this information (Filmer and Pritchett, 2001). For these reasons, the construction of a wealth index (poverty index or socio-economic status index) has been proposed by many in the literature (see Howe et al., 2009; McKenzie, 2004). This approach consists of reducing a set of variables (ownership of durable goods and living conditions) to a single component using principal component analysis; and allows to obtain the relative position of households. These individual pieces of information (ownership of a car, radio, TV, etc.) are less prone to reporting errors compared to income and expenditures. However, one must think carefully about which assets to include, especially in the presence of both urban and rural populations in the sample. See more Rutstein (2008) for more discussions.
Besides, whether you want to measure wealth at the household level or at the individual level matters; knowing that intrahousehold inequality is a serious concern (Haddad and Kanbur, 1990; Klasen and Lahoti, 2016), this deserves careful considerations. Finally, your research question matters: for instance, if you are looking into the impact of natural disasters on farm households, the literature stresses that the impacts of the shock, the response of households and their resilience will be very specific to each socio-economic status proxy (income, productive asset, non-productive assets, savings, consumption, expenditures) and that it is necessary to look at each one individually.
About measuring food consumption and expenditures, chapter 5 of Glewwe and Grosh (Eds.) (2000) and the articles published in this special issue of Food Policy are great resources; For income, see chapters 9, 11, 12, 17 and 18 of Glewwe and Grosh (Eds.) (2000); and for assets, see Rutstein (2008), Howe et al. (2009) and the Poverty Probability Index.
[7] Does your questionnaire contain too many questions which are difficult or sensitive? A good questionnaire can’t contain an hour of questions about agricultural practices on each plot, followed by an hour of questions about sexual behaviours. A questionnaire on sexual behaviours has to be handled very carefully as it is very intrusive and sensitive for respondents. A questionnaire about agricultural practices fully mobilizes the cognitive capacities of respondents and can be quite draining for them. In short, a questionnaire should not contain too wide a range of difficult topics.
[8] Have you shared the questionnaire with local researchers/stakeholders? During the training of field teams, you will get a lot of very useful feedback on the questionnaire. However, in order to avoid too many last-minute changes, before piloting the questionnaire you should share the tool with people who have a sound knowledge of your research topic and who know the local context well. Questions that work well in Tanzania, may not work in Uganda despite the proximity of the two countries so working with local experts is key.
[9] Have you piloted the questionnaire? Piloting is different from outdoor practice (which is a general rehearsal in real conditions at the end of the training). Piloting usually involves researchers, a field coordinator and a couple of interviewers. It consists of testing the questionnaire (as well as field protocols and other logistics). If your questionnaire has always been tried and tested, then a small pilot (e.g. 1 to 3 days depending on its length) may be sufficient. However, if your questionnaire is new, or contains difficult (or sensitive) questions, consider a longer pilot and even several pilots to test the changes implemented after the first pilot. Ask everyone to takes notes and allow for a good face-to-face debriefing session with the team. Piloting questionnaires is not wasted time/money at all and should be an explicit phase of the research project.
[10] Is the consent form in line with national and international standards? This is actually the first and most important part of any questionnaire. Obtaining informed consent is key to the process of protecting survey respondents and to the validity of the research. The key principles a consent form should follow, as well as recommendations about the content and format, can be found in a previous blog post I wrote, here. In addition, and depending on the country, it is essential that the questionnaire adheres to institutional review boards standards. For example, in Tanzania, depending on the topic and scope of the survey, questionnaires must be submitted to one or several of the following commissions: Tanzania National Bureau of Statistics (NBS), Tanzania Commission for Science and Technology (COSTECH), National institute for Medical Research (NIMR) and President’s Office Regional Administration and Local Government (PO-RALG).
This check-list is obviously far from being exhaustive. Questionnaire design is a complex art which requires a sound knowledge of the literature on the subject being studied, a prior study of the questionnaires already used on the subject, a good understanding of the key concepts of survey methodology research, and a strong sense of ethics. Here are a couple of additional references for interested readers:
- Demographic and Health Surveys (DHS) methodological reports on surveying households about health and demographics.
- Development Impact Evaluation (DIME) Wiki of the impact evaluation unit at the World Bank with some very useful resources on questionnaire design.
- Glewwe P., and Grosh M.E. (Eds.) (2000). Designing household survey questionnaires for developing countries: lessons from 15 years of the living standards measurement study. World Bank.
- International Household Survey Network Survey Catalog which provides links to thousands of surveys across the globe.
- LSMS publications, with a very rich list of articles and guidebooks on specific topics such doing household surveys about energy use, land area, ownership and use of assets, non-standard units for food consumption, food consumption and expenditure, education expenditure, adaptation to climate change.
- United Nations. Statistics Division (2008). Designing household survey samples: practical guidelines (Vol. 98).
