Using Surveys to Estimate the National Prevalences of Modern Slavery: Experience and Lessons Learned

Addressing modern slavery remains a significant challenge for governments and non-government organizations (NGOs). Efforts are hampered by a range of factors, not least the absence of a road map for measuring progress. Central to measures of progress is a baseline against which changes in the scale of modern slavery can be assessed.

The Millennium Development Goals (MDGs) signaled a global commitment to measurement as a driver of progress in international development. This commitment has been renewed through the Sustainable Development Goals (SDGs). In the same way that the MDGs ensured a focus on priorities, elevating ending slavery, trafficking, and forced labor to a mandatory goal of the SDGs will bring much-needed focus and encourage consensus in the field.

As is the case for any crime, measuring modern slavery is complex, involving everything from differing definitions to the hidden nature of the crime. However, these challenges are not unique to slavery. For example, advocates and policymakers faced similar challenges in wanting to respond in an informed, proportionate way to gender-based violence in the 1970s (Kilpatrick, 2004; Krug, et al., 2002).

In the absence of clear crime statistics, researchers turned to random sample population surveys to better capture previously under- or unreported instances of sexual violence and domestic violence, among other relatively hidden crimes (Biderman and Reiss, 1967; Group of Experts on Gender Statistics, 2006). Household surveys are now widely used in that context.

In 2014, Walk Free tested a small number of random sample household surveys to estimate the prevalence of modern slavery at the national level. Although the methodology continues to be refined, early indications are that this approach holds promise for measuring what had been thought to be unmeasurable. A summary of the development of the instrument and remaining challenges follows.

Defining Modern Slavery

Governments and NGOs use multiple terms to describe modern forms of slavery and related concepts, such as human trafficking, forced labor, debt bondage, forced or servile marriage, and the sale and exploitation of children. However, all of these terms essentially refer to situations of exploitation that a person cannot refuse or leave because of threats, violence, coercion, deception, and/or abuse of power. Modern slavery is a useful umbrella term that focuses attention on the commonalities among these concepts (Global Slavery Index).

The Gallup World Poll

Walk Free selected the Gallup World Poll as the survey vehicle because of its global coverage, methodology, and consistency of quality survey delivery. Used by large international organizations such as the World Bank; Organization for Economic Co-operation and Development (OECD); Food and Agriculture Organization (FAO); and International Labor Organization (ILO), the World Poll continually surveys in more than 150 countries, representing more than 99% of the world’s adult population.

In the World Poll, face-to-face or telephone surveys are conducted in more than 140 languages. Countries for the Walk Free Foundation survey were selected only from those where face-to-face interviewing was an option. A detailed description of the World Poll methodology is available online.

The target population for the World Poll is the entire civilian, non-institutionalized population, aged 15 and older. Samples are probability-based and nationally representative, with the only exception of areas that are scarcely populated or present a threat to the safety of interviewers. The questionnaire is translated into the major languages of each country, and in-depth training is conducted with field staff, who also receive a standardized training manual.

All face-to-face interviews take place at a person’s home. To be eligible, a household has to have its own cooking facilities, which could be anything from a standing stove in the kitchen to a small fire in the courtyard.

Gallup follows European Society for Opinion and Marketing Research (ESOMAR) standards for quality control that includes 30% quality control at minimum. The supervisor accompanies each interviewer for one full interview within the first two days of interviewing. The supervisor accompanies interviewers on a minimum of 5% of subsequent interviews. Interviewers re-contact a minimum of 15% of households to ensure correct execution of random route procedures and household selection. The remainder of the 30% validations can be accomplished through any validation method, including accompaniment, in-person re-contact, or phone re-contacts.

In 2014, Walk Free commissioned Gallup to conduct random sample surveys in seven countries: Brazil, Ethiopia, Indonesia, Nepal, Nigeria, Russia, and Pakistan. These countries were chosen as the pilot sites for several reasons, primarily: First, each is considered a “source” country, with significant populations of returned migrant workers. This increased the likelihood that surveys might identify returned migrant workers who had been subjected to modern slavery in their labor or other migration overseas.

Second, the Gallup methodology in each of these countries is delivered through face-to-face surveys, as opposed to telephone interviews. Face-to-face interviews provide better opportunities for interviewers to read nonverbal cues, observe where clarification is required, and build an appropriate level of rapport with respondents, which is particularly important given the sensitive nature of the questions for this topic area.

Developing the Instrument

The Walk Free Foundation, in consultation with the GSI Expert Working Group (a convening of anti-slavery and methodology experts who come together to examine and build on the Global Slavery Index methodology) and Gallup, developed an initial set of questions designed to capture a range of scenarios that could be classified as modern slavery. Due to the limited time available, the questions were direct and focused on isolating situations where an individual’s freedom had been restricted to exploit them. Four screening questions were designed to identify cases that fell into two broad categories: unfree labor and forced marriage.

The draft screening questions, before any testing or refinement, were:

1. Have you or anyone in your family ever been tricked into or forced to work and were not allowed to quit that work?

2. Have you or anyone in your family ever had to work for an employer who was owed a large debt and were not allowed to quit that work until the debt was repaid?

3. Have you or anyone in your family ever been offered one kind of work, but then were forced to do something else and not allowed to leave?

4. Have you or anyone in your family ever been forced into an arranged marriage and could not refuse to marry this person?

To partly address the limitations of a household-based sample when the target population is largely hidden, the Walk Free Foundation instrument incorporated a network sampling approach. That is, respondents are asked to respond about their own experiences or those of members of their networks. These surveys use “family” rather than “household” as the reference group, to increase the likelihood of identifying victims in a random sample survey.

For network sampling to provide an unbiased estimate, the size of the network has to be determined. To this end, the respondent is asked about his/her parents, spouse, siblings, children, aunts, uncles, and cousins.

Cognitive testing

The draft questions were cognitive-tested in six countries, including interviews with 58 adults from the general population with no experience of modern slavery and 36 adults who had been in modern slavery in Indonesia, Nigeria, Pakistan, and Ethiopia. The results of cognitive testing were generally positive: Respondents generally understood the questions, recalled the information being sought, wanted to provide the information, and could respond in the format required.

However, some specific issues that required amendments were highlighted through the testing.

  • The original forced-labor question was confusing due to the two clauses: (1) “been tricked into or forced to work” and (2) “not allowed to quit that work.” Some respondents focused on only one of the two clauses. The question was simplified and a follow-up question on coercion was added.
  • Cultural differences were apparent in the forced marriage question. For testing purposes, the language of both arranged and forced marriage was used. In countries where arranged marriages are common, the difference between the two was clearly understood. Conversely, in countries where arranged marriages were neither the norm nor a rare exception (in this case, Ethiopia, Indonesia, and Nigeria), there were significant difficulties in understanding the difference between arranged and forced marriages.

We simplified the question to “forced to marry” and added questions on consent and age at time of marriage.

  • Reports about extended family were unreliable. It was difficult for some respondents to accurately respond to questions about the size of their network, and to provide the required information about extended family members, including uncles and cousins. As a result, “family” was redefined to include self plus living family members, including spouse/partner, children, siblings, and parents. Explicit definitions for each type of relative were included to ensure clarity.
  • We made a few additional refinements to the survey instrument in 2015. One of the more significant changes was to ask respondents to explain their experience in their own words. This has provided vivid accounts of slavery, as well as allowed the review and, where necessary, reclassification of borderline cases.
The final screening questions

Following the round of amendments that cognitive testing revealed were necessary, the final questions were:

1. Have you or has anyone in your immediate family ever been forced to work by an employer or a recruiter?

2. Have you or has anyone in your immediate family ever been forced to work to repay a debt with an employer or recruiter and were not allowed to leave?

3. Have you or has anyone in your immediate family ever been offered one kind of work, but then were forced to do something else and not allowed to leave?

4. Have you or has anyone in your immediate family ever been forced to marry?

We added an additional screening question in Mauritania to ensure traditional forms of slavery were also captured:

5. Have you or has anyone in your immediate family ever been forced to work for a master as a slave?

Follow-up questions

When respondents answer “yes” to any of the screening questions about unfree work or forced marriage, on behalf of themselves or an immediate family member, they are then asked a series of follow-up questions to capture more information about:

  • the experience, including when and where it occurred,
  • the ways in which the victims were kept from leaving that work,
  • the type of work the victims were forced to do, and
  • in the case of forced marriage, whether they consented to the marriage.

A copy of the full instrument is available on the Global Slavery Index website.

Although the screening questions were designed to capture as many cases of modern slavery as possible within the sample, it was important to collect additional information about the experience to reduce the likelihood of false positives and help make decisions about appropriate cut-off points for estimation. For the purpose of estimation, eligible cases are limited to people who, either in relation to their own experience or on behalf of a family member, answered yes to any of the forced-labor questions and reported having been coerced, or answered “yes” to the forced-marriage question and reported that they did not consent to the marriage.

Furthermore, the timeframe for estimates derived from these surveys is limited to cases occurring in the five years preceding the survey, although information on all experiences is collected.

Results and Lessons Learned

A total of 28,206 respondents were interviewed in the 25 survey countries. Surveys in every country identified cases of modern slavery. Given that interviews were undertaken with approximately 1,000 respondents in an entire country (with larger samples in Russia, N=2,000, and India, N=3,000), and without any effort to target hot spots or vulnerable populations, this result is quite extraordinary. A fuller report on population coverage and sampling design can be downloaded from the Global Slavery Index website.

Prevalence Estimates From Surveys

Based on the random sample surveys conducted in 2014 and 2015, the proportion of the population estimated to be in modern slavery in each country was calculated using the counting rules set out above. These proportions are shown in Table 1. Indonesia, for example, has a figure of 0.286%. This means that approximately 0.3% of the Indonesian population is estimated to be in modern slavery.

Table 1—Estimated Number of People in Modern Slavery in the General Population Based on WFF Surveys and GSI Adjustments Available in Online Methodology Paper

Limitations of the Methodology

It’s important to acknowledge that random sample surveys are not the best method to apply in every country. While nationally representative, random sample surveys are ideal for estimation, victims of modern slavery are unlikely to be identified in samples based on a census framework in countries where the suspected number of people in modern slavery is very small, or individuals too hidden. They also are unlikely to be picked up through the amended network sampling frame. This also suggests that our estimates are likely to be conservative in countries where surveying is possible.

Representative sample surveys provide an adequate estimation methodology in countries where we know there is a significant problem, and when we know that victims of modern slavery are likely to be identified in a random sample of the national population. Nonetheless, surveys do involve asking people a series of sensitive questions, so it is likely that certain information will be disclosed more readily than other information (particularly in relation to taboo subjects such as engagement in the sex industry or experience with sexual violence).

Given the very small number of respondents who reported sexual exploitation as the type of work they were forced to engage in, we know other approaches are required to obtain more-accurate information about this form of forced labor. Similarly, the target sample is the entire civilian, non-institutionalized population, aged 15 and older, which limits the amount of information obtained about the extent of slavery among children. Walk Free and Gallup have recently tested new questions and alternative formats to elicit more information about children, and early indications are promising.

Another potential limitation we substantiated during the cognitive testing phase and the subsequent quantitative data collection is the information asymmetry inherent to the network sampling approach. Generally, the discovery rate of modern slavery cases tends to be higher when respondents reflect on their own experiences than on the experiences of their immediate family networks. We are continuing to refine the survey methodology to address these issues and are testing different ways of asking the questions. Survey data must be considered alongside administrative data and “gray” literature that provides critical context, and can help identify gaps in survey results.

A further question remains regarding the standard World Poll sample sizes and whether they meet minimum requirements, even after boosting through the network sampling frame. In the first round, calculations suggested that an average sample size of 1,600 was required. We will re-run several surveys in 2017 to assess the outcome of the larger sample.

Next Steps

As part of our ongoing work to improve the accuracy of the prevalence estimates, the Walk Free Foundation and the International Labor Organization commissioned Gallup to conduct an additional 28 random sample surveys in 2016. The countries surveyed to date represent 50% of the world’s population (see Figure 1) and form the most extensive survey program on modern slavery ever undertaken.

Figure 1. Countries surveyed from 2014 to 2016 (in red).

All organizations working to end modern slavery—including human trafficking, forced labor, or slavery—desperately need stronger and more-accurate data to better inform and target prevention and victim support programs. Taking into account other sampling methods and different contexts, random sample surveys—undertaken at scale—represent a significant leap forward in measuring this crime. Doing so adds substantially to our collective understanding of modern slavery and efforts to measure progress in abolishing it.

Further Reading

Biderman, A., and Reiss, A.J. 1967. On Exploring the ‘Dark Figure’ of Crime, Annals of the American Academy of Political and Social Sciences 1(114), p. 375.

Gallup World Poll, Client Services, accessed April 20, 2015.

Group of Experts on Gender Statistics. 2006. Towards international standards for data collection and statistics on violence against women. Conference of European Statisticians, September 11–12, Geneva, Switzerland.

Kilpatrick, D.G. 2004. What is Violence Against Women? Defining and Measuring the Problem. Journal of Interpersonal Violence 19(2), pp. 1,209–1,234.

Krug, E.G., Mercy, J.A., Fahlberg, L.L., and Zwi, A.B. 2002, The world report on violence and health. The Lancet 360.

Minderoo Foundation. 2016. The Global Slavery Index 2016. Walk Free Foundation.

About the Authors

Jacqueline Joudo Larsen is a criminologist and senior research manager at the Walk Free Foundation and author of the Global Slavery Index. She oversees the survey research program and previously led research on human trafficking, international students, and violent extremism at the Australian Institute of Criminology.

Pablo Diego-Rosell, PhD, is a senior consultant at Gallup and has led quantitative and qualitative research studies on child labor, forced labor, and human trafficking on five continents and in more than 50 countries. Most recently, he facilitated the development of new methods to estimate modern slavery as a technical advisor to Walk Free and ILO.

Back to Top

Tagged as: , , , , , , ,

Leave a Response

Please note: comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.