Kiryandongo refugee settlement

Uganda is one of the world’s leading refugee-hosting countries, with 1.5 million refugees and asylum-seekers across 14 settlements. Uganda has progressive policies; these permit refugees to move freely outside the settlements, work, start businesses, engage in farming activities on rented land and access public services including education and healthcare.18–21

We focus on households registered in the Kiryandongo refugee settlement, situated next to the main Kampala-Gulu highway and the district’s commercial centre, Bweyale. Unlike some settlements, and not counting the COVID-19 lockdown, movement between the settlement and town is short (between a 10 and 40 min motorcycle ride) and with no physical barriers.

Kiryandongo is co-managed by the Office of the Prime Minister and the United Nations High Commissioner for Refugees (UNHCR) providing shelter, land and support to ~10 000 households. Established on cleared ranch land in the 1970s, Kiryandongo now hosts mostly (99%) South Sudanese refugees facing protracted displacement. Most current settlement inhabitants arrived in or after 2014, which marked South Sudanese independence and the ensuing civil war. On arrival, UNHCR registered household heads and their families—broadly defined—and issued an attestation card; households use this to access settlement services.22 Each household received around two acres of land and a few materials, such as a tarp, to build a shelter.

Most households are Nuer, Dinka and Acholi/Luo, though the settlement houses over 10 ethnicities. Many households live in makeshift or traditional huts; some have built semipermanent houses using half-burnt bricks and iron sheets. Permanent houses, using fully burned bricks and cement, are officially forbidden.

Health system and access

For health infrastructure, Bweyale has two government health centres, which serve both refugee and host communities. In the settlement are two large clinics, run jointly by the UNHCR and the Government of Uganda (GoU). Non-Governmental Organisations, with UNHCR support, provide healthcare and psychosocial support. While Uganda operates an integrated refugee policy that enables refugees and Ugandans to access the same health services, there are barriers to accessing quality care in the Ugandan public health system, such as stockouts, negative perceptions of staff attitudes, and long waiting times.23–26 In areas around Kiryandongo refugee settlement, there are several, mostly Ugandan-run, private clinics and drug retail shops. In part, private facilities serve as a (costly) buffer against the well-documented stockouts of supplies, diagnostics and tests in the Ugandan public health system.27 28

COVID-19 and lockdown

On 30 March 2020, Uganda entered a nationwide lockdown to prevent COVID-19 spread, restricting almost all movement and commerce in the country and across international borders. In June 2020, GoU, UNHCR and some NGOs started issuing free masks and small bottles of sanitisers to people in the settlement. Masks were also available for sale, ranging from UGX1000 to UGX2500 (~US$1 to US$2 PPP) (We use the World Bank’s Purchasing Power Parity (PPP) conversion factor for private consumption for the official exchange rate per international US$ in 2020, which was 1267.2. The price level ratio of PPP conversion factor (GDP) to Ugandan Shilling market exchange rate for 2020 was 0.356 (World Bank, 2021)) per mask and UGX20 000 (~US$16 PPP) per 1 L bottle of sanitiser, with smaller 100 mL bottles priced around UGX8000 (~US$6 PPP).

By October 2020—during our last round of phone surveying—GoU had announced a phased reopening of schools though schools did not open until January 2022. Additionally, non-essential businesses could operate and religious gatherings could take place under daylight curfew conditions.

Aid and aid cuts

The World Food Programme (WFP) provides households with monthly food or cash rations (their choice) for each registered household member. These constitute roughly one-quarter (23%) of baseline consumption. In April 2020, WFP hastened already-planned aid cuts as COVID-19 stretched its budget.5 WFP trimmed rations by 30%, to around US$17 PPP per registered household member. Further, as a COVID-19 precaution, WFP rescheduled ration distribution to take place every 2 months rather than monthly.29

Targeting

As per its most-recent operating model, GD aimed to saturate the settlement with transfers, meaning that each household would receive a transfer. GD saturates a setting when no clear, easily explainable rationing criterion exists.

GD planned to deliver transfers to UNHCR-registered household heads. In the Kiryandongo settlement, and in our sample, about three-quarters of registered household heads are female. Official headship can mask a variety of de facto living and power arrangements.

Amount

GD aimed to provide a US$ 1000 (~3000 PPP) UCT to all households, in line with GD’s standard practice. This represents 3 months of household consumption as per average baseline results and is equivalent to nineteen months of monthly household consumption support from WFP (assuming an average household size of nine members). While the US$1000 transfer is one-off (not recurring), GD disbursed in three instalments—a smaller first instalment of UGX520 000 (~US$418 PPP) and two larger instalments of UGX1.605 million (~US$1289 PPP) each. These amounts cover mobile-agent withdrawal fees. Recipients could opt to receive a basic mobile phone from GD, in which case GD would deduct UGX68 000 (~US$55 PPP) from the final instalment to cover the cost. Around 90% of treated households chose a phone.

Transfers to our treatment group began in February 2020, and continued until August 2020. We summarise the transfer timing by cohort in table 1. Cohort 1 recsueived transfers earlier on average, starting in February 2020; cohort 2 households started receiving their first instalment in March 2020. During the first few months of the COVID-19 lockdown (April–June 2020), households in cohort 1 had received more transfers on average than households in cohort 2. By end-April, GD had distributed 39% of transfers to cohort 1 but only 18% to cohort 2. By July, operational processes allowed cohort 2 to catch up in transfer receipt and by August 2020, cohort 1 and cohort 2 had received, on average, ~80% and ~83% of their transfers, respectively.

Sensitisation

For refugees, their first contact with GD came in community sensitisation meetings (barazas). GD staff held at least one baraza in each settlement administrative cluster. Through barazas, GD staff, in their dark green vests, became recognisable and well known in the settlement, building trust. Staff explained the transfer and how the roll out would work. They further shared security advice to prevent fraud or theft, and warned against rumours. This meant that most people in the settlement were likely aware of the transfer, regardless of cohort assignment. GD does not impose conditions on how households can use their transfer, they suggest recipients could invest in farming activities, buy livestock, build houses, enrol children in school, buy clothes, start small businesses, buy food and seek medication among other things.

Registration and audit

To enrol a household, GD staff register each household with a mobile telecommunication company and provide them with a free SIM card. GD staff also provide basic digital and financial literacy training, covering phone use and safe-keeping for SIM cards and personal details. During later audit visits, GD staff verified details collected during registration and reiterated digital literacy and safety points.

Distribution

GD aimed to follow a 1-month cadence between enrolment and disbursement of each of the three instalments. Logistical factors disrupted this plan; we provide more information on the actual timing of the transfers for our study population in a later section.

Follow-up, mediation and ongoing support

After each instalment, GD calls customers to ensure they got the money and everything is in order. This includes addressing household conflicts or possible fraud, if applicable. In case of conflicts, GD pauses the transfer until they have investigated. The investigation may lead GD to take mitigative action, like splitting the transfer or stopping it completely in extreme cases.

GD runs a toll-free hotline in several languages to address any questions or concerns of recipients. During the COVID-19 pandemic, GD shared information about the virus and provided referrals to other organisations or service providers if requested by its customers.

During COVID-19, with the increased vulnerability of the study population, there are legitimate concerns that sticking to the planned intervention schedule would have amounted to withholding critical support from the participants during a crisis. We agree that participants’ welfare would have been most likely negatively affected and speeding up the transfer would have been helpful. However, it is important to note a few key points. First of all, even in normal circumstances, GD’s speed of enroling households was limited by manpower and logistics, which was the original reason for distributing the transfers over 24 months. With new COVID-19 preventive measures including lockdowns, these operational constraints were magnified as GD transitioned to remote operations, causing delays in the speed of transfers. Overall, operational constraints simply prevented the potential strategy of speeding up distribution once COVID-19 hit. Second and most importantly, was the suspension of GD’s licence to operate in Uganda from early September 2020. This suspension by the GoU implied that all operations, including a new COVID-19 relief package to close to 200 000 households, were halted.30 The licence was only reinstated in November 2021 with a ‘second chance’ from the president.31 These were conditions beyond which GD would have had any control. Therefore, even had GD been able to overcome operational constraints to speed up transfers during the pandemic, the suspension would have eventually prevented this.

Finally, to ensure more data security, we encrypted and uploaded all data to a secure central database. We stored back-ups on password-protected computers and folders to ensure data confidentiality. Only the research management team had access to the encrypted raw data. Finally, we provide a structured appendix32 and a reflexivity statement33 both in online supplemental file 1.

Quantitative sample and data collection

At baseline, we sought to reach 840 households assigned to cohorts 1–2 (treatment) and a random sample of 840 households from cohorts 17 to 20 (control). (These cohorts were chosen to balance a few factors. For treatment, we chose the earliest cohorts to have the longest exposure time before our endline survey. For control, we wanted to ensure that all participants in the study would eventually receive transfers, which was not guaranteed at the outset due to funding limitations, which eliminated the last cohorts. And we wanted to guard against anticipation effects, so we did not want to select cohorts that would receive transfer right after our endline survey around month 12. Therefore, we chose cohorts 17–20 for control to balance these concerns). During data collection, however, we discovered that a significant number of UNHCR-defined households used during randomisation did not always match the realities of household composition in the settlement. This was problematic for the study as some of our control households lived with family members who would receive a cash transfer before the follow-up survey. Additionally, finding households using the UNHCR household list was challenging, as information was often outdated. Therefore, we surveyed households where all members were exclusively listed in cohorts 1 and 2 and 17–20 with no overlaps. Our final sample comprised 1264 households.

At baseline, we conducted in-person data collection from end-September to end-November 2019. Surveys were translated and administered in Acholi, Dinka, English, Juba-Arabic, Nuer and Bari languages. We collected data using Survey CTO on tablets; the average survey length was about 2 hours. Most enumerators were South Sudanese refugees living in or near the settlement.

Between July and October 2020, we conducted three rounds of rapid phone surveys with a subset of the baseline sample. The phone surveys took about 10 days per round with interviews lasting, on average, 30 min. A group of baseline enumerators also conducted the phone surveys. Online supplemental figure S1 shows the timelines of events and data collection from initial lotterising to the third phone survey that acts as our endline.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

The Consolidated Standards of Reporting Trials (CONSORT) flow diagram for sample selection. PSN, Persons with Specific Needs.

Out of 1264 baseline households, we had active phone numbers for 1202 households. Based on experience, we predicted a response rate of 50%–60%, allowing for a meaningful minimum-detectable effect size (0.23, assuming alpha of 0.05% and 80% power). To improve response rates, we implemented a callback protocol whereby enumerators made multiple call attempts across different days. We offered respondents a small compensation for participating (approximately US$ 3 PPP mobile money transfer).

During each round, we reached and interviewed between 61%–65% of the treatment group and 57%–59% of the control group. Overall, 74% of eligible households (baseline households with a phone number) responded to at least one of the three survey rounds. In each survey round over all, our response rates were higher than other phone surveys in similar contexts during crises.35 The Consolidated Standards of Reporting Trials diagram in figure 1 shows our final sample.

We conducted balance checks between our treatment and control groups of key baseline variables, including baseline values of our regression outcomes (when available) and pre-specified covariates (see online supplemental table S1). Most variables were balanced but there were significant differences in some ethnicity categories. We believe this stems from the deviations from pure randomisation, mentioned above. In table 1, we show the development of the sample and phone survey response rates across all three rounds.

In online supplemental table S2, we explore differences in household characteristics between those who responded to our phone survey and those who did not. We find the responders are more educated, have spent a longer time in the settlement and are less likely to be in the Nuer ethnic group.

Qualitative sample and data collection

We randomly selected 32 refugee households for our qualitative sample, stratified on ethnicity and gender (34% of our original sample were male and 66% female respondents). Absent time to ‘soak and poke’ towards thematic saturation, we relied on random selection to determine our sample.36 A 32 household sample exceeds the saturation point in other studies.37 38 In our original plan, enacted in February and March 2020, each respondent was interviewed in-person once a month. However, when the COVID-19 pandemic reached Uganda and government authorities enacted lockdown measures, our team switched to remote data collection. This resulted in a reduction of our sample size to 17 respondents. The reduction was caused by multiple factors, including some respondents not owning a phone or living in areas with no network. We also could not rely on our interpreters during phone interviews, so we had to drop some households that could only communicate in a language not spoken by our enumerators. The reduced sample included 47% male and 53% female respondents. It also distorted ethnic representation in our sample; most ethnicities remained represented although not proportionally). Because of connectivity and attention issues when going remote, we decided to speak to respondents more frequently but for shorter durations. (Frequency increased from once to thrice per month; duration reduced from ~90 min to ~30 min per interview). Between January and September 2020, we interviewed each of these respondents up to 15 times.

Two interviewers carried out all semistructured interviews. They each spoke with the same set of respondents over time, building rapport. Both interviewers are male enumerators from baseline that we trained and coached in qualitative interviewing, note-taking and transcription. One interviewer is half-Ugandan, half-Sudanese and lives in Bweyale; the other is Ugandan. Interviews took place in a language comfortable for both respondent and interviewer and were audiorecorded.

Throughout our interviews, we covered multiple topics. In each interview, we asked about general well-being and life updates before going in-depth on topics that we regularly revisited (such as household decision-making and market behaviour) and standalone topics (such as the health and education system). We provide a more detailed overview of topics covered in online appendix supplemental table S3. (All data collection instruments for both our quantitative and qualitative work can be found at https://www.idinsight.org/project/unconditional-cash-transfers-in-kiryandongo-refugee-settlement-uganda).

At the beginning of the project, we had high acceptance into our small-n panel, including equal acceptance across the sexes. There was also similar heterogeneity across men and women in terms of the quality and depth of responses given, which we interpret as being a function of how we built our sample (through random selection) and provides reassurance that having male interviewers did not have differential effects on response quality. When we transitioned to telephonic interviews, we had a harder time reaching female respondents. Among the 17 respondents we reached at least once during the telephonic interviews, men responded to 69% and women to 63% of all call attempts. We interpret this as more likely related to competing demands on women’s time rather than the influence of having male interviewers.

Quantitative impact evaluation

Our analysis is a simple ordinary least squares regression of each outcome variable on treatment status, that is, the cohorts’ intended transfer receipt. As specified in the pre-analysis plan for our original study (https://www.socialscienceregistry.org/trials/6271/history/73771) on medium-term outcomes, our regressions control for various baseline characteristics including sex of household head, household size, time in the settlement and ethnic group. If we measured an outcome variable at baseline, we include the baseline value of this variable as a control. The analysis is ‘intention to treat’, in that we assign a household to the treatment group regardless of whether they actually received the transfer.

In our original preanalysis plan, we include household consumption, a food security index and psychological well-being as primary indicators. As these were included in our phone surveys, we treat them as primary indicators (Note that we only collected the food portion of household consumption). Our food security index combines several questions about hunger and meals, as used by other researchers of cash transfers.8 Our psychological well-being index combines three popular well-being measures: the World Values Survey (WVS) Happiness Questionnaire, WVS Life Satisfaction Questionnaire39 and Center for Epidemiological Studies (CES) Depression Scale. We also add an index of COVID-19-related practices as a primary indicator, as determining the impact of cash on COVID-19-related practices motivated the phone surveys. We created the index by asking about COVID-19 practices—mask-wearing, hand washing, staying home, social distancing—and combined them in an inverse-covariate-weighted index.40 We also report results from each index component separately, and related secondary outcomes. We account for the possibility of false detection due to multiple outcomes by using the Benjamin41 methods of sharpened q values.

There is considerable attrition in each phone survey round from the baseline sample. Since the non-response is not correlated with treatment, we believe our results are unbiased for the sample of respondents. As a robustness check, we include results that incorporate inverse probability weighting to correct for non-response.

Qualitative analysis

We conducted a thematic content analysis in Excel, accounting for the longitudinal nature of some of our questions.42 Before beginning analysis, we developed a deductive codebook; we added codes inductively, in consultation with one another, when unanticipated responses arose or to provide further nuance. We include direct quotes from our analysis when they represent a commonly held view.