Article Text
PDF
Abstract
Background Multisectoral approaches are thought necessary to prevent HIV among adolescents. We examined whether an economic strengthening (ES) and an HIV-prevention education intervention improved outcomes when combined versus separately.
Methods We conducted a full-factorial randomised controlled study to randomise participants into all possible intervention groups based on the two interventions: economic strengthening only (ES-only), HIV-prevention only (HIV-only), both interventions combined (ES+HIV) and no intervention (control). We measured sexually transmitted infections (STIs), self-reported economic and sexual behaviours/knowledge, and pregnancy at a pre-intervention and two post-intervention assessments. Eligible participants were adolescents 14 to 17 years old from a programme supporting vulnerable families in Gauteng Province, South Africa. We estimated intervention effects using repeated measures, generalised linear mixed models.
Results A total of 1773 adolescents participated (57% female). ES+HIV adolescents had the lowest STI prevalence at first endline; however, the comparison with the control was not significant (OR 0.62, 95% CI 0.27 to 1.41). ES-only or HIV-only groups were not significantly better than the control on STI prevalence (OR 1.53, 95% CI 0.73 to 3.20 and OR 1.47, 95% CI 0.69 to 3.12, respectively). STI prevalence became more similar among the groups at second endline.
ES-only adolescents were more likely to participate in savings groups (p=0.004) and plan to save for education (p=0.001) versus the control. ES+HIV adolescents were more likely to plan to save for education versus the control (p=0.001) and HIV-only groups (p=0.002) but did not differ significantly from the ES-only group (p=0.803). The ES+HIV intervention’s effect on HIV knowledge was significant compared with the control (p=0.03) and ES-only groups (p<0.001), but not when compared with the HIV-only group (p=0.091). Effects on pregnancy, sexual behaviours or other economic behaviours were not significant.
Conclusions We could not confirm the ES and HIV interventions, separately or combined, were effective to reduce STI prevalence. Evaluations of multicomponent interventions should use full-factorial designs to fully assess effects.
Trial registration number NCT02888678
- prevention strategies
- HIV
- randomised control trial
- child health
- public health
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key questions
What is already known?
Adolescents are at high risk for HIV infection, especially in South Africa.
Structural factors such as poverty increase risk for HIV acquisition; economic strengthening (ES) is a strategy that could address poverty; however, limited rigorous evidence exists on ES interventions for HIV-prevention outcomes.
Combination prevention is a recommended strategy, but there are few rigorous studies investigating the separate and combined effects of multisectoral interventions for HIV prevention.
What are the new findings?
The ES and HIV-prevention education interventions we assessed did not significantly affect sexually transmitted infection prevalence or sexual behaviours in this context, although adolescents receiving the ES intervention were more likely to participate in savings groups and plan to save for their education.
What do the new findings imply?
While combination prevention is endorsed by many, we could not confirm this multisectoral intervention led to better HIV-prevention outcomes. Nonetheless, these types of interventions should continue to be examined using full-factorial designs to adequately assess the effect of combining multiple interventions.
Introduction
Globally, an estimated one-third of all new HIV infections occurs among adolescents ages 15–24, highlighting the importance of an HIV response targeting this population.1 In South Africa, youth ages 15–24, especially black youth and girls, are disproportionately affected by the HIV epidemic.2 Many factors increase adolescents’ vulnerability to HIV. Adolescents who are poor, experience gender-based violence and who have lost a parent are at increased risk of acquiring HIV.3–8 Adolescents who live in these contexts are more likely to engage in behaviours associated with HIV, including early sexual debut, age-disparate partnerships, multiple partners and low condom use.4 8–12 To reduce adolescents’ risk of becoming infected with HIV, global guidance suggests intervening on multiple factors through combination prevention.13–15
HIV-prevention interventions that address the social context in which HIV risk behaviours occur and have been shown to build young people’s communication and negotiation skills, knowledge and risk awareness.16–22 Economic strengthening (ES) interventions, including individual and group savings, financial education, cash transfers, income generation and vocational training, have the potential to help address the structural factors such as poverty that increase vulnerability to HIV, particularly among girls23–26; however, there is a need for more rigorous evidence on ES interventions that address HIV-prevention outcomes, especially with adolescents and using biomarker outcomes.27 28
Another important question, which is the focus of this paper, is whether approaches that couple HIV-prevention and ES interventions yield complementary effects, with potential for a major impact on HIV vulnerability. Several studies have yielded promising results that suggest that combined programmes build skills to improve financial well-being, women’s empowerment and gender equity, and thus reduce vulnerability to HIV.29–32 For example, the SHAZ! project with adolescent girls in Zimbabwe was a randomised controlled trial of a combined intervention package that included life skills, reproductive health services, health education, financial literacy and vocational training, and integrated social support, compared with the control programme of life skills, reproductive health services and health education.32 Adolescents in the combined intervention group reported a decrease in transactional sex over the study period, greater likelihood of using a condom with a current partner over time and reduced food insecurity, and were more likely to have their own income when compared with the control group.29–32 Despite these promising results, most studies have compared combined programmes to a control group, and few studies have been designed to assess the added effect of combined programs beyond single interventions.33 34 Furthermore, synergistic effects in multisectoral integration—that is, whether the integration of two interventions produces a combined effect greater than the sum of the interventions’ separate effects—have not been commonly assessed or discussed in these studies.33
We designed and implemented a full-factorial randomised study to rigorously evaluate an integrated ES and HIV-prevention education intervention that was being implemented with adolescents at high risk of acquiring HIV in South Africa.35 Based on the literature, and consistent with Network-Individual-Resource model for HIV prevention,36 our theory of change proposed that an HIV-prevention intervention that increases mental resources by teaching adolescents about safe sex practices and builds communication and negotiation skills would lead to safer sex practices, lower rates of STIs including HIV, and fewer unintended pregnancies.22 37 Economic interventions can be used to reduce economic vulnerability, a key structural factor contributing to risky sexual behaviours, by increasing tangible resources.38 Components of the ES intervention package that were evaluated in this study included financial capability training and savings, which aimed to address the participants’ economic vulnerability. Such ES interventions may foster greater financial independence, which may reduce the incentive to participate in transactional sex and increase bargaining power in sexual relationships.4 39 We hypothesised that combining an ES intervention with an HIV-prevention intervention would reduce STI prevalence and improve HIV risk behaviours and ES outcomes in the study population, and that the combined intervention would improve outcomes more than the individual interventions.
Methods
Participants and study setting
Participants were eligible for the study if they were ages 14–17 years at the time of enrolment and clients of Future Families (FF), a non-profit organisation rendering services to orphans and vulnerable children (OVC) and people living with and affected by HIV and AIDS. Pregnant and/or HIV-positive adolescents meeting the mentioned criteria were eligible for the study. The research took place in four communities in Gauteng Province, South Africa.
FF staff introduced the study through community meetings or at potential participants’ homes. If the parent/guardian gave written informed consent for their child to participate in the study, study staff then asked adolescents if they would like to learn more about the study, and if they agreed to participate, obtained written informed assent. Parents/guardians and adolescents were consented separately to ensure that adolescents were not pressured to participate in the study.
Trial design
This was a full-factorial randomised controlled trial. From January to July 2016 and after baseline data collection, study staff randomised adolescents into four study groups: ES+HIV (combined economic strengthening and HIV-prevention interventions), ES (economic strengthening intervention only), HIV (HIV-prevention intervention only) or control (no interventions). All participants received FF’s standard services, which included food and education support and linking families with community services. We collected data from participants at three time points: baseline (pre-intervention), endline 1 (shortly after the interventions ended) and endline 2 (approximately 8 months post-intervention).
Randomisation
Adolescents were randomly assigned (1:1:1:1) to each of the study groups in accordance with a computer-generated block randomisation schedule with block sizes of 8 and 16, and stratification by study community. Allocation concealment was achieved with sequentially numbered opaque envelopes. The randomisation schedule and envelopes were prepared by an independent statistician who was not otherwise involved in the study. Neither study staff nor participants were blinded to study treatment groups after randomisation. The statistical team was blinded until key decisions regarding the data and primary analysis were made and the primary analysis results were considered final.
Interventions
The ES intervention—Impumelelo—was developed by YouthSave Kenya and covered the topics of budgeting and saving, education on different savings options, and earning money.40 The HIV-prevention intervention was Vhutshilo 2.0 and covered topics including dealing with loss and grief, decision-making, drugs and alcohol, HIV and other STIs, healthy relationships, communication skills, safer sex and contraception.41 42 Each intervention consisted of 16 in-person sessions with a group of up to 15 youth. Each session lasted approximately 90 min. For the ES+HIV group, the two interventions were delivered sequentially. Participants in the ES+HIV group, therefore, received 32 sessions in total, though the order of the interventions varied due to programme staffing constraints. FF staff implemented the interventions at school or community facilities outside of school hours. FF considered an adolescent to have completed the ES or HIV intervention when s/he had attended at least 14 of 16 sessions.
Data collection
Survey data and urine for STI/pregnancy testing were collected from all participates at three time points (pre-intervention and two post-intervention assessments). Adolescents completed a behavioural survey using audio computer-assisted self-interview (ACASI) on a laptop computer or tablet. The ACASI survey included questions about demographics, self-esteem, gender norms, relationships and sexual activity, knowledge about HIV and STIs, financial knowledge and savings. Data collection sites included mobile computer laboratories, community sites, or temporary data collection tents located in study communities after school and on weekends. Trained research assistants were present to assist with computer/tablet problems but did not assist the participants in completing the survey. Participants could complete the survey in four languages (Sotho, Tswana, Zulu and English) and switch between languages throughout the survey.
After completing the ACASI survey, participants were counselled by a study nurse prior to collecting urine for STI testing (and pregnancy testing for girls). All participants were asked to provide a first-catch urine sample to test for gonorrhoea, trichomoniasis and chlamydia infection, and for girl participants, pregnancy. Participants received instructions on how to collect the urine sample and privately collected the urine themselves (unsupervised collection).
For each assessment, participants received a snack (valued at 25 ZAR/US$1.80) and a gift voucher (150 ZAR/US$10.40) for a local shop selling clothes and stationery. Because transportation was identified as a barrier to participating in the study during baseline, participants received reimbursement for round-trip transportation to/from the data collection site of up to 40 ZAR/US$2.80.
The National Institute for Communicable Diseases conducted STI testing on urine specimens using the Aptima Combo 2 Assay (Hologic, San Diego, USA) for Neisseria gonorrhoeae and Chlamydia trachomatis and using the Aptima Trichomonas vaginalis Assay (Hologic) for Trichomonas vaginalis. Pregnancy testing was also performed on girls’ urine specimens using the Alere hCG dipstick (25 mIU/mL) test (Alere, San Diego, USA) standard human chorionic gonadotropin dipsticks or dip strips.
Participants with positive STI results were notified by trained study nurses, given information and counselling, and referred for free treatment in accordance with local guidelines. They were also advised to notify sexual partners of any positive results. Partners were referred for free treatment if requested by the participant. Participants with positive STI results were retested for STIs again 2 to 3 weeks after completing the treatment regimen to confirm whether the STI was cured, and if not, were referred for further treatment. Participants with a positive pregnancy test were notified as soon as results were available and provided referral letters to healthcare services in accordance with local guidelines.
Data collection and intervention cycles overlapped. Baseline data were collected January–July 2016. Each single intervention (ES or HIV) lasted approximately 4 months and therefore the ES+HIV intervention lasted approximately 8 months. However, due to FF staffing constraints, not all participants received the interventions at the same time. Interventions were conducted in three cycles from April 2016 to April 2017. Endline 1 data collection took place December 2016–August 2017, with endline interviews staggered as participants completed the interventions. Endline 2 data collection took place September 2017–May 2018. To ensure retention throughout the study, we verified and updated participants’ contact information at every assessment.
Patient and public involvement
We developed the study recruitment and data collection procedures in collaboration with FF programme staff. The intervention developers (South African collaborators who work closely with the study population) provided feedback on the survey instrument. The survey instrument was pretested with adolescents of similar ages and communities as the study population to ensure that questions and response options were understandable to them. The survey was revised to incorporate the pretest feedback. The type and amount of participant compensation was determined in consultation with FF programme staff and approved by the ethics committees who reviewed the research study. Finally, in June 2019, study results were presented to participants and their families, local communities in which participants lived, FF, the South African National Department of Health and other stakeholders working with vulnerable youth in South Africa.
Outcomes
The primary outcome was STI prevalence, defined as a positive test result for gonorrhoea, trichomoniasis or chlamydia infection. We selected these non-viral STIs because they are common and treatable and can be tested in urine samples obtained from both males and females. The secondary outcomes (all self-reported, except pregnancy) were (1) pregnancy (based on urine sample for female participants), (2) engaging in protective sexual behaviour (abstinence or a condom used every time over the past 6 months), (3) engaging in transactional sex in the past 6 months, (4) having two or more sexual partners in the past 6 months, (5) HIV knowledge, (6) financial literacy, (7) participation in a savings group, (8) opening a savings account, (9) net change in savings in past year, (10) saving for education, (11) caregiver being primary provider of money to youth for savings and (12) participation in household budgeting. Outcomes were assessed at baseline, endline 1 and endline 2.
We did not exclude adolescents reporting not having had sex for the analysis of any of the outcomes. Sex or sexual intercourse was defined in the survey as any time a male’s penis enters a person’s vagina or anus. However, the survey was designed to skip follow-up questions on sexual behaviour for those adolescents reporting not having had initiated sex, which contributed to the amount of missing data. The only outcome defined for a subgroup was the pregnancy outcome, which was only relevant for females.
Items included in the survey were informed by previous research conducted with adolescent girls living in sub-Saharan countries under the Go Girls! Initiative and the Violence Against Children Surveys conducted by the U.S. Centers for Disease Control and Prevention conducted with boys and girls.43 We also drew from a verbal assessment interview for symptomatic HIV, which uses a list of common AIDS-related symptoms in questions about whether the youth respondent has a parent/caregiver who is sick or has died from AIDS.44
Questions for the HIV knowledge and financial literacy scales were developed with the curricula developers. Example HIV knowledge questions include, “What can a person do to avoid getting HIV?” and “Is it okay to put cling wrap on your penis to prevent pregnancy and HIV if you don’t have a condom?” and for the financial literacy scale examples include, “What are the three most important elements of a budget?” and “Which of the following is a short-term money goal?”. For the HIV knowledge and financial literacy models, we created composite scores using a Rasch model. Assuming a single underlying latent trait, Rasch models are better at combining the items taking the distribution of their responses and discriminatory value into account. Also, this modelling approach was well suited for these constructs given that all items are binary. Each item was coded zero for “incorrect” and one for “correct” so that higher scores mean great knowledge. The scores are the raw (unscaled) scores from the Rasch model fit, as implemented in the R package eRm_0.15-7.45–47 Across both Rasch models, all item outfit and infit mean square statistics were between 0.71 and 1.82 (data not shown), suggesting the models were a good fit to the data and the items were in both cases useful for obtaining a composite score. Rasch models were fit for all time points, but scores were anchored on the baseline values to allow for comparisons over time.
Decisions regarding outcome data were made and documented prior to unblinded analysis.
Sample size and power
The sample size was fixed by programming constraints. FF estimated that their staff would be able to enrol approximately 2000 adolescents. With this sample size, we estimated having 80% power to detect a minimum decrease of 7.8% (ie, from 13% to 5.2%) in the prevalence of any one of the three STIs for a given time point. We assumed a baseline STI prevalence of 13% based on a study of reproductive age women in Kwazulu-Natal, South Africa.48 Based on a study conducted with a youth population similar to ours, we considered a 5% loss-to-follow-up rate in our calculations and assumed 45% of the youth in our study would be sexually active.34 As described later, we decided to include all participants in the primary outcome analysis instead of only those who were sexually active because participants’ self-report of sexually activity was found to be unreliable. This decision was made during the blinded review of the data. We expected that power would be slightly higher when using the entire sample, even after adjusting the effect size to the lower STI prevalence.
Statistical analyses
The primary analysis was conducted under the intent-to-treat (ITT) approach, which analyses outcome data according to the intervention as randomly assigned. We used a repeated measures generalised linear mixed model to estimate intervention effects. The measures at endline 1 and endline 2 were considered as repeated measures, while the measure at baseline was used as a control variable in the models as it was a pre-randomisation measurement. Random effects in the mixed model accounts for the correlation between the repeated measurements within participants in the estimation of standard errors (SEs). We used a logit link for dichotomous outcomes and an identity link for continuous outcomes, as appropriate. We estimated the primary effects of interest for comparisons at endline 1. Comparisons at endline 2 were used to explore trends and sustainability of the effects. Focusing the test for the primary evaluation hypothesis on one time point helped to limit type I error inflation. Effects were provided as odds ratios (ORs) or mean differences with 95% confidence intervals (CIs) for dichotomous and continuous outcomes, respectively. All tests were assessed at the 5% significance level for two-sided comparisons. Analyses were performed using SAS V.9.4.49
Strategy for interpreting intervention effects
For the interpretation of the intervention effects, we first considered whether the ES+HIV group was better than the control. Only then did we progress to analysing whether the ES+HIV group was better than each of the single groups. According to this strategy, if the combined ES+HIV group was better than both the single groups and the control group, we would conclude that the integrated intervention was beneficial beyond what could be achieved with single interventions, which was our primary hypothesis of interest. Next, if the ES+HIV group was better than both the single interventions and the control, we would then also test for synergy—whether its effect was larger than the sum of the effects that could be achieved with the single interventions. In contrast, if the data did not support our hypothesis, we would conclude that promoting the integrated intervention may not be warranted. This analysis strategy provided some protection for multiplicity when testing for the effectiveness of the integrated intervention, as we required that certain conditions be met to determine its effectiveness instead of relying on all possible comparisons.
Ideally, we would want combined programmes to have synergistic effects, but this is a higher bar to meet, requiring larger sample sizes to detect. We posited that determining that the combined intervention is better than single interventions is enough to conclude the combined intervention is worthwhile, barring cost and implementation considerations.
Sensitivity analyses
For sensitivity analyses, we used attendance data as recorded by FF. Attendance data were collected on paper forms and entered into a database using REDCap.50 51 Participants were considered to have received an intervention if they attended a minimum of 14 sessions of either of the two interventions, which was defined by FF as the minimum effective exposure. Participants were considered to have received the combined intervention if they attended a minimum of 14 sessions of each intervention. Participants who attended zero sessions were considered to have received no intervention. In these analyses, we excluded those who attended between 1 and 13 sessions of any intervention to focus only on those receiving what we considered the minimum effective intervention dose or zero intervention. The comparative analysis among study groups described above was then repeated using this subset of participants.
Exploratory subgroup analyses
We ran additional models to explore if the interventions had effects on STI prevalence at endline 1 for specific subgroups within the ITT population. Subgroup analysis was run for many groups, including adolescents who
Had an STI at baseline.
Reported ever having had sex at baseline.
Were new to FF.
Were double orphans.
Were HIV affected.
Receive a household grant.
Reported saving money in the past year at baseline.
Reported participation in saving group at baseline.
Reported that their caregiver provided money for savings at baseline.
We also explored if there were intervention effects by community, age group (14–15 vs 16–17) and gender.
All potential confounding factors were included in the repeated measures generalised mixed model as well as their interaction with the intervention group. A backward selection process was carried out to select only significant interaction terms and confounding factors. Then a stratified analysis based on the identified interactions were carried out and effects at endline one were estimated by subgroup.
Results
A total of 1773 participants were randomised into the four groups (ie, ES+HIV, ES, HIV and control) (figure 1). Of the 1773 participants enrolled in the study, three did not complete the ACASI survey at baseline due to a programming error that caused the ACASI program to stop early. Three participants died during the study for reasons unrelated to their participation in the interventions or the trial. Retention in the study was high, with 88% of participants retained at endline 1% and 86% retained at endline 2. Our analysis population, consisting of participants who provided data for at least one endline, was composed of 93% of the youth randomised at baseline.
Participant flow diagram. ITT, intent to treat.
Baseline characteristics
At baseline, participants’ mean age was 15.4 years (SD 1.1) (table 1). Slightly more than half of the participants (57%) were girls. Ninety-six per cent of participants were currently in school or had completed 12th grade. Eighty-three per cent of participants lived in households that receive household social grants for financially disadvantaged families. Twenty-eight per cent of participants were single orphans (one parent was deceased) and 6% were dual orphans (both parents were deceased). Thirty-two per cent of the participants reported that their parent/caregiver was currently sick with or had died from HIV/AIDS.
Baseline characteristics
Among participants, 35% reported ever having had sex. At baseline, 26% of participants had had sex within the 6 months preceding the baseline survey. Among sexually active youth, the mean age of first sex was 13.8 years. Seventeen per cent of all participants reported ever engaging in transactional sex (having sex in exchange for money or other types of gifts). The most common type of gift exchanged for sex was money, followed by mobile airtime, clothes and food (data not shown).
Primary and secondary outcomes at each time point are presented in table 2. The baseline prevalence of any STI was 8.1%, and the most common STI was chlamydia. Two per cent of participants did not provide urine for testing at baseline. Thirteen per cent of females compared with 1% of males tested positive for any STI (data not shown). Among all girls, 2% were pregnant at baseline.
Primary and secondary outcomes at each time point by intervention group
Ninety per cent of participants reported engaging in protective sexual behaviour (abstaining from having sex or consistently using condoms) at baseline. Among all youth, 34% reported refusing sex in the 6 months preceding baseline. The most common reasons reported for refusing sex were fear of HIV/AIDS, followed by fear of pregnancy (data not shown). Over 60% of sexually active youth reported consistent condom use (using a condom every time they had sex) in the previous 6 months.
Because our outcomes included prevalence of STIs and self-reported sexual behaviours, we were interested in comparing participants’ self-report of ever having sex with whether they tested positive for an STI at baseline to check the reliability of the self-reported measure. Importantly, 32% of youth who tested positive for an STI at baseline self-reported that they had never had sex (data not shown).
Outcomes
Table 2 presents the primary and secondary outcomes at each time point by intervention group according to ITT and table 3 presents the estimates of effects from the models. STI prevalence increased over time in all groups, from 8.1% at baseline to 11.0% at endline 2. At endline 2, females still had higher rates of STIs compared with males (17% vs 3%; data not shown). Pregnancy also increased over time, with 2.3% of girls testing positive for pregnancy at baseline and 5.6% testing positive for pregnancy at endline 2. The percentage of adolescents engaging in protective sexual behaviour in the previous 6 months decreased from 89.9% at baseline to 80.7% at endline 2. Several secondary outcomes increased over time: adolescents engaging in transactional sex in the past 6 months (10.0% to 13.0%), adolescents having two or more sexual partners in past 6 months (12.3% to 22.5%), and adolescents who opened a bank savings account in the past year (13.9% to 18.5%). The other secondary outcomes showed little or no change over time: HIV knowledge, mean percentage correct (74.5% at endline 1); financial knowledge, mean percentage correct (52.0% at endline 1); adolescents who participated in savings group (31.2% at endline 1); mean net savings in past year (425.5 ZAR at endline 1); adolescents planning to use savings for education expenses (31.8% at endline 1); and adolescents participating in household budgeting (45.6% at endline 1).
Effect estimates for primary and secondary outcomes at endline 1
The estimates of effect and hypothesis test results in table 3 should be interpreted following our pre-specified analysis strategy in three steps. First, we looked at the comparisons between each intervention group and the control group, which are always interpretable in this context. Second, if the ES+HIV intervention was significantly better than the control, we looked into the comparisons between the ES+HIV intervention and each of the single intervention groups. Third, if the comparisons in the second step were significant, indicating the ES+HIV group was more effective and our primary hypothesis was confirmed, then we looked into synergistic effects.
The ES+HIV intervention group had the lowest STI prevalence at endline 1. However, the comparison with the control group was not significant. We also found no evidence that the ES or HIV single intervention groups performed better than the control on STI prevalence. Given our pre-specified interpretation strategy, we did not consider other comparisons across the different intervention groups relevant. Results for endline 2 comparisons are not presented, but STI prevalence became more similar among the groups at endline 2.
We found no evidence of an effect of ES-only or HIV-only compared with the control for the HIV knowledge outcome. The ES+HIV intervention was significantly better compared with the control and better than ES only, but there was no evidence of an effect of the ES+HIV intervention compared with HIV-only. HIV knowledge was high in all groups at baseline and increased only slightly over time.
For ‘participation in a savings group’, ES was significantly higher than the control, but there were no significant differences between the combined and the control, nor between the HIV only and the control. Furthermore, the range for all groups varied from a minimum of 25% (control at baseline) to a maximum 38% (ES at endline 1).
At endline 1, 24% of participants in the ES-only and the ES+HIV interventions reported that they planned to use savings for education, and these groups were significantly different than the control group. Also, the ES+HIV intervention was significantly higher than the HIV-only intervention. However, there were no significant differences between the HIV-only and control groups nor between the ES+HIV and ES-only interventions. Moreover, all groups declined at endline 2.
As described above, we only planned to assess synergistic effects if we found that the ES+HIV intervention was better than the single interventions and the control. The comparisons for each of the outcomes did not meet this condition, so we conclude that we did not find evidence of synergistic effects of the combined intervention.
Sensitivity analyses
The purpose of the sensitivity analysis with attendance data was to determine if, for youth who attended at least 14 sessions of the interventions, the intervention influenced their outcomes. Although the sample size in this exploratory, non-randomised analysis was smaller than in the primary ITT analysis, there was high retention (n=1536/1773, or 87% of those randomised at baseline). Moreover, baseline characteristics were very similar between the full analysis sample and those included in these analyses. Only two small differences were observed for the baseline characteristics: 83% (ITT) versus 82% (sensitivity) received a household social grant, and 32% (ITT) versus 31% (sensitivity) were HIV affected.
Comparing the attendance data with the randomisation assignments, 67% (n=1194/1773) of participants in the study received the intervention to which they were randomly assigned. Most (68% or 396/579) participants who did not receive the intervention to which they were randomly assigned received no intervention at all. Completion of the minimum number of sessions in the ES+HIV intervention group (35.4%; 141/398) was lower than the HIV (60.2%; 239/397) and ES (68.8%; 282/410) groups.
Using attendance data did not change the findings substantially (table 4). When changes in significant effects were noted, these were mostly associated with effects observed in the primary analysis becoming more or less significant, but not with change in direction or unexpected effects becoming apparent.
Effect estimates for primary and secondary outcomes at endline 1 (exploratory analysis with attendance data)
Exploratory subgroup analyses
We ran additional models to explore if the interventions had effects on STI prevalence at endline one for specific subgroups (eg, community, age group, and gender) within the ITT population.
Independent from the intervention, having had an STI at baseline, being a girl, and reporting ever having had sex at baseline were predictive of having an STI (data not shown). Intervention interaction effects only emerged as potentially significant during the selection process with two variables: ever having had sex at baseline and gender. However, these effects lost significance in the final model that included only significant confounders. Even though intervention effects may be potentially different between boys and girls, and between adolescents reporting at baseline to have initiated sex and those who did not, these effects were not strong enough to conclude that there were true intervention effects. All groups were not significantly different than the control in any of the subgroups examined. Please also note that these subgroup analyses had low power to detect true effects.
Harms
Only one social harm event was reported during the study, which was unrelated to the interventions being studied.
Discussion
We did not find evidence that the interventions tested, separately or combined, impacted STI prevalence or sexual risk behaviours. We also found that youth who received the ES+HIV intervention had higher HIV knowledge overall, but we could not conclude that this combined intervention was better than the HIV-only intervention. We also conducted sensitivity analyses using program-reported attendance data to investigate whether the degree of participants’ exposure to the interventions affected the findings. Despite this stringent re-classification, we still did not observe many effects of the interventions, and we observed no significant effect on STI prevalence or sexual behaviours. The strongest trend we observed was that females had higher STI prevalence at all timepoints compared with males. Furthermore, STI prevalence increased as youth aged.
We found youth receiving the ES intervention were more likely to participate in savings groups and to plan to save for their education. It is possible that the broad-base financial literacy curriculum targeted a set of ES topics that will not yield results rapidly in this group of youth. These youth may see greater benefit from changing savings behaviour and increasing financial literacy knowledge when they are older and earning their own money. Youth in this context may realise more immediate benefits from interventions that transfer resources directly to them, such as matched savings or assistance with small income generating activities.
To our knowledge, this study is the first to implement a full-factorial randomised controlled trial to determine both the separate and combined effects of an integrated ES and HIV-prevention intervention on the prevalence of STIs among adolescents. Thurman et al conducted a study similar to ours by randomly assigning clusters of adolescents recruited from a support programme for HIV-affected families in the Eastern Cape Province of South Africa to receive one of the following: (1) a structured group-based behavioural health intervention (a previous version of the Vhutshilo curriculum), (2) interpersonal psychotherapy group sessions, (3) both interventions or (4) no new interventions (control group).34 After comparing the interventions with the control group, they found that exposure to both interventions was associated with some self-reported risk-reduction behaviours that were not found when either intervention was provided in isolation. Our study would have had similar findings—that is, concluding that the ES+HIV group performed better for the HIV knowledge outcome—had our main analysis compared the combined group with the control group only. The additional layer of analysis we conducted in which we compared the combined group with each individual intervention, as well as to the control, resulted in the finding that the combined group did not perform better than the HIV intervention alone in improving HIV knowledge in this context.
In addition to differences in our analysis strategy and measurement of the primary outcome (Thurman et al used self-report, whereas our study used the biomarker of STI testing), another key difference is that Thurman et al analysed data separately by gender and found differences, whereas we pooled our analyses to increase power to detect changes in STI prevalence. We did conduct exploratory analyses to see if there may be differing effects by gender, but we found none (data not shown). These differences notwithstanding, both Thurman et al’s study and our study contribute to the relatively nascent field of rigorous research of multisectoral programming for HIV prevention.28 33 As the HIV prevention field continues to implement complex multicomponent programmes, research methods must effectively evaluate their impact to inform decision-making for the use of finite resources—indeed, the health of youth depends on it.
Limitations
One limitation of the study was that it may have been underpowered to see an effect of the intervention at the effect sizes we targeted. While our goal was to enrol 2000 youth into the study, we did not achieve our sample size because we exhausted the eligible population, despite continuing baseline enrolment for several months longer than originally planned. In our study, we estimated 80% power to detect a decrease of 8% in STI prevalence. At endline 1, the ES+HIV group had an STI prevalence of 6%, and the control group had a STI prevalence of 11% for a difference of five percentage points. These observed differences are indistinguishable from the null effect due to sampling error. Therefore, we cannot conclude that the intervention effected STI prevalence.
A second limitation is that when comparing attendance data with the randomisation assignments, we found that approximately one-third of participants in the study did not receive the intervention to which they were randomly assigned. Most participants who did not receive the intervention to which they were randomly assigned received no intervention at all. As a result, it is possible that intervention effects seen in the primary analysis may have been attenuated. However, the analysis using attendance data did not change the findings substantially, which increases our confidence in the findings from the primary analysis.
It is also possible that the study groups suffered from contamination or a cross-over effect, as our study randomised participants at the individual level and participants may have shared information with others in different groups, including the control. Furthermore, we did not assess if our participants participated in other interventions or studies though the randomisation process should help to distribute this potential source of bias across the study groups.
Finally, although we used ACASI to ensure the privacy of participants and increase the accuracy of self-reported information, we still struggled to obtain valid self-reported data.52 53 Participant self-report has known limitations, and in our study, like other studies of youth, we experienced underreporting of sexual activity.54–59 More validation of these measures is needed.60 61
Strengths
This study had several strengths. First, we maintained high participant retention throughout the study—data from more than 90% of youth who were randomised were used in the analysis. Retention of adolescent participants, especially OVC, in longitudinal studies is challenging. We were able to achieve high retention through maintaining and updating contact information, placing data collection sites in multiple locations convenient to participants, and collaborating with FF, whose staff had developed relationships with families over time and assisted with locating hard-to-reach participants.
Second, we used a biomarker to assess the primary outcome which reduces measurement bias associated with self-reported data. Third, this study also represents implementation of a rigorous study design in a real-world programme. The randomised controlled design increases the internal validity of our study, and the real-world implementation increases the generalisability of our findings. Moreover, the full-factorial design is the only way to rigorously tease out the separate and combined effects of integrated programmes and to assess potential synergistic effects.
Interpretation
We cannot conclude that integration of these two interventions in this context worked to improve the HIV-prevention outcomes we sought to change. However, we found effects of the interventions on a few secondary outcomes. We also found important gender trends in STI prevalence, independent of the interventions, which confirm the appropriateness of many of development partners’ approaches to OVC programming, especially their focus on reducing HIV risk among girls.62
Recommendations
Youth attendance during the interventions was a challenge. Furthermore, completion of the minimum number of sessions of the ES+HIV intervention was lower than the HIV and ES interventions. This is not surprising since the number of sessions of the ES+HIV was double that of the single interventions. We recommend reducing the number of sessions. Future programmes could focus on the sessions on behavioural skills and negotiation, and trim those that present only knowledge, given that HIV knowledge already tends to be high at baseline. This may be achieved by transferring knowledge within skills-building activities, which may be facilitated by more fully integrating the content of multicomponent interventions instead of simply combining them.
Our finding related to self-reported sexual behaviour data has implications for future HIV-prevention research and programming. To reduce measurement bias, we recommend that biological endpoints such the incidence of STIs serve as primary outcomes for evaluations of programmes intending to reduce sexual behaviours associated with HIV risk. When biological endpoints are not possible, it may help to not skip youth out of future questions if their first response is that they never have had sex. Future research could also explore if having separate questions about consensual and non-consensual sex increases validity of self-reported responses. Furthermore, HIV-prevention programmes should consider targeting all youth, not only those who report that they are sexually active, because a greater number of youth participants are likely having sex than is reflected in survey data.
Even though full-factorial designs are difficult to plan, require larger sample sizes, and complicate the analysis and interpretation (eg, see Burke et al35 for further discussion), we recommend that future evaluations of promising multicomponent interventions use these designs.35 Full-factorial designs provide the necessary data for a more complete assessment of the effects of interest including potential synergistic effects.
Acknowledgments
The authors are grateful to the staff, families and youth from Future Families for their participation in this study, and especially Penny Learmouth and Taurai Ndemera for their assistance. We thank the research assistants from Foundation for Professional Development (FPD), and field co-ordinators, Ms Keabetswe Kodi and Mr Ralph Mawarire, for collecting the data for this study. We are grateful to Leila dal Santo (FHI 360) for her research management support, and Angie Wheeless and Andres Martinez of FHI 360 for their statistical support. We appreciate Natasha Mack (FHI 360) for editing the manuscript and Jennine Carmichael and Donna McCarraher, both from FHI 360, for their review of the manuscript.
References
Footnotes
Handling editor Seye Abimbola
Contributors HMB and MC conceived and designed the study. CB, PN and AM-M collected the data. AB, MC, KM and HMB analysed the data. HMB, MC and KM wrote the first draft of the article. HMB, MC, KM, CB, PN, AB and AM-M made substantial intellectual contributions to the article.
Funding This paper was produced under United States Agency for International Development (USAID) Cooperative Agreement No. AID-OAA-LA-13-00001 and was made possible by the generous support of the American people through the United States Department of State, USAID and the United States President’s Emergency Plan for AIDS Relief.
Disclaimer The contents are the responsibility of FHI 360 and do not necessarily reflect the views of USAID or the United States Government. The study sponsor was not involved in the collection, analysis or interpretation of data; the writing of the report; or the decision to submit the manuscript for publication. The named authors wrote the first draft of the manuscript. No honorarium, grant or other form of payment was given to anyone else to produce the manuscript.
Competing interests Authors do not have any potential, perceived or real conflict of interest to declare. The study sponsor (USAID) was involved in the study design by requesting the authors to evaluate the intervention in South Africa and they also reviewed the protocol.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Patient consent for publication Not required.
Ethics approval This study was reviewed and approved by the federally registered institutional review board of FHI 360 (Protection of Human Subjects Committee) and the University of Pretoria Faculty of Health Sciences Research Ethics Committee in South Africa.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data may be obtained from a third party and are not publicly available. Data have been submitted to USAID’s Development Data Library (DDL).