Introduction Short-term missions providing patients in low-income countries with reconstructive surgery are often criticised because evidence of their value is lacking. This study aims to assess the effectiveness of short-term reconstructive surgical missions in low-income and middle-income countries.
Methods A systematic review was conducted according to PRISMA guidelines. We searched five medical databases from inception up to 2 July 2018. Original studies of short-term reconstructive surgical missions were included, which reported data on patient safety measurements, health gains of individual patients and sustainability. Data were combined to generate overall outcomes, including overall complication rates.
Results Of 1662 identified studies, 41 met full inclusion criteria, which included 48 546 patients. The overall study quality according to Oxford CEBM and GRADE was low. Ten studies reported a minimum of 6 months’ follow-up, showing a follow-up rate of 56.0% and a complication rate of 22.3%. Twelve studies that did not report on duration or follow-up rate reported a complication rate of 1.2%. Fifteen out of 20 studies (75%) that reported on follow-up also reported on sustainable characteristics.
Conclusions Evidence on the patient outcomes of reconstructive surgical missions is scarce and of limited quality. Higher complication rates were reported in studies which explicitly mentioned the duration and rate of follow-up. Studies with a low follow-up quality might be under-reporting complication rates and overestimating the positive impact of missions. This review indicates that missions should develop towards sustainable partnerships. These partnerships should provide quality aftercare, perform outcome research and build the surgical capacity of local healthcare systems.
PROSPERO registration number CRD42018099285.
- systematic review
- other infection
- public health
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What is already known?
There is rising concern about the accountability, patient safety and sustainability of short-term reconstructive surgical missions; however, data on these parameters are lacking.
What are the new findings?
Evidence provided by research on surgical outcomes is limited and of low quality, and the safety of missions is likely to be overestimated by studies in which the quality of follow-up is not reported. Our data suggest that engagement in sustainable development of the local healthcare system and the feasibility of conducting high-quality, long-term follow-up go hand in hand.
What do the new findings imply?
We call for implementing longer-term outcome research of future missions.
One approach that could provide a framework to conduct such research, is to implement diagonal development missions. These missions combine the positive impact of the short-term vertical inputs (e.g. providing surgical services) and long-term horizontal investments (e.g. development of sustainable healthcare systems), with the aim of improving access to, and capacity of, the local surgical healthcare systems in the long-term.
Conditions that are treatable by reconstructive surgery make up a large part of the global burden of surgical disease. Examples are burns (8 .1 million disability-adjusted life years [DALYs]),1 orofacial clefts (0.23 million DALYs),1 complex wounds including trauma-related wounds (unknown DALYs, but estimated to be significant), pressure sores (0.67 million DALYs)1 or noma (a neglected tropical disease, a roughly estimated 1–10 million DALYs).2 Short-term reconstructive surgical missions are a well-established routine method of addressing these conditions and reducing their impact on global health by providing specialised care in underserved populations.3 Such missions are commonly short term, disease specific, focus on service delivery and have a tendency to work outside the local healthcare system. This is also referred as a ‘vertical approach to healthcare development’.4
Despite being a common model, the impact of reconstructive surgical missions is hardly known.5 6 Medical missions in general are commonly debated in the literature.7–15 There is rising concern about the limited accountability of missions, with little data reported back to healthcare authorities due to a lack of outcome measurements.8 9 Quality of care is debated, as missions often have limited capacity to provide ancillary services or follow-up.16 Furthermore, sustainability is questioned in terms of lasting positive impact on the local healthcare system or its cost-effectiveness. The question is whether short-term surgical missions are the most rational allocation of resources to address local healthcare needs.4 7 8 12 15 Ultimately, the ethical implications of surgical volunteerism often ignite debate.17 18
These concerns are discussed in several reviews of medical missions in low-income and middle-income countries (LMICs).7 9 10 12–15 19 For instance, Martiniuk et al and Roche et al argued that global standards are needed for short-term medical missions,7 14 and Sykes shows that only 6% of all published studies on medical missions report on empirical data.10 Only a few studies reviewed surgical missions specifically.9 12–15 19 Shrime et al systematically compare three types of charitable platforms for global surgery (short-term missions, self-contained surgical platforms and specialty surgical hospitals run by non-governmental organisations [NGOs]). Although they conclude that evidence in the literature is scarce, they state that self-contained temporary platforms and specialised surgical centres appear to provide more effective and cost-effective care than short-term surgical missions, except when no other delivery platform exists.12
These reviews provide valuable insights into medical missions in general. However, the diversity of medical and surgical missions is large, which hampers the interpretation of empirical data. This review aims to systematically review evidence on the impact of short-term reconstructive surgical missions specifically, and critically analyses the quality of the available empirical data. In this review, four key aspects are addressed: basic characteristics of missions, patient safety, health gains of individuals and sustainability.
We conducted a systematic review of the literature following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines.20
All original studies that analysed empirical data of short-term missions pertaining to reconstructive surgical care in LMICs were eligible. Studies lacking analyses of empirical data, reviews, studies of specialty surgical hospitals that provide continuous year-round care, mobile surgical platforms sent from in-country hospitals, studies in conflicts zones, studies not related to LMICs or studies of patients that were transported to high-income countries (HICs) were excluded. The studies were restricted to English and Dutch language. No restrictions were applied regarding publication dates. Duplicates were excluded.
Search strategy and data sources
PubMed, Embase.com, Clarivate Analytics/Web of Science and Open Grey were searched up to 1 July 2018; Proquest up to 1 July 2017 (by CR, TH and JK). The last database was no longer available to us after 2017. The following terms—including synonyms—were used as index terms or free-text words: ‘plastic surgery’, ‘reconstructive surgical procedures’, ‘cleft lip’, ‘post-burn contractures’ or ‘noma’ combined with ‘medical missions’, ‘humanitarian’ or ‘charity’. More studies were identified by reviewing the bibliographies of retrieved studies. The full search strategies for all databases can be found in the online supplementary file 1.
Studies were screened for eligibility by two independent investigators (TH and CR): in case of disparity, a third author was involved (MB). Two investigators independently extracted the data to create tables and figures (TH and CR).
Data on mission characteristics and individual patient-level data were extracted and analysed across four key features:
Basic characteristics of the missions, including mission length, number of patients who received surgery, and age and gender distribution.
Patient safety by means of complication registration. Data were collected on three indicators: follow-up length, follow-up rate and complication rate. The follow-up rate was calculated by the number of patients who completed follow-up divided by the total number of patients who were included for follow-up. The complication rate was calculated as follows: the number of patients with complications divided by the total number of patients who completed follow-up.
Health gains: data on surgical outcomes were collected, for example, improvement of range of motion, patient-reported outcome measures (PROMs) or DALYs averted per patient. All reported PROMs were recorded, for example on surgical outcomes, complications or the quality of care provided. All types of formats, questionnaires or any other tool describing these outcomes were included.
DALYs are used to define the overall disease burden over a population and are calculated by ‘adding the number of years of life lost due to premature mortality to the number of years of healthy life lost related to disability’.21 This means that 1 DALY can be defined as one lost year of healthy life.21 Many limitations of this approach are described in the literature. The biggest challenge is that it is not based on health data from countries, but on complex estimation techniques. DALYs are therefore estimations and many concerns exist about their reliability and uncertainty.22 23 Despite these challenges, DALY metrics are commonly applied in global surgery studies.1 21 24
Sustainable characteristics of missions: studies were reviewed for data on long-term partnerships, training objectives and involvement of local staff. Also, data on the cost-effectiveness of missions were collected. Missions were categorised either as stand-alone or consecutive missions to the same hospital, region or country.
Data analysis and synthesis
After a pilot, data were extracted independently and in duplicate using a data extraction sheet (TH and CR). Authors were contacted when data on complication registration were missing. Quantitative data synthesis consisted of compiling total number of patients (eg, total number of patients who were included, total number of patients with complications) to generate overall outcomes. Table 1 and the online supplementary appendix file 1 provide details of the data extracted from each reference. Due to the heterogeneity of studies in types of surgery, local healthcare settings or available resources, statistical analyses were not feasible. Study quality assessment was performed independently by authors TH and CR according to the Oxford CEBM Level of Evidence classification25 and the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) system.26
The search identified 1662 unique citations. After screening titles and abstracts 1570 studies were excluded because they did not concern short-term reconstructive surgical missions. Ninety-two studies concerned reconstructive surgical missions and were reviewed full-text. Of the full-text studies, 51 were excluded. Please see figure 1 for the screening and selection process. Studies reviewing specialty hospitals missions were excluded, as this was outside the scope of this review. After full-text analyses, 41 met full inclusion criteria (figure 1 and table 1).27–67
The studies included predominantly consisted of case series, with 37 studies graded at Oxford CEBM Level IV. The remaining four economic analyses were graded at level IIB. This resulted in a C grade of recommendations for our review, according to CEBM. The overall GRADE score was 2.7 (low to moderate quality), meaning that our confidence in the effect estimate is limited (table 2).26 68
Twenty-eight of 41 studies included pertained to cleft care (78% of the total study population). The number of patients in all studies totalled 48 546, with a mean age of 13.4 years (SD 8.5) (table 3). The average mission length was 10 days (SD 3.8; range, 6–21 days). Organisations were active in Africa, South-East Asia, Eastern Europe, and South and Central America. A typical mission team consisted of two or three plastic and/or maxillofacial surgeons, one or two anaesthetists, a mission coordinator, theatre nurse and one or two resident doctors, totalling 8 to 10 individuals for one single mission.32 33 37 46 47 50 51 54 58 Some teams were considerably larger, up to 40 individuals.36
Regarding patient safety, nine studies (22%) did not report on complications. Twelve studies (29%) only reported a complication rate without reporting on follow-up length or rate. The overall complication rate in these 12 studies was 1.2%. Ten studies (24%) reported a follow-up length shorter than 180 days, with a mean follow-up rate of 81.3% and a complication rate of 7.1%. Ten studies (24%) provided a follow-up length longer than 180 days, reporting a mean follow-up rate of 56.0% and a 22.3% complication rate (table 3). Mortality after cleft surgery was reported in three studies, totalling 3 out of 14 551 patients included in these studies.16 28 64 For general reconstructive surgical missions—not specified for a single disease—one single study reported one death54 and no mortality was reported in contracture and noma missions.
Twelve studies (29%) reported on health gains of the mission, reporting heterogeneous methods and outcomes (table 1). Methods used in cleft studies included photographic assessment of aesthetic outcomes,30 69 speech evaluation16 47 or DALYs averted.41 43 48 55 Three cleft studies evaluated speech functionality postoperatively, either by questionnaires61 or speech tests.16 47 Study methods were clear and showed overall improvements of speech. Four cleft studies reported on DALYs averted by cleft lip and palate repair surgery. DALYs averted per patient were 3.9,43 6.041 and 10.1 per patient.48 In three noma missions, a surgeon-reported outcome scale was used to score aesthetic and functional outcome.32 50 52 Overall findings showed that high-complex surgery is associated with greater risks of unsatisfactory results. Three studies used PROMs. One contracture study reported improvements in quality of life and disability by using validated questionnaires, and reported overall positive outcomes.63 Two cleft studies used self-developed questionnaires to assess PROMs, reporting positive results.61 66 None of the studies reported on patient-reported outcomes on the quality of the care provided.
With regard to the sustainable characteristics of missions, 29 studies reported qualitative data (71%) on sustainability, while none of the studies reported quantitative data. Fifteen out of twenty studies that reported on follow-up and complications also reported on sustainable characteristics such as long-term partnerships or training activities (table 4). Ten organisations (24%) were engaged in longer-term partnerships, and thirteen missions (32%) returned to the same regions or hospitals. Few data were available on the frequency of missions, although several studies reported conducting yearly missions.32 35 43 50 Fifteen studies (35%) described teaching objectives as a goal during their missions. Activities mentioned were lectures,29 37 55 training of local surgeons,16 35 38 54 healthcare workers47 55 60 or fellowships in donor countries.16 54 55 However, none of the studies published empirical data on the effects of training or elaborated on how the training of local healthcare personnel was organised.
Four cost-effectiveness studies were available for short-term cleft missions. Three studies reviewed the effectiveness per DALY averted, reporting US$33.94/DALY,48 US$56.0/DALY55 and US$247.42/DALY.41 The variation is explained by the differences in study populations, sample sizes, effectiveness measurements and ‘costing approaches’ used.
Several systematic reviews about short-term medical missions are available.7 9 10 12–15 This is the first systematic review that specifically assesses the quality of available data on short-term reconstructive surgical missions.
Although all the studies included in our review reported a positive impact of surgical missions, the level of evidence remains low. It seems that follow-up of treated patients is a challenge. Although a majority of studies provide data on complication rates, the varying quality of this outcome measure makes it difficult to draw any conclusions. The results showed that reported complication rates were considerably higher when the quality and length of follow-up increased. This suggests that without data on quality of follow-up, there is a high risk of reporting bias due to under-reporting of complications. This also means that without comprehensive information on follow-up, the safety of missions is likely to be overestimated.
Furthermore, studies used different control groups to benchmark their respective findings with regard to complication rates. Three of cleft care studies included compared complications between mission patients and patients who underwent similar procedures in a HIC. Results showed substantially higher complication rates in mission patients.34 49 60 One study showed that fistula risk was 15.6 times that for a US cohort.34 Maine et al 49 state that complication rates were 20 times higher in the mission cohort compared with a US cohort, independently of whether the surgery was performed by Ecuadorian or American surgeons. It should be mentioned that comparisons of complication rates between HICs and LMICs cannot be made without taking into consideration that HICs have more resources at their disposal to limit complications. Therefore, we would suggest developing benchmark complication rates of LMICs, which can be used to assess the outcomes of short-term missions.
Some authors argue that longer-term specialty surgical hospitals may be provide more effective care than short-term missions.9 12 39 Specialty hospitals provide continuous care all year round in a LMIC. The cleft care centre of Operation Smile in India,39 or Smile Train’s model are examples of this approach.69–78
Both organisations report lower complication rates than those reported in short-term missions. The centre of Operation Smile reports a short-term complication rate of 4.0% (cleft lip repair) and 15.8% (cleft palate repair), which is lower than the rates of their counterpart short-term missions.39 Smile Train studies report lower rates, between 0.88% and 3%.70 71 73 78 However, they note that there might be a risk of under-reporting or selection bias due to a dependence of Smile Train surgeons on payment-per-patient (risking fewer referrals when higher complication rates are reported) and a limited capacity of surgeons to treat complex cases.70 71 73 78 Furthermore, with only one Smile Train study reporting on follow-up lengths,73 these complication rates should be interpreted with caution. To be able to compare the strengths and weaknesses of different approaches of providing surgical care in a LMIC, there is a need for more high-quality studies.12 39 Apart from registrations of complications, such studies should assess long-term outcome using validated outcome measures and PROMs. Specialty hospitals, which provide services all year round, could provide good conditions for longer-term outcome research.
Several studies in this review consistently report on follow-up, showing that substantial efforts are being made to improve the data output of missions.16 34 35 49–51 61 63 Ten studies reported significant follow-up lengths of more than 6 months and high numbers of patients returning for follow-up were shown.16 34 35 47 49 58 60–62 66 The majority of these missions were engaged in long-term partnerships. This included training of local healthcare personnel, which was likely to improve the feasibility of organising follow-up. Several strategies were implemented to ensure the quality of follow-up. Some missions deployed medical students to assess palate fistulas34 or sent a speech pathologist in-country to review outcomes.61 Others trained local surgeons on follow-up and revision surgery.54 The relatively high number of complications seen in noma missions could be partly explained by a stringent follow-up, done by an independent researcher who consistently reported on follow-up. All the studies mentioned above provide examples of how to ensure patient safety during and after missions.50 79
Although some studies reported on health gains, with several studies reporting positive functional outcomes,16 47 61 63 66 the methods and evidence are heterogeneous and results are too limited to draw conclusions. The role of PROMs are effective in reconstructive surgery to assess the quality and outcomes of healthcare.80 81 Only few of the studies included reported successfully on outcomes using PROMs61 63 66 and none assessed the quality of care experienced by patients. Patient experience of outcomes and quality is important.82 Future studies should include PROMS on surgical outcomes and quality of care. Only a few studies report on the sustainable characteristics of missions. Data on this topic are usually qualitative and highly variable. It is noteworthy that reporting on sustainability and higher quality of patient follow-up often go hand in hand. This suggests that more sustainable missions may be better able to follow their patients for a longer period. However, as empirical evidence on sustainability is still non-existent, there is an urgent need for further studies.12
This systematic review has several limitations. Literature on short-term reconstructive missions is scarce and of limited quality, limiting the strength of this review.25 As the majority of studies are cleft studies, the conclusions and recommendations of this review may not be fully applicable to other types of reconstructive surgical missions.
The studies included represent just a small proportion of the many reconstructive surgical missions conducted worldwide. This may introduce a potential bias. It is likely that the small proportion likely does not fully represent the actual effect of all reconstructive surgical missions. In our view, this emphasises the need to incorporate standard monitoring and evaluations into missions.
Furthermore, this review addresses only short-term missions and does not attempt to make a direct comparison with long-term surgical platforms such as specialty hospitals. It is often argued that specialty hospitals are safer and have a more positive effect on local healthcare systems.12 76–78 83 84 Comparative studies of short-term missions and specialty hospitals can identify strengths and weaknesses of each approach. However, a definitive comparison between missions and specialty hospitals seems to be premature at present given the lack of comparative studies.12 39
Concerns regarding the use of DALY metrics are applicable to the studies included in this review. It is argued that surgical conditions are underestimated in the global burden of disease studies.1 Attempts to estimate the surgical burden across all disease conditions have been challenging.85 86 In a recent study, it was argued that the current DALY approach is inadequate to quantify the burden of paediatric surgical conditions.87
There are opportunities for NGOs to develop short-term missions towards more sustainable partnerships. In the past, missions have been a ‘vertical’ approach to healthcare development.4 Such missions have limitations, for example in building local capacity of surgical services. The results of this study indicate that longer-term follow-up is frequently lacking, with complications being potentially missed. To address these shortcomings, the ‘diagonal development’ approach has been proposed.4 It combines the short-term vertical inputs of missions with longer-term horizontal benefits, with the ultimate aim of improving access to, and surgical capacity of, the local healthcare system. Such goals may be achieved through long-term development of surgical infrastructure, continued training of the local surgical workforce or building an academic culture.4
One example of such a diagonal approach is to aim for standardised tracking of longer-term outcomes of missions in strong collaboration with local partners. This might yield several advantages. Besides empowering local researchers and building an academic culture, outcomes can be reported back to patients and healthcare authorities. This will enhance the accountability of NGOs8 9 and allow for evaluations of the quality of care provided.
Another example of long-term investments in the local surgical capacity is strengthening of the training activities of surgical NGOs. Such activities should be integrated into existing national or regional training activities. The training should be adapted to local settings, needs-driven and should focus on bilateral knowledge exchange.4
This review shows that evidence for the effectiveness of short-term reconstructive surgical missions is both of limited substance and quality. Given the overall lack of evidence, there is an urgent need to incorporate outcomes research in future missions. This should include longer-term complication registration and measurements of health gains among individual patients. The effectiveness of training activities should also be evaluated. One approach to achieve this is to develop short-term missions towards diagonal development missions, which aim to build surgical capacity of local healthcare systems through long-term investments.
Handling editor Soumyadeep Bhaumik
Contributors The contributorship will be proceeded with the plastic surgery department at VU University Medical Center Amsterdam.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.