Introduction Countries with strong primary healthcare (PHC) report better health outcomes, fewer hospital admissions and lower expenditure. People-centred care that delivers essential elements of primary care (PC) leads to improved health outcomes and reduced costs and disparities. Such outcomes underscore the need for validated instruments that measure the extent to which essential, evidence-based features of PC are available and applied to users; and to ensure quality care and provider accountability.
Methods A systematic scoping review method was used to identify peer-reviewed African studies and grey literature on PC performance measurement. The service delivery dimension in the Primary Healthcare Performance Initiative conceptual framework was used to identify key measurable components of PC.
Results The review identified 19 African studies and reports that address measuring elements of PC performance. 13 studies included eight nationally validated performance measuring instruments. Of the eight, the South African and Malawian versions of Primary Care Assessment Tool measured service delivery comprehensively and involved PC user, provider and manager stakeholders.
Conclusion 40 years after Alma Ata and despite strong evidence for people-centred care, significant gaps remain regarding use of validated instruments to measure PC performance in Africa; few validated instruments have been used. Agreement on indicators, fit-for-purpose validated instruments and harmonising existing instruments is needed. Rigorous performance-based research is necessary to inform policy, resource allocation, practice and health worker training; and to ensure access to high quality care in a universal health coverage (UHC) system—research with potential to promote socially responsive, accountable PHC in the true spirit of the Alma Ata and Astana Declarations.
- Public Health
- Health policy
- Health systems evaluation
- Health services research
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What is already known?
Measurement of primary care (PC) performance globally informs and guides primary healthcare (PHC) systems strengthening.
There is however scanty information about PC performance measurement in Africa despite the commitment to strengthen PHC and universal health coverage.
What are the new findings?
Little is being done to measure PC service delivery in Africa using validated instruments.
Few validated tools have been used to date and no research has been conducted to determine which instruments are best suited to the African context.
What do the new findings imply?
Validated PC performance measures are needed in Africa to guide, support and evaluate efforts to improve access to quality PC.
Research is needed in Africa to evaluate instruments used and/or develop new measures to provide evidence necessary to strengthen PC and ensure universal health coverage.
The primary goal of a health system is to optimise health.1 Primary healthcare (PHC) is the backbone of a healthcare system and central to improving accountability in health service delivery.2 Countries with strong PHC report lower health expenditure, better outcomes and fewer hospital admissions.3 4 In 2008 the WHO call for PHC reform recognised the need to reorganise primary care (PC) around people’s needs and expectations with potential to reduce the occurrence of common illnesses by up to 70%.5 The emphasis was underscored in 2017.6 As a core dimension of PHC, comprehensive people-centred PC can reduce the burden of disease by improving opportunistic screening, disease prevention and health promotion.7 8 Comprehensive PC is in line with the Alma Ata and Astana Declarations; with universally accepted definitions of PC; and with universal principles of family medicine and their practice in Africa.9–13 Given high-level support for PC and growing evidence of its contribution to improving health outcomes, evaluating PC performance using validated instruments should be a key activity of PHC strengthening and reform.
Kidd et al13 note the substantial evidence for PC as central to strengthening health systems. When practiced as a multidisciplinary team and population-based approach, person-centred PC as the entry point to healthcare meets most healthcare needs including health promotion and disease prevention. Evidence-based PC delivery involves key elements shown to be essential for cost-effective PC–namely access to PC as first contact; continuity; comprehensive; coordinated; family-orientated and community-orientated care.13–15 When these elements are available to PC users and applied in a person-centred approach to care, outcomes include improved health and satisfaction, and reduced costs and health inequities.3 Evidence in favour of such outcomes underscores the need to measure PC using appropriately validated instruments.
This scoping review aimed to determine the current state of PC performance measurement in Africa and identify knowledge gaps in PC service delivery that can be addressed by research.
Fifteen members of the Primafamed PHC African Network,16 faculty from university departments of family medicine in 11 sub-Saharan African countries and Belgium met over 3 days in response to a call for scoping reviews identifying gaps in PC research in Africa. Participants self-selected themselves to three teams, each tasked with identifying topics to review. Our five-member team used the nominal group technique method17 to generate and prioritise a list of PC research challenges in Africa. The need for validated, contextualised instruments to measure PC performance obtained the highest score by group consensus. This review therefore addresses the question: ‘What is already known about valid tools or measures or instruments to measure primary care performance in Africa?’ We used the staged scoping review method recommended by Peters et al18 to identify, extract and categorise relevant data from peer-reviewed and grey literature on instruments used to measure PC performance in Africa. We used the Preferred ReportingItems for Systematic Reviews and Meta-Analyses-Extensionfor Scoping Reviews checklist to guide reporting the results.19
The focus of the review is PC measurement. The terms ‘primary care’ and ‘primary healthcare’ and their definitions are frequently used interchangeably in the literature.20 21 This presents a challenge when searching the literature for studies focusing specifically on PC performance measurement that is, on essential elements of personal PC as a dimension of the wider PHC system including population-based approaches to healthcare.22 Definitions used to distinguish PC and PHC are available in Appendix A. We used the five-dimension Primary Healthcare Performance Initiative (PHCPI) conceptual framework (Appendix B) to assist making the distinction by locating the study in the service delivery dimension.23 Service delivery domains–namely population health management, PC facility organisation and management, access, availability of effective PHC services and high quality PHC–and their respective subdomains are in line with the essential elements of PC noted above.2 4 15
Given the breadth of PC, we conducted an initial search of electronic databases using iterations of a basic search strategy comprising PC MeSH terms and keywords to identify literature that addressed the review question. The initial results were then used to design a stronger search strategy.
We elicited the assistance of experienced reference librarians to design a more focused, unified PC search strategy using the Pubmed search builder (Appendix C). Additional MeSH terms and keywords from the initial search results meeting our criteria were included. The Pubmed strategy was adapted to search other databases. We searched in Pubmed; Web of Science; Africa-wide (EBSCOhost); Scopus; Hinari and CINAHL. Each member of the team was assigned a database(s) and applied the inclusion and exclusion criteria (table 1) to the titles and abstracts of the search results. The full texts were examined for more information and to assist with inclusion/exclusion if the abstracts were insufficient. We sought to identify grey literature by searching in Google Scholar, Sabinet and Google. All results selected were submitted to the team leader (GB) who removed duplicates and captured key information in a spreadsheet including titles and abstracts.
Each team member examined and applied the selection criteria to the results in the spreadsheet compiled in Step 2, adding her/his comments to justify inclusion/exclusion. Reference lists and reports were examined for any African studies not identified in Step 2. The unified search notwithstanding, results included studies that focused only on PHC dimensions other than service delivery or other PHC dimensions and selected service delivery domains. Identifying research focusing specifically on PC—especially those using validated performance measuring instruments—was problematical. We therefore developed a rating scale for relevance to determine which studies to include or exclude at this stage: High Relevance Grade 1 (HR-1)—African studies using validated instruments to measure PC performance comprehensively that is measuring performance on most PHCPI service delivery domains (C1–C5); and High Relevance Grade 2 (HR-2)—African studies measuring performance on a single or limited number of service delivery domains that is, not comprehensively, and/or focusing largely on other PHCPI dimensions. The final rating for each study was decided on by consensus after each team member examined the full texts of studies in the spreadsheet along with the others’ ratings. In this way studies with full texts available were rated for their relevance to the review question. Studies that did not achieve team consensus were discussed in conference calls. Full texts of the final selection were examined further to extract additional data for entry in a standardised template (table 2).
Patient and Public Involvement
No patients nor members of the public were involved in the review.
The databases search yielded 5215 peer-reviewed studies (figure 1). At this stage a newly published study24 not yet indexed in Medline was added; it met the HR-1 rating. All titles were screened; abstracts of titles that screened positive or uncertain were examined. After removing duplicates from the results submitted by each team member, 86 results remained in the spreadsheet. Sixteen16 grey literature results were added giving a total of 102 results identified as potentially eligible for further examination. Two results were excluded because the full texts were unavailable online. The full texts of the remaining 100 results were examined by team members as described in Step 3. HR-1 and HR-2 rated results were combined. Team consensus was achieved on 19 results as meeting the inclusion criteria (table 2)—18 published, peer-reviewed studies and one published, peer-reviewed report.
Thirteen studies24–36 addressed eight nationally validated PC performance instruments—namely the South African and Malawian versions of the Primary Care Assessment Tool (ZA PCAT and PCAT-Mw respectively)26 34 ; the electronic Tool to Improve Quality of Healthcare (e-TIQH) in Tanzania25 ; the Nigerian Patient Evaluation Scale (PES)31; the South African Ideal Clinic Realisation and Maintenance (ICRM) instrument29 ; SafeCare Essentials tool33 ; the European Task Force for Patient Evaluation of General Practice (EUROPEP)37 ; and the Client Satisfaction Survey Questionnaire.35 (Information on instruments is summarised in Appendix D uploaded as a supplementary file.) Seven of the 13 validated tools studies were PCAT studies—five conducted in South Africa and two in Malawi. The ZA PCAT, PCAT-Mw and PES studies included the theoretical basis and methods used to validate them in the African context. Table 3 compares the validated instruments against the PHCPI dimensions and domains covered and shows the dominant dimensions in each case. As is evident in table 2, most of the studies assessed performance on more than one PHCPI dimension. The remaining six of the 19 results focused largely on measuring one or more dimensions in the PHCPI framework in addition to service delivery. Three3 of the six studies used data analysis methods to analyse existing data from household surveys, WHO health accounts and other sources.38–40 The remaining three studies41–43 concerned patient questionnaires and qualitative methods such as focus groups to determine users’ evaluation or satisfaction with PHC services or their perceptions and expectations of services. They include a systematic review of patient questionnaires evaluating PC in Nigeria.41 These studies are included because they mention known tools which were adapted although they fail to describe how these were adapted and whether they were validated. Five studies entered in the spreadsheet in Step 2 were excluded because information on the origin or local development of their user survey questionnaires (for use in local communities) was not provided.
The review aimed to document the current state of PC performance measurement in Africa and to identify gaps in knowledge and research. In order to manage the challenge of the interchangeable use of ‘primary care’ and ‘primary healthcare’ and their definitions in the literature, we chose the service delivery dimension of the PHCPI conceptual framework to define and situate PC within the PHC system. Notwithstanding efforts to limit the search to PC (results therefore do not reflect the range of PHC performance research in Africa), it continued to yield many studies addressing other PHC dimensions. Each selected study measured at least one service delivery subdomain. Few studies measured performance on most service delivery domains and subdomains (table 3) despite global calls and strong evidence supporting PC.
Eight nationally validated instruments (table 3) measuring PC performance in Africa were identified; five—namely the PES, e-TIQH, SafeCare Essentials, ICRM and Client Satisfaction Survey Questionnaire—were developed in Africa. The ZA-PCAT, PCAT-Mw and EUROPEP—developed in the USA and Europe respectively—were cross-culturally adapted and validated. Only the e-TIQH, ZA PCAT, Malawian PCAT, Nigerian PES and EUROPEP studies described the validation processes followed.
Regarding coverage of service delivery domains, the PCAT measured PC performance on more domains and subdomains than other instruments (table 3)—that is, measured PC performance comprehensively—and involved user, provider and manager stakeholders with potential therefore to address demand and supply performance features. Only the PCAT and EUROPEP are among the six validated instruments selected in a review by Haggerty et al.44 The PCAT also features in a literature review on PC assessment tools45 and in a paper emphasising the need for internationally validated instruments and indicators that ‘measure safety and quality of PC (and) integration with the rest of health system and workforce development’.13 Seven of the 13 studies on validated instruments focused on the PCAT compared with one paper each for the other seven validated instruments.
The Tanzanian-developed e-TIQH25 measures performance on 6 PHC components (Appendix D) addressing processes and structural adequacy of healthcare—but configured differently to PHCPI dimensions. Of note is the use of the e-TIQH to provide service providers and managers with baseline performance data and to measure impact afterward. The study illustrates how a performance measuring instrument can be used as an intervention to influence practice positively after feedback is given directly to providers and managers. Other comprehensive instruments listed in table 3 could be used to similar effect.
The PES (full version), developed and validated for use in Nigeria, and the EUROPEP, developed in Europe and validated for use in South Africa, enable user evaluation of PC. The PES enables patients to evaluate elements of facility organisation and health services provided (table 2); the EUROPEP enables patient evaluation of provider competence, patient–provider communication and practice organisation. The EUROPEP study37 assessed the psychometric properties of the South African (isiXhosa) version.
The Client Satisfaction Survey study35 illustrates the need for agreement on indicators, definitions and instruments to assess PC performance13 46—in this case from the user perspective. It measures patient satisfaction on ‘eight domains’ of performance namely access, empathy, referral, general satisfaction, service standards, reliability, health promotion, tangibles and assurance. As with other studies, it is difficult to map these domains onto the PHCPI service delivery dimension. For example, ‘tangibles’ comprises 10 questions covering service infrastructure and management that is, PHCPI B and C dimensions; ‘assurance’ comprises questions regarding provider competence (C4b), coordination (C5c) and information on prevention and promotion (C5d). The South African ICRM instrument is a standardised checklist-based audit tool focusing mainly on PC facility compliance with operational and structural components rather than PC process.
Three studies applied data analysis methods38–40 to assess performance using existing survey data from various sources. A Nigerian study40 analysed data from health and household surveys, WHO health accounts and workforce data. Different data analysis methods were used to assess performance on at least one domain of each of the five PHCPI dimensions. These methods contrast with instruments noted above that can potentially be used by PC and district health services stakeholders—including users, providers and managers. On the other hand the five-dimension coverage in the Nigerian study points to shortcomings of other instruments in table 3. Despite the ZA PCAT’s relatively comprehensive measure of service delivery, coverage of C1 and C2 domains is minimal.
What do these findings mean? Thematic categorisation of the results is problematical and perhaps premature given the paucity of studies meeting the review criteria. At this stage of African PC performance research there seems little value categorising instruments according to whether nationally or not nationally developed; whether nationally validated or not; measuring performance comprehensively or selectively; user-survey questionnaires only or user-survey instruments and/or collecting data from other sources. Similarly, categorising results by method for example, desktop methods analysing existing data or using instruments to collect data de novo (with potential to involve key PC stakeholders). Attempting categorisation may however be useful as an exercise to identify gaps in PC performance research for example, regarding instruments required for relevant and reliable data needed to ensure high quality PHC in Africa.
The need for collaboration and agreement on indicators and instruments: The review findings reflect a lack of collaboration and agreement among researchers regarding the most appropriate indicators and instruments needed to evaluate African PC performance—highlighting a need for a unified approach in order to address knowledge gaps comprehensively. Studies failed to address the need for consensus on performance indicators and instruments most suited to African PC performance measurement—reflecting a global gap in health systems research.13 46 The difficulty mapping performance indicators used in most studies onto a common framework such as the PHCPI or comparing study findings, underscore the need for agreement on indicators and their definitions within a unified approach. Few studies focused on service delivery and none covered the C-domains sufficiently comprehensively—further gaps in African PC performance research. Knowledge gaps and research questions include which existing methods and instruments are fit-for-purpose in Africa and which need validating; whether there is a need to develop new instruments specifically for the African context and why; and whether instruments validated nationally (in Africa) can be cross-culturally adapted to measure PC performance effectively in other African states and enable comparison of local districts and African countries.
The fragmentation of instruments and research efforts to measure performance is not limited to Africa. Studies in high-income countries46–48 note that many validated instruments available to measure PHC do not all address important attributes nor the same range. Such fragmentation is emphasised by several studies (excluded in Methods Step 3) that sought to measure performance using locally developed questionnaires to survey patient satisfaction or perceptions of services. They failed to provide the necessary information on methods and instruments used; to describe how or where the instruments were developed; or how they were ‘pretested’. These findings highlight a need in Africa to improve the rigour of instrument development and use, and to reduce research fragmentation—especially in a resource-scarce context. The studies identified do not inform decision-making regarding evidence required to achieve agreement on instruments most suited to measure PC performance; nor help determine what constitutes cost-effective PC in the African context; nor do they discuss how the absence of agreement on indicators, instruments and methods should be addressed. These findings suggest an urgent need for a unified, African PC performance research framework.
The need for comprehensive evaluation of performance: The role of user surveys is supported by evidence that patient-centred care and patient feedback improve health outcomes. However, the studies using the e-TIQH, SafeCare Essentials and ZA PCAT instruments show the value of sampling more than one source. Their results (Key findings in table 2) show significant discrepancies between staff and user evaluations of service performance—in effect questioning the value of results from one source only and suggesting a need for technically based as well as user experience-based assessments. Results obtained using the SafeCare Essentials tool show a negative association between technical quality and user-perceived quality of care and significant user-provider differences on all indicators measured. The ZA PCAT studies similarly show significant differences between users, service providers and managers on key elements of PC performance including community-orientated PC and comprehensiveness of services provided. The SafeCare Essentials study nevertheless emphasises user evaluation—‘Improving technical quality alone will not lead to better user-perceived quality care and willingness to utilise accredited health facilities’—important to note when considering methods and instruments to use.
The need for a skilled workforce: The Nigerian baseline study43 refers to a key component of performance when noting the impact of insufficient skilled health workers on the operational efficiency of PC facilities and their ability to provide services required. Studies primarily addressing performance on features of workforce provision at the upstream PHCPI input dimension level will have been largely missed by a search strategy focused on service delivery. The importance of a sufficient and adequately trained workforce for PC performance in all domains is important to note and highlights a shortcoming of a review limited to one PHC dimension. Cost-effective PC is dependent on upstream dimensions A and B. In similar vein such resources may be wasted, and their intended outcomes not realised if essential features of PC are not available to users and applied in their care. At the service delivery (C) level, the ICRM and PCAT instruments (table 3) address availability of sufficient (C4a) and appropriately skilled (C4b) providers.
In addition to the absence of a comprehensive discussion on indicators and instruments, the studies also failed to address the potential to link validated instruments and performance results with development and implementation of interventions to strengthen PC—and ultimately PHC. Instruments and methods such as the e-TIQH, SafeCare Essentials and PCAT that involve users, providers and managers have added value given the potential to initiate user-provider-manager stakeholder collaboration during data collection as well as intervention development and implementation. Such instruments could help identify common ground among key stakeholders despite different roles and responsibilities, thereby strengthening PHC.
Addressing the knowledge gaps and research questions highlighted should be integral to PHC strengthening and health sector reform in Africa. It is important to note however that change driven by political agendas and narrow, group-based interests may ignore evidence advocating genuine reform45—particularly when evidence challenges a biomedical, diseased-based approach to care at entry to the health system. Involving key stakeholders who collaborate when measuring PC performance and applying the findings may provide checks and balances needed to reduce the impact of group-based agendas.
Africa is a resource-scarce continent challenged by a quadruple disease burden and recalcitrantly poor health indices. A bottom-up approach to performance assessment may generate innovative, inexpensive and context-appropriate interventions required. Realising such goals is more likely if instruments measuring PC performance in the field are placed in the hands (following training) of district-level stakeholders including managers, providers and users. Other stakeholders include postgraduate-trained PC physicians (family physicians) specifically trained to deliver PC. It is here in the PHC system that PC physicians have their biggest impact—applying their clinical, public health and behavioural science expertise to ensure essential PC elements are available to users and applied in their care; a role within a multidisciplinary PHC team approach being promoted by the Primafamed PHC African Network promotes.16 28 PC physicians, nurse practitioners and other members of the PHC team should be trained (jointly) to use validated instruments measuring their performance.
Although this review focused on the PHCPI service delivery dimension, research aimed at developing a compendium of cross-culturally adapted and validated instruments should be considered—to select from as needed—to rigorously and comprehensively measure PHC performance in Africa across the five PHCPI dimensions. Wide stakeholder commitment and collaboration on performance evaluation should also consider developing instruments that are country-specific where necessary and yet enable comparison of findings and experiences13 within and between African countries. Such developments could stimulate cross-border collaboration to include development and implementation of interventions and strategies that strengthen PHC, thereby reducing wasteful duplication, fragmentation and cost. Such benefits may extend beyond Africa. Development of an Africa PCAT was first proposed at a Primafamed workshop in Nairobi in 2016 (Summary report, unpublished, of PCAT workshop by G. Bresick and K. von Pressentin). The Ibero-America (IA) PCAT is an example of a regional PCAT—the result of a collaborative research project involving Ibero-American countries.49 International PCAT researchers including researchers from Africa presented their research at the 2018 World National Colleges and Academies of Family Medicine/Primary Care conference in Seoul. Areas for collaboration identified included the need to adapt the original PCAT for universal use and to establish an international PCAT research forum. This review contributes to Africa’s role in research to improve and promote PC performance research and measurement globally.
Notwithstanding the case made for a specific PC focus, the review addresses a single dimension of the multi-dimensional PHC system. It was confounded by the interchangeable use of terms in the literature—hence the use of the PHCPI conceptual framework to reduce the impact of consequent challenges. The framework is however a recent development not yet widely accepted—as indicated by its use in only one result.40 Mapping indicators onto service delivery domains and subdomains (table 2 Scope column; and table 3) was loosely done and needs closer examination. Most studies failed to define performance indicators measured, confounding alignment of results with service delivery domains and underscoring the need for collaboration among PC researchers, service providers, managers and other key stakeholders. These and other factors—such as extracting reported data from English language papers only—add to the limitations of the review. We believe nevertheless that the results reflect a dearth of studies addressing PC performance measurement in Africa and highlight key gaps in an important area of health systems research.
Health systems research conducted in Africa to date has not addressed PC performance measurement adequately despite global calls for PHC reform; the pressing need for rigorous performance measurement to guide reform; and strong evidence that comprehensive, people-centred PC improves health outcomes at reduced cost. Insufficient attention is being given to use of validated instruments. Our review findings show low use of validated instruments and limited evidence to support validation of measurement properties (reliability, validity and acceptability) of most instruments used. Excluded studies included those addressing patient perceptions and experiences of PC using locally developed instruments that were poorly described and unvalidated—arguably a waste of scarce resources. Using validated instruments to measure performance is necessary for reliable data to inform PHC quality improvement23—for example, to guide development and implementation of appropriate interventions. The paucity of studies using such instruments to measure PC comprehensively should be addressed. The knowledge and research gaps highlighted are more glaring 40 years after the Alma Ata Declaration and 10 years after the WHO call for ‘PHC: now more than ever’ and similar initiatives. The global call for UHC further underscores the need to improve health service quality—fundamental to achieving UHC and its goal of ensuring access to quality healthcare for all.25 50 51 High quality research is required for evidenced-based data to justify directing scarce resources to strengthening PHC systems—particularly in Africa; and to support decision-making generally. Health services that cannot show evidence of valid performance appraisal will be increasingly difficult to justify.52–54 Research addressing PC performance may have potential to promote more socially responsive and accountable PHC in the true spirit of the Alma Ata and recent Astana Declarations.9 10
A brief outline of this review was presented at a satellite session at the 5th Global Symposium on Health Systems Research, 8–12 October 2018, Liverpool, UK. We wish to thank Professor Lisa Hirschhorn, Professor of Medical Social Sciences, Northwestern University Feinberg School of Medicine. Chicago, for her assistance in drafting the manuscript. We also wish to thank University of Cape Town Health Sciences Faculty Library research librarians Ms Mary Shelton, Ms Saskia Vonk and Ms Dilshaad Brey for their advice and assistance in designing the Pubmed search strategy.
Handling editor Stephanie M Topp
Contributors The study outline was formulated by GB with contributions from FC, IB, MM, LD and SM. All authors participated in the initial search, selection and data extraction. GB collated the data extracted by each author, coordinated the selection process, and wrote the initial draft to which the other authors contributed. GB conducted the second search, data extraction and final study selection with feedback from the co-authors. GB and FC drafted the final manuscript. All authors approved the final manuscript.
Funding The workshop during which the NGT was conducted (methods section) was funded by the Division of Family Medicine and Primary Care, Stellenbosch University.
Competing interests As noted in the results section GB and LD were co-investigators in the validation of South African and Malawian versions of the PCAT respectively.
Patient consent for publication Not required.
Ethics approval This scoping review involved data taken from published studies; no ethical approval was required from any of the institutions represented.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.