Table 2

Summary of 19 studies that met the criteria (ordered alphabetically and publication date)

ReferenceCountryType of studyStudy aim/ purposeMethodsInstrument(s)Scope: what does it measure?*Key findings
Bresick et al. Evaluating performance of South African primary care24South AfricaPC performance measurementCross-sectional descriptive survey of South African PC performance in seven provincesPC performance determined by user, provider & manager questionnairesSouth African Primary Care Assessment Tool (ZA PCAT)PC performance as determined by users; providers & managers; performance on eight core domains; 11 sub-domains; 115 items (user PCAT) (*(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d; C5a,b,c,d,e)Users rated accessibility, continuity and community orientation as poor; coordination of information and family-centredness inadequate; and comprehensiveness, coordination of care, cultural competency and PHC team availability adequate. Managers and providers significantly more optimistic about performance than users. Strengthening PC and improving users’ experience should focus on access, continuity and community-orientated PC; and reduce gaps between user experience and provider assessment of performance
Renggli et al Toward improved health service quality in Tanzania25TanzaniaInstrument analysis and performance measurementAnalyse an electronic tool developed to assess and monitor PHC quality in TanzaniaQuantitative and qualitative methods used to investigate tool appropriateness.Tanzanian electronic Tool to Improve Quality of Healthcare (e-TIQH)A range of quality dimensions; assessment focused on processes and structural adequacy of healthcare (B2;C2b; C4b,c)Robustness checks confirmed e-TIQH’s ability to score facilities and group indicators into different dimensions. Findings show accuracy measuring and monitoring quality and steering improvement measures. A feasible option for routine measures of quality and a foundation to improve services
Dullie et al. Develop-ment and validation of Malawian PCAT26MalawiValidation of PCAT for use in MalawiDevelop the PCAT-Mw; evaluate reliability and validity; measure PC performance for adult usersForward and backward translation of ZA-PCAT before cross-sectional survey to test validity and intra-rater and inter-rater reliabilityPCAT-MwMeasures five PC domains (first contact access, continuity, comprehensiveness, coordination and community orientation); 7 subdomains and 29 items. (C1a, b; C3c, C4d, C5a, b, c, d, e; E1)Comprehensive metric analyses showed PCAT-Mw reliability and validity in assessing core PC domains based on adult users’ experience. PCAT-Mw useful for Malawian health service research
Dullie et al. Quality of primary care from patients' perspective27MalawiBaseline audit of PC in a Malawian rural health districtEvaluate PC performance based on users experience of services and association with socio-demographic characteristicsCross sectional survey to determine mean scores for total PC and PC domains; linear regression to determine socio-demographic associationsPCAT-MwMeasures PC organisation and performance on access; continuity; comprehensiveness of services available and provided; and community orientation (C1a, b; C3c, C4d, C5.a, b, c, d, e; E1)Partial functioning of Malawian PC. Performance is weak for access, relational continuity and comprehensiveness of services available. Sex, geographical location, self-rated health status, duration of association with facility and facility affiliation associated with user assessment of PC
Macarayan et al. Assessment of quality of primary care with facility surveys38Ethiopia, Kenya, Malawi, Namibia, Nepal, Rwanda, Senegal, Tanzania, UgandaPerformance assessmentAssess whether existing facility surveys capture relevant aspects of PC performance; report on PC data qualitySelected data analysed from Service Provision Assessment (SPA) surveysData analysis using survey data. Three indicators selected from SPA surveys: competent systems; evidence-based care; and user experienceMeasures aspects of user experience, health outcomes and processes such as timely action, choice of provider, affordability, ease of use, dignity, privacy, non-discrimination, autonomy and confidentiality (C3a,c; C4b,d;C5e; E1,2)Gaps in measuring user experience, health outcomes and processes—including timely action, choice of provider, affordability, ease of use, dignity, privacy, autonomy and confidentiality. No information available on care competence outside maternal and child health. PC quality scored low
von Pressentin et al. Influence of Family Physicians in a district health system28South AfricaImpact studyDetermine the influence of family physicians on PC performance to assist human resource planningCross-sectional study comparing district hospitals & PC centres with and without family physicians in South AfricaZA PCATEvaluate impact of family physicians on South African district health system. (C1;B; C5a,b,c,d,e; E2)District hospitals with family physicians scored better on indicators of health system performance and clinical processes. In contrast PC centres with family physicians had significantly poorer mean scores for continuity and coordination of care
Wisniewski et al. Comparison of objective measures and patients’ perceptions of quality of services39Democratic Republic of CongoCross-sectional; visit-level data from household & government facility surveyCompare objective measures with user perceptions of health service qualityHousehold & government facility surveysData analysis using survey dataMeasures user perception of quality related to availability of drugs and equipment; patient-centeredness; and safety compared with objective measures of quality (B1;C4e; C5e)User assessment of quality inaccurate; expectations and prior experience impacted perceptions. Future research should examine whether improving user knowledge of what to expect and transparency of facility quality data improves accuracy of user assessments
Hunter et al. The Ideal Clinic in South Africa29South AfricaReportReport progress on implement-tation research model211-item ICRM checklist of performance indicators administered to PC facilities to measure complianceStandardised checklist (ICRM) comprising five domains; 10 components; 32 sub-components each comprising a number of itemsMeasures performance on range of dimensions—mainly operational and infrastructural components that do not map easily onto the PHCPI framework for example, administration includes signage, staff identity and dress codes; record archival & retrieval. (A1c, A3a; B1,2,3 4; C3b, C4b,c,e)Progress shown; SA Health Department staff described the initiative’s progress and ongoing challenges against a 2013 baseline audit
Mukiapini et al. Baseline measure of PHC team function and overall PHC performance30South AfricaBaseline audit of PC and team functionMeasure baseline PHC team effectiveness and overall PHC performanceCross-sectional study using the Nominal Group Technique and ZA PCAT to assess team function and baseline performanceZA PCAT user, provider & manager instrumentsPC performance and organisation measured on eight domains including PHC team function; 11 sub-domains; 115 items (user PCAT) *(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a, b,c,d; C3(a),(b),c; C4a,(b),(c),d;C5a,b, c,d,ePHC team members rated team as well-functioning (70% agreement on 7-item PHC team function sub-domain) yet NGT method revealed communication and leadership as key challenges to effective function; baseline results similar to Western Cape PCAT study32
Ogaji DS et al. Development and validation of the PES for PHC31NigeriaValidation of a patient measure of PHCDevelop and validate a patient evaluation scale (PES) for use in Nigerian PHCIterative process to develop and validate questionnaire with users at PC centres used literature reviews; user and expert interviews; cognitive testing; cross-sectional surveysFull version of patient evaluation scale (PES)Items measure facility, organisation and health services provided. (B2;C1c; C3a,c E2)The PES and PES-SF (full and shortened versions; 27 and 18 items respectively) may be useful in practice and research aimed at patient evaluation, comparing performance, understanding trends and testing patient-focused improvements in PHC in Nigeria
Ogaji. Questionnaires for Patient Evaluation of PHC: a systematic review41NigeriaSystematic ReviewSystematic search for patient questionnaires evaluating PC and implications for practiceSystematic ReviewPatient questionnaires evaluating PCQuestionnaires include patient evaluation of dimension C domains and other dimensionsReview of instruments needed for patient evaluation of PC and implications for practice in Nigeria
Bresick et al. Western Cape PCAT study—PC organisation and performance32South AfricaPC performance measurementBaseline measure of comprehensive PC centres’ performance & organisationZA PCAT administered to 1432 users; 100 clinicians; 64 managers; 13 PC facilities; 10 sub-districtsZA PCAT user, provider & manager instrumentsPC performance and organisation measured on eight domains including PHC team function; 11 sub-domains; 115 items (user PCAT) *(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d;C5a,b,c,d,e)Managers scored access, family centeredness and cultural competence significantly lower than practitioners; users scored comprehensive services available, services provided, and community orientation significantly lower than practitioners and managers
Kress et al. Assessment of PHC performance using the PHCPI framework40NigeriaPHC system performance assessmentAssess PHC performance and causes of under-performanceAnalysis of facility data from World Bank Service Delivery Indicators SurveyPHCPI Conceptual Framework used as a toolExamined all PHCPI dimensions. Multiple data collection methods and tools used. (A;B;C;D;E)PHC performance hindered by segmented supply chains; lack of financial access, infra-structure, drugs, equipment and vaccines at facility level; poor health worker performance. Financing and governance identified as root causes of dysfunction
Alhassan et al. Comparison of perceived and technical healthcare quality in PHC facilities33GhanaSurvey involving users & staff (household & PHC facility)Compare perceived & technical healthcare quality in PHC facilities; determine if perceptions are associated with technical quality proxies in health facilitiesCross-sectional surveys examine differences in user and staff perceptions on quality healthcare; and determine technical quality of careSafeCare Essentials tool (Household & staff question-naires)(A1b, c; B2; C4b, e; D1 a, b, c, d, e, f)Negative association between technical quality & user-perceived quality care. Significant staff-client differences in all healthcare quality proxies suggesting information imbalance between clients & service providers. Improving technical quality alone will not lead to better user-perceived quality care and willingness to use accredited health facilities
Bresick et al. Adaptation and cross-cultural validation of the United States PCAT34South AfricaValidation of measurement toolAdapt and cross-culturally validate original adult expanded USA PCAT (AE) for use in South AfricaEvaluation of item phrasing; modified Delphi & Nominal Group Technique methods with expert panellists; user focus group to obtain consensus on content relevant to South African usersValidated ZA PCAT—PC adult user, provider & manager versionsMeasures PC performance on access, service availability, continuity, coordination, comprehensiveness, community and family orientated care; cultural sensitivity; and team function (*(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d; C5a,b,c,d,e)Consensus on inclusion of all original PCAT domains. PHC team availability and function added as a new domain. Three of 95 original items achieved <70% agreement and excluded as not relevant to SA; 19 new items added. A few items needed rephrasing for local use. ZA PCAT isi-Xhosa and Afrikaans translations produced
Babatunde et al. PHC consumers' perception of quality of care and its determinants42NigeriaCross-sectional descriptive studyMeasure PHC user satisfaction and assess user perception of quality of care and determinantsSemi-structured interviewer- administered questionnaire adapted from QUOTE (Quality of care from patient's eyes) (CMAI 2004)Adapted semistructured questionnaireUser perception of quality of care including greeting patients; respect for patients’ opinion; whether satisfied with time spent with doctor, with treatment & care received; and uninterrupted consultations (C3.a-c)Waiting time had inverse relationship (not statistically significant) with level of satisfaction. Perception of care was lowest in domains of privacy and respect for patients' opinion
Phaswana et al. Patient satisfaction with PHC services in a selected district35South AfricaCross-sectional descriptive studyMeasure patient satisfaction with PHC services in a rural districtPatient survey using adaptation of a validated user satisfaction questionnaireAdapted Health System Trust (2004) Client Satisfaction Survey QuestionnaireMeasured PC user satisfaction on eight domains: access, empathy, referral, general satisfaction, service standards, reliability, health promotion, tangibles, assurance. (elements of C2–C5)Regarding access to services: 3/4 women felt they could reach clinic by phone, get appointments, language not a barrier, treated the same day without being turned away. for 7/10 journey takes longer than 1 hour and cost more than ZAR 10
Peltzer et al. Use of the European Task Force for Patient Evaluation of General Practice (EUROPEP)36South AfricaEUROPEP psychometric property assessment for SA useTo examine psychometric properties of South African (isiXhosa) version of the EUROPEPValidation process described factor analysis done on data collected from large patient sample using the EUROPEP toolEUROPEPPatient evaluation of provider competence; patient-provider communication; and practice organisation (C2b; C4b;C5e)Exploratory factor analysis identified three subdimensions: medical technical content of care, patient–provider communication, and practice organisation. Reliability of items and sub-scales was high. The EUROPEP can be a valuable instrument to assess patients' evaluations of PHC in South Africa
Chukwuani et al. A baseline survey of the PHC system43NigeriaBaseline survey of PHCAudit PC operations; measure community perceptions and expectations of services; develop performance indicators; obtain baseline data for policy and planningRapid health facility operations audit and community surveyStructured questionnaires to collect qualitative and quantitative dataEvaluate stewardship; service provision; administrative and financial management; measure user utilisation of health services, perception of service delivery and healthcare financing (A1.b, A2.b, C2a, C2b,C4b, C4c)Lack of operational efficiency in most facilities; poor maintenance; services needed not provided; not enough skilled health workers; poor service provision. Access to essential drugs a key indicator for community evaluation of performance. Poor funding, management and infrastructure. Operations lack a budget, financial management system and policy on resource generation. Low utilisation precludes efficient PHC
  • *See table 3: PHCPI dimensions

  • ICRM, Ideal Clinic Realisation and Maintenance; PC, primary care; PCAT-Mw, Malawian South African Primary Care Assessment Tool; PHC, primary healthcare; PHCPI, primary healthcare performance initiative.