Summary of 19 studies that met the criteria (ordered alphabetically and publication date)
| Reference | Country | Type of study | Study aim/ purpose | Methods | Instrument(s) | Scope: what does it measure?* | Key findings |
| Bresick et al. Evaluating performance of South African primary care24 | South Africa | PC performance measurement | Cross-sectional descriptive survey of South African PC performance in seven provinces | PC performance determined by user, provider & manager questionnaires | South African Primary Care Assessment Tool (ZA PCAT) | PC performance as determined by users; providers & managers; performance on eight core domains; 11 sub-domains; 115 items (user PCAT) (*(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d; C5a,b,c,d,e) | Users rated accessibility, continuity and community orientation as poor; coordination of information and family-centredness inadequate; and comprehensiveness, coordination of care, cultural competency and PHC team availability adequate. Managers and providers significantly more optimistic about performance than users. Strengthening PC and improving users’ experience should focus on access, continuity and community-orientated PC; and reduce gaps between user experience and provider assessment of performance |
| Renggli et al Toward improved health service quality in Tanzania25 | Tanzania | Instrument analysis and performance measurement | Analyse an electronic tool developed to assess and monitor PHC quality in Tanzania | Quantitative and qualitative methods used to investigate tool appropriateness. | Tanzanian electronic Tool to Improve Quality of Healthcare (e-TIQH) | A range of quality dimensions; assessment focused on processes and structural adequacy of healthcare (B2;C2b; C4b,c) | Robustness checks confirmed e-TIQH’s ability to score facilities and group indicators into different dimensions. Findings show accuracy measuring and monitoring quality and steering improvement measures. A feasible option for routine measures of quality and a foundation to improve services |
| Dullie et al. Develop-ment and validation of Malawian PCAT26 | Malawi | Validation of PCAT for use in Malawi | Develop the PCAT-Mw; evaluate reliability and validity; measure PC performance for adult users | Forward and backward translation of ZA-PCAT before cross-sectional survey to test validity and intra-rater and inter-rater reliability | PCAT-Mw | Measures five PC domains (first contact access, continuity, comprehensiveness, coordination and community orientation); 7 subdomains and 29 items. (C1a, b; C3c, C4d, C5a, b, c, d, e; E1) | Comprehensive metric analyses showed PCAT-Mw reliability and validity in assessing core PC domains based on adult users’ experience. PCAT-Mw useful for Malawian health service research |
| Dullie et al. Quality of primary care from patients' perspective27 | Malawi | Baseline audit of PC in a Malawian rural health district | Evaluate PC performance based on users experience of services and association with socio-demographic characteristics | Cross sectional survey to determine mean scores for total PC and PC domains; linear regression to determine socio-demographic associations | PCAT-Mw | Measures PC organisation and performance on access; continuity; comprehensiveness of services available and provided; and community orientation (C1a, b; C3c, C4d, C5.a, b, c, d, e; E1) | Partial functioning of Malawian PC. Performance is weak for access, relational continuity and comprehensiveness of services available. Sex, geographical location, self-rated health status, duration of association with facility and facility affiliation associated with user assessment of PC |
| Macarayan et al. Assessment of quality of primary care with facility surveys38 | Ethiopia, Kenya, Malawi, Namibia, Nepal, Rwanda, Senegal, Tanzania, Uganda | Performance assessment | Assess whether existing facility surveys capture relevant aspects of PC performance; report on PC data quality | Selected data analysed from Service Provision Assessment (SPA) surveys | Data analysis using survey data. Three indicators selected from SPA surveys: competent systems; evidence-based care; and user experience | Measures aspects of user experience, health outcomes and processes such as timely action, choice of provider, affordability, ease of use, dignity, privacy, non-discrimination, autonomy and confidentiality (C3a,c; C4b,d;C5e; E1,2) | Gaps in measuring user experience, health outcomes and processes—including timely action, choice of provider, affordability, ease of use, dignity, privacy, autonomy and confidentiality. No information available on care competence outside maternal and child health. PC quality scored low |
| von Pressentin et al. Influence of Family Physicians in a district health system28 | South Africa | Impact study | Determine the influence of family physicians on PC performance to assist human resource planning | Cross-sectional study comparing district hospitals & PC centres with and without family physicians in South Africa | ZA PCAT | Evaluate impact of family physicians on South African district health system. (C1;B; C5a,b,c,d,e; E2) | District hospitals with family physicians scored better on indicators of health system performance and clinical processes. In contrast PC centres with family physicians had significantly poorer mean scores for continuity and coordination of care |
| Wisniewski et al. Comparison of objective measures and patients’ perceptions of quality of services39 | Democratic Republic of Congo | Cross-sectional; visit-level data from household & government facility survey | Compare objective measures with user perceptions of health service quality | Household & government facility surveys | Data analysis using survey data | Measures user perception of quality related to availability of drugs and equipment; patient-centeredness; and safety compared with objective measures of quality (B1;C4e; C5e) | User assessment of quality inaccurate; expectations and prior experience impacted perceptions. Future research should examine whether improving user knowledge of what to expect and transparency of facility quality data improves accuracy of user assessments |
| Hunter et al. The Ideal Clinic in South Africa29 | South Africa | Report | Report progress on implement-tation research model | 211-item ICRM checklist of performance indicators administered to PC facilities to measure compliance | Standardised checklist (ICRM) comprising five domains; 10 components; 32 sub-components each comprising a number of items | Measures performance on range of dimensions—mainly operational and infrastructural components that do not map easily onto the PHCPI framework for example, administration includes signage, staff identity and dress codes; record archival & retrieval. (A1c, A3a; B1,2,3 4; C3b, C4b,c,e) | Progress shown; SA Health Department staff described the initiative’s progress and ongoing challenges against a 2013 baseline audit |
| Mukiapini et al. Baseline measure of PHC team function and overall PHC performance30 | South Africa | Baseline audit of PC and team function | Measure baseline PHC team effectiveness and overall PHC performance | Cross-sectional study using the Nominal Group Technique and ZA PCAT to assess team function and baseline performance | ZA PCAT user, provider & manager instruments | PC performance and organisation measured on eight domains including PHC team function; 11 sub-domains; 115 items (user PCAT) *(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a, b,c,d; C3(a),(b),c; C4a,(b),(c),d;C5a,b, c,d,e | PHC team members rated team as well-functioning (70% agreement on 7-item PHC team function sub-domain) yet NGT method revealed communication and leadership as key challenges to effective function; baseline results similar to Western Cape PCAT study32 |
| Ogaji DS et al. Development and validation of the PES for PHC31 | Nigeria | Validation of a patient measure of PHC | Develop and validate a patient evaluation scale (PES) for use in Nigerian PHC | Iterative process to develop and validate questionnaire with users at PC centres used literature reviews; user and expert interviews; cognitive testing; cross-sectional surveys | Full version of patient evaluation scale (PES) | Items measure facility, organisation and health services provided. (B2;C1c; C3a,c E2) | The PES and PES-SF (full and shortened versions; 27 and 18 items respectively) may be useful in practice and research aimed at patient evaluation, comparing performance, understanding trends and testing patient-focused improvements in PHC in Nigeria |
| Ogaji. Questionnaires for Patient Evaluation of PHC: a systematic review41 | Nigeria | Systematic Review | Systematic search for patient questionnaires evaluating PC and implications for practice | Systematic Review | Patient questionnaires evaluating PC | Questionnaires include patient evaluation of dimension C domains and other dimensions | Review of instruments needed for patient evaluation of PC and implications for practice in Nigeria |
| Bresick et al. Western Cape PCAT study—PC organisation and performance32 | South Africa | PC performance measurement | Baseline measure of comprehensive PC centres’ performance & organisation | ZA PCAT administered to 1432 users; 100 clinicians; 64 managers; 13 PC facilities; 10 sub-districts | ZA PCAT user, provider & manager instruments | PC performance and organisation measured on eight domains including PHC team function; 11 sub-domains; 115 items (user PCAT) *(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d;C5a,b,c,d,e) | Managers scored access, family centeredness and cultural competence significantly lower than practitioners; users scored comprehensive services available, services provided, and community orientation significantly lower than practitioners and managers |
| Kress et al. Assessment of PHC performance using the PHCPI framework40 | Nigeria | PHC system performance assessment | Assess PHC performance and causes of under-performance | Analysis of facility data from World Bank Service Delivery Indicators Survey | PHCPI Conceptual Framework used as a tool | Examined all PHCPI dimensions. Multiple data collection methods and tools used. (A;B;C;D;E) | PHC performance hindered by segmented supply chains; lack of financial access, infra-structure, drugs, equipment and vaccines at facility level; poor health worker performance. Financing and governance identified as root causes of dysfunction |
| Alhassan et al. Comparison of perceived and technical healthcare quality in PHC facilities33 | Ghana | Survey involving users & staff (household & PHC facility) | Compare perceived & technical healthcare quality in PHC facilities; determine if perceptions are associated with technical quality proxies in health facilities | Cross-sectional surveys examine differences in user and staff perceptions on quality healthcare; and determine technical quality of care | SafeCare Essentials tool (Household & staff question-naires) | (A1b, c; B2; C4b, e; D1 a, b, c, d, e, f) | Negative association between technical quality & user-perceived quality care. Significant staff-client differences in all healthcare quality proxies suggesting information imbalance between clients & service providers. Improving technical quality alone will not lead to better user-perceived quality care and willingness to use accredited health facilities |
| Bresick et al. Adaptation and cross-cultural validation of the United States PCAT34 | South Africa | Validation of measurement tool | Adapt and cross-culturally validate original adult expanded USA PCAT (AE) for use in South Africa | Evaluation of item phrasing; modified Delphi & Nominal Group Technique methods with expert panellists; user focus group to obtain consensus on content relevant to South African users | Validated ZA PCAT—PC adult user, provider & manager versions | Measures PC performance on access, service availability, continuity, coordination, comprehensiveness, community and family orientated care; cultural sensitivity; and team function (*(A1b,c); (A3a,b); (B4); C1a,b,(d); C2a,b,c,d; C3(a),(b),c; C4a,(b),(c),d; C5a,b,c,d,e) | Consensus on inclusion of all original PCAT domains. PHC team availability and function added as a new domain. Three of 95 original items achieved <70% agreement and excluded as not relevant to SA; 19 new items added. A few items needed rephrasing for local use. ZA PCAT isi-Xhosa and Afrikaans translations produced |
| Babatunde et al. PHC consumers' perception of quality of care and its determinants42 | Nigeria | Cross-sectional descriptive study | Measure PHC user satisfaction and assess user perception of quality of care and determinants | Semi-structured interviewer- administered questionnaire adapted from QUOTE (Quality of care from patient's eyes) (CMAI 2004) | Adapted semistructured questionnaire | User perception of quality of care including greeting patients; respect for patients’ opinion; whether satisfied with time spent with doctor, with treatment & care received; and uninterrupted consultations (C3.a-c) | Waiting time had inverse relationship (not statistically significant) with level of satisfaction. Perception of care was lowest in domains of privacy and respect for patients' opinion |
| Phaswana et al. Patient satisfaction with PHC services in a selected district35 | South Africa | Cross-sectional descriptive study | Measure patient satisfaction with PHC services in a rural district | Patient survey using adaptation of a validated user satisfaction questionnaire | Adapted Health System Trust (2004) Client Satisfaction Survey Questionnaire | Measured PC user satisfaction on eight domains: access, empathy, referral, general satisfaction, service standards, reliability, health promotion, tangibles, assurance. (elements of C2–C5) | Regarding access to services: 3/4 women felt they could reach clinic by phone, get appointments, language not a barrier, treated the same day without being turned away. for 7/10 journey takes longer than 1 hour and cost more than ZAR 10 |
| Peltzer et al. Use of the European Task Force for Patient Evaluation of General Practice (EUROPEP)36 | South Africa | EUROPEP psychometric property assessment for SA use | To examine psychometric properties of South African (isiXhosa) version of the EUROPEP | Validation process described factor analysis done on data collected from large patient sample using the EUROPEP tool | EUROPEP | Patient evaluation of provider competence; patient-provider communication; and practice organisation (C2b; C4b;C5e) | Exploratory factor analysis identified three subdimensions: medical technical content of care, patient–provider communication, and practice organisation. Reliability of items and sub-scales was high. The EUROPEP can be a valuable instrument to assess patients' evaluations of PHC in South Africa |
| Chukwuani et al. A baseline survey of the PHC system43 | Nigeria | Baseline survey of PHC | Audit PC operations; measure community perceptions and expectations of services; develop performance indicators; obtain baseline data for policy and planning | Rapid health facility operations audit and community survey | Structured questionnaires to collect qualitative and quantitative data | Evaluate stewardship; service provision; administrative and financial management; measure user utilisation of health services, perception of service delivery and healthcare financing (A1.b, A2.b, C2a, C2b,C4b, C4c) | Lack of operational efficiency in most facilities; poor maintenance; services needed not provided; not enough skilled health workers; poor service provision. Access to essential drugs a key indicator for community evaluation of performance. Poor funding, management and infrastructure. Operations lack a budget, financial management system and policy on resource generation. Low utilisation precludes efficient PHC |
*See table 3: PHCPI dimensions
ICRM, Ideal Clinic Realisation and Maintenance; PC, primary care; PCAT-Mw, Malawian South African Primary Care Assessment Tool; PHC, primary healthcare; PHCPI, primary healthcare performance initiative.