Article Text
PDF
Abstract
Healthcare reforms are often not coupled with a relevant and appropriate monitoring framework, leaving policymakers and the public without evidence about the implications of such reforms.
Kazakhstan has embarked on a large-scale reform of its healthcare system in order to achieve Universal Health Coverage. The health-related 2020 Strategic Development Goals reflect this political ambition. In a case-study approach and on the basis of published and unpublished evidence as well as personal involvement and experience (A) the indicators in the 2020 Strategic Development Goals were assessed and (B) a ‘data-mapping’ exercise was conducted, where the WHO health system framework was used to describe the data available at present in Kazakhstan and comment on the different indicators regarding their usefulness for monitoring the current health-related 2020 Strategic Development Goals in Kazakhstan.
It was concluded that the country’s current monitoring framework needs further development to track the progress and outcomes of policy implementation. The application of a modified WHO/World Bank/Global Fund health system monitoring framework was suggested to examine the implications of recent health sector reforms. Lessons drawn from the Kazakhstan experience on tailoring the suggested framework, collecting the data, and using the generated intelligence in policy development and decision-making can serve as a useful example for other middle-income countries, potentially enabling them to fast-track developments in the health sector.
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Key questions
What is already known about this topic?
Healthcare reforms are always complex and thus require feedback about their impact.
Healthcare policy initiatives are often not coupled with a relevant and applicable monitoring framework.
What are the new findings?
The existing national monitoring concept was reviewed and found to have substantial shortcomings in the health policy aspects covered and the usefulness of indicators.
A simplified framework was presented, coupling national needs and the comprehensive monitoring framework developed by the WHO, the World Bank and the Global Fund.
Recommendations for policy
Initiate a dialogue about the appropriateness of the suggested framework with national counterparts, develop national capabilities and apply the simplified framework to feedback results.
Introduction
Reforming a healthcare system is always a complex undertaking, characterised by conflicting interests and limited evidence, and it is often not known whether well-intended initiatives will improve people's lives.1
Since such complexity and uncertainty can be found in most development and reform projects, the United Nations has launched a large-scale ‘data revolution for sustainable development’ in order to provide the right information for designing, monitoring and evaluating effective policies.2 The WHO, in collaboration with the World Bank and the Global Fund, published a comprehensive operational framework on the monitoring and evaluation of health systems strengthening.3 In 2010, the WHO presented ‘Monitoring the Building Blocks of Health Systems: A Handbook of Indicators and Their Measurement Strategies’.4 In 2014, the WHO and The World Bank published a framework for monitoring Universal Health Coverage (UHC) as part of a comprehensive framework for monitoring national health system performance at national and international levels with guiding principles and methodological considerations.5 The need for encompassing Civil Registration and Vital Statistics as a core component of sustainable development has also been emphasised.6 The importance of effective and comprehensive measurement of change in health systems is increasingly being recognised.
In this paper, the health reform experience in Kazakhstan is used as a case study to explore what data are needed for the meaningful monitoring of reforms. The findings contribute to the global debate on measurement and data production for development in the health sector.
The following section (Theoretical background and methods) provides a theoretical background both for monitoring the implications of policy changes and the case study approach taken. Context: healthcare in Kazakhstan section then sets the context, giving a brief overview of the Kazakh health system and recent reforms. Next, the findings of the monitoring framework as applicable to Kazakhstan and available data are presented (section Findings). The discussion (section Discussion) presents the development of a comprehensive yet manageable monitoring and assessment framework. Finally, in the Conclusion section, we conclude what could be learnt from the Kazakh experience and provide the outlook for further developments.
Theoretical background and methods
Monitoring the implications of policy changes—a framework
Given that health policy is frequently not evidence-driven,7 policymaking regarding the provision and financing of healthcare is often as much based on the perceptions of politicians, the influence of advisors and personal empirical evidence from healthcare systems elsewhere as it is on rigorous analysis and systematic identification of strengths and weaknesses of the status quo. Furthermore, the ‘measurement of success’ of health financing and delivery arrangements is still in its infancy.1 Figure 1 depicts the most general approach to looking at a healthcare system, starting from ideas and taking outcomes as the final goal.
Ideas, interests and institutions in healthcare. Source: Modified after.1
Several approaches have been developed to establish a conceptual framework with a clear distinction between objectives and instruments, that is, between what the system does and what the system is trying to achieve.8 One of the most widely used frameworks is based on (A) the WHO's six building blocks of any healthcare system, a mix of inputs, processes and management issues4 and (B) the core functions of a health system: Improved health (both level and equity), responsiveness and financial protection (see figure 2).9 ,10 This framework is simple yet allows one to capture structures, processes and results.
The WHO Health System Framework with building blocks and goals outcomes. Source: Adopted from.4
Formal progress towards UHC assessed by intermediate goals will not automatically mean better population health,11 so measuring effects at the population level will be as important as measuring intermediate indicators like equitable financing and access.12 ,13 Resonating with its strategy of consolidating and unifying data collection and evaluation for health systems worldwide, the WHO in collaboration with the World Bank and the Global Fund in 2009 published a comprehensive operational framework on the monitoring and evaluation of health systems strengthening based on the International Health Partnership (IHP+) common evaluation framework (figure 3).3 ,5
The results framework for health system strengthening monitoring and evaluation. Source: Modified from.3
This concept was taken up again in 2010 with a handbook on suitable indicators and measurement strategies ‘for monitoring performance and evaluating progress in the scale-up for better health’.4
The recent UN initiative on the ‘data revolution’ has put forward the importance of data for sustainable development. ‘Data are the lifeblood of decision-making and the raw material for accountability. Without high-quality data providing the right information on the right things at the right time; designing, monitoring and evaluating effective policies becomes almost impossible’.2
Method: the case study approach
We use the case study approach14 primarily to describe the current context and need for policy monitoring in Kazakhstan. On the basis of our personal involvement and experience, we (A) assessed the indicators in relation to national policy in Kazakhstan and (B) conducted a data-mapping exercise, where we used the WHO health system framework to describe the data available in Kazakhstan and to comment on their usefulness in augmenting the health-related 2020 Strategic Development Goals in Kazakhstan.
‘The case study approach allows in-depth, multi-faceted explorations of complex issues in their real-life settings.’14 It has only recently become more widely used in health service research. Early examples of the case study approach in health policy have been more descriptive with little theory15 or have emphasised the personal involvement and experience of the authors.16 Later research has tried to provide a more nuanced theoretical grounding and combined methods from the social sciences with an epistemological reflection and established processes for ensuring rigour in case study and qualitative data collection and analysis.17 ,18 We used publicly available data together with non-classified information that has not been published. In addition, we conducted a literature search in PubMed, Google Scholar and general Internet search engines with the combination of key words ‘Kazakhstan’ AND ‘public health’ OR ‘health care’ OR ‘epidemiology’ AND ‘data’ OR ‘monitoring’.
The case study team based the assessment of the current health-related indicators in the national 2020 Strategic Development Goals on three categories. A traffic light system was used with red indicating that the target could only be assessed subjectively and would thus be prone to discussion and criticism; yellow indicated that some element of verifiable assessment was possible, whereas green indicated clear (and often easy) measurement. To the best of our knowledge, such an assessment has not been done elsewhere, but this categorisation is self-explanatory and can easily be replicated. There is a need for such assessment of usefulness and measurability because unspecific, unclear and simply unmeasurable indicators are counterproductive.
Context: healthcare in Kazakhstan
Kazakhstan is a middle-income country that aims to become a member of the Organization for Economic Cooperation and Development (OECD) by 2050. The national strategy ‘Kazakhstan 2050’ has set strategic goals for further development of the country in all areas of social and economic development.19 It is one of the least densely populated countries in the world with only six people per square kilometre on average. Kazakhstan's Gross Domestic Product (GDP) was United States Dollar (USD) 218 billion, or USD 12 600 per capita, in 2014.20 The country's GDP is to a large degree dependent on mineral resources, and thus the economy is particularly prone to external shocks. However, the country has a low public debt of just 12% of GDP.21
Kazakhstan's current state of health and health care provision
Compared to countries in the OECD and even fellow post-Soviet states, key population health indicators in Kazakhstan are at the lower end (see table 1). However, the government of Kazakhstan has acknowledged this and has made progress in the field of healthcare provision an explicit target of state policy.22
Population health indicators in selected former Soviet Union countries/Eastern European countries
The country inherited a Semashko-style healthcare system, which has an integrated approach to providing and financing care with primary healthcare and the district physician at the centre, a strong referral system and an extensive hospital network. From 1970, the system was not able to respond to demand for better quality care and new medical technologies. A move towards specialisation in outpatient care led to a much-reduced role of the district physician and direct access to specialist care, but this resulted in unmet demand for quality care.23 At the same time, despite substantial reductions in the past decade, hospitalisation rates remain well above OECD levels.24 Very soon after independence from the Soviet Union in 1991, private medical practice was allowed in Kazakhstan and the country started an ill-fated attempt to introduce mandatory health insurance (1996–1998).25 Since 2000, the country has initiated two comprehensive reform programmes: the National Programme for Health Care Reform and Development 2005–2010 and the State Health Care Development Programme for 2011–2015 (‘Salamatty Kazakhstan’) with the specific aims of reducing the size of the hospital sector and strengthening primary healthcare.25
Funding health care
At present, health funds are collected by the central government through general taxation and then either used for paying providers directly or transferred to regional authorities to cover medical services for ‘socially important diseases’ (like tuberculosis). In addition, there are specific healthcare budgets at the local authority level.25
National level budget expenditures amount to 62.9% and local budget expenditures amount to 37.1% of total government health expenditures (2013 National Health Accounts (NHA) by the Ministry of Healthcare and Social Development and Republican Centre for Health Development, latest NHA not yet published). When comparing Kazakhstan's expenditure on health with other Eastern European, post-Soviet countries or the OECD average, Kazakhstan stands out as a country that spends a relatively low share of its GDP on health (see table 2). It seems that the provision of universal high quality services with this level of funding might pose a serious challenge.
Total health expenditures in selected Former Soviet Union countries/Eastern European countries and OECD average (2012 Data)
The government of Kazakhstan is aware of the below-average healthcare spending. A substantial increase in the health budget as well as a comprehensive budget review has been identified as one of the key factors for improving financial protection and better service levels in the Kazakh health sector.26
Healthcare reform
There is a strong political will to pursue health sector reform. With multiple levels of immediate (Salamatty Kazakhstan 2011–2015), medium-term (Vision 2020 and 2030) and long-term (Kazakhstan 2050 Strategy) policy plans,22 there is also a clear time frame with defined objectives for health sector reform. While reform efforts are aimed at all building blocks of the healthcare system (K Obermann, T Chanturidze, B Tokezhanov, et al. Reforming the health system in Kazakhstan: A review based on WHO's concept of ‘building blocks’. Manuscript submitted), one central theme is the introduction of shared responsibility for health.27 In this context, shared responsibility for health refers to the idea that workers, employers and the government all participate equally in the financing of healthcare and the maintenance of public health and healthy lifestyles. A key reform element is the plan to introduce a mandatory contribution component, akin to a SHI, into the current tax-financed health system. The country is not seeking to introduce SHI with the objective of completely redesigning the financing of its healthcare system, but rather the augmentation of an existing healthcare system with elements of a SHI scheme.28 If the reforms are to maintain their momentum and culminate in sustainable improvements, dedicated measurement of change is needed as well as timely interpretation of the results that allows these changes to be explained and suggests future action.
Findings
The government of Kazakhstan released a ‘National Health Strategy 2020’ approved by Presidential Decree No. 922 of 1 February 201022 ,25 with the primary aim of improving the accessibility and quality of health services. This is to be done via specific healthcare investment plans, broader health financing and more efficient provider payment systems. Healthy lifestyles are supposed to be an integral part of overall health policy. The Kazakh government is determined to measure the impact of its policies and has detailed its ambitions in the ‘Program Realisation Indicators’ of the Salamatty Kazakhstan Programme.
The goals of the strategy are reasonable and could be expected to result in improvements in the health of the population. However, looking at the concrete targets defined in the strategy, it becomes clear that they are for the most part rather imprecise. Most aim only for an increase or decrease in a certain indicator or the implementation of a policy, without specifying the current baseline and the degree of improvement that is desired. For example, internal data show that one of the productivity indicators of the NHA, the ‘number of operations per 1 million Tenge’, had a tendency to fall between 2010 and 2013. This could either be interpreted as a drop in efficiency or, quite to the contrary, be connected to the improvement of hospital infrastructure, allowing for more complex and expensive operations. A detailed technical analysis is required to ensure that policymakers interpret these indicators appropriately.
The assessment of health-related indicators in the 2020 Strategic Development Goals is based on their usefulness and measurability as described above (red: target can only be assessed subjectively and will thus be prone to discussion and criticism; yellow: some element of verifiable assessment is possible; green: clear—and often easy—measurement) and is presented in table 3.
Kazakhstan's health-related 2020 Strategic Development Goals
Whereas population health is well covered overall, there is no mention of equity between different population groups. The possible financial burden when seeking healthcare is also not covered in the national health strategy. However, both issues are extensively discussed in the social health insurance (SHI) concept and the draft SHI Law, as submitted to the Parliament, specifically aims to address financial equity in the system.
In the data mapping exercise (see table 4), we describe the data available at present in Kazakhstan and, using the available evidence, comment on the different indicators regarding their usefulness to augment the current health-related 2020 Strategic Development Goals in Kazakhstan. Data are in principle available in Kazakhstan for most of the measurements recommended by the WHO. Data are currently lacking for the General and Specific Service Readiness Scores and for the Health Information System Performance Index. A Health Facility Assessment has been conducted in order to obtain data on the Readiness Scores, but as of October 2015 no data were available. These data would also provide up-to-date information on the availability of essential medicines, which is a key objective of the 2020 Strategic Development Goals, and would allow for regional differentiation of workforce and infrastructure data. The Health Information System Performance Index so far has not been used.
Monitoring the implications of health policy changes in Kazakhstan based on the WHO health system framework: a summary of available data and suggestions for the health-related 2020 Strategic Development Goals
Discussion
This case study indicates that the standard WHO/World Bank/Global Fund health policy monitoring framework needs to be tailored to the country context. At the same time, the health-related indicators presented in the Kazakh 2020 Strategy have to be refined to attain consistency in quality, usefulness and measurability. On the basis of these findings, a number of indicators, quantitative and qualitative, that would fulfil SMART criteria (specific, measurable, action-oriented, realistic, timely) can be selected. In some instances, perception-based qualitative data would be most appropriate. This mix of approaches and methods would provide as broad a picture as possible. In addition, equity measurements comparing the data for the richest versus the poorest 20% of the population could be incorporated. Roberts et al,29 in a similar vein, proposed a ‘step pyramid’ that allows one to disaggregate the UHC cube.
A simplified WHO/World Bank/Global Fund health policy monitoring framework would assist in examining the effects of Kazakhstan's ambitious health policy reforms (figure 4).
The modified WHO health system monitoring tool: Assessing inputs and processes, outputs, outcomes and impact. Source: Authors.
This approach incorporates the WHO health system framework, but differs from the WHO/World Bank/Global Fund common evaluation concept in three important points as (1) it does not primarily focus on international comparability, but rather looks at monitoring for the needs of the country, (2) it takes a broader view on the key input parameters (like staff, IT, governance) rather than the composite measures of service coverage and financial protection and (3) it views health equity as a cross-cutting indicator that should be looked at throughout the delivery chain rather than assessing equity only at the outcomes/impact level. Ambiguous indicators, for which it would be difficult to define specific targets, were left out. This framework can then be reapplied to the health-related Strategic Development Goals 2020 in Kazakhstan to provide a revised monitoring concept (see table 5).
A revised monitoring concept for the health-related Strategic Development Goals 2020 in Kazakhstan
In addition to strengthening/implementing the General and Specific Service Readiness Scores and the Health Information System Performance Index, a comprehensive household survey would allow for more precise health expenditure and utilisation data, stratified by the wealth quintile, place of residence, ethnicity and gender, which in turn could guide health policy when it comes to specifically addressing equity issues.
As for measuring, estimating and monitoring risk factors, the WHO's STEPwise approach to non-communicable disease risk factor surveillance (STEPS, see http://www.who.int/chp/steps/riskfactor/en/) is a well-established instrument for obtaining core data on risk factors that determine the burden of disease. The STEPS instrument includes a questionnaire as well as physical and biochemical measurements and provides concrete advice on surveillance planning, adapting and implementing the instrument.
Owing to its simple structure and comprehensive approach, the adopted WHO/World Bank/Global Fund health system monitoring tool (see figure 4) seems suitable to determine whether reforms are aimed at the key system features needed for the provision of quality healthcare. However, such a monitoring concept has so far not been used to evaluate reform agendas with respect to their coverage of essential health system features. This novel approach derived from the Kazakh experience should be tried and tested in other contexts as there is a clear gap between policy initiatives and measuring the effects they have.
It is both a scientific and a political question to define what constitutes an improvement and what goals should ideally be achieved by which means. The proposed framework should thus be viewed as a suggestion and stimulus for health policy debate in order to support policymakers and administrators in working for better health, more equitable health and better financial protection while taking into account the non-medical needs of the population.
Conclusion
Kazakhstan's goal to provide universal healthcare coverage for its population is laudable and much needed, but challenging to achieve. While reform efforts have been under way for some time, overall population health indicators still show significant room for improvement. A key philosophy that Kazakhstan hopes to put into action is the concept of ‘shared responsibility for health’. The current plans for imbedding mandatory SHI, in a primarily tax-funded health system, would formally demonstrate the ideal of this concept (although a tax-based system is also based on the idea of solidarity). At the same time, the proposed SHI reform could potentially tackle many of the current challenges faced by Kazakhstan's health system. It might stimulate an increase in total health expenditure, bringing it closer to levels required to meet Kazakhstan's healthcare objectives. It may also prove to be a major step forward in improving the purchase and provision of healthcare under the general notion of a shared responsibility for health. A stand-alone health financing reform, however, will not resolve the numerous challenges in the sector. The Government will need to assess and review the service organisation and delivery practices and, in some cases, build on the best practices of the ‘Semashko’ system simultaneously. For example, strengthening the polyclinic model might be a valuable element in improving the provision of care,23 as well as in building up strong public health systems on the concept of the sanitary-epidemiological system, used throughout the former Soviet Union.30
In this complex endeavour, the inability to measure progress towards policy objectives will seriously hamper visualising and effectively communicating the value and implications of the proposed reforms. The modified WHO/World Bank/Global Fund health system analysis framework could be linked to the country's 2020 Strategic Development Plan to form the basis for a revised monitoring framework including new potential data sources and possible measures for equity.
Making full use of such a framework, however, requires primary collection as well as tailor-made studies and surveys that will require substantial capacity development. If the Government of Kazakhstan wants to assess and publish the impact of health financing reforms, resources need to be clearly designated to fine-tune and use these instruments. Systematic monitoring, assessment and transparency will be key technical aspects for visualising and communicating the implications of health financing reforms with respect to tangible health benefits for the population. This has been successfully implemented in other settings such as Bangladesh.31 Since Kazakhstan is a vast country with a partially but not fully developed infrastructure for health information management systems, there is potential to leap forward in embracing information technologies in a way that might go well beyond current uses.32
Keeping equity in the picture throughout assessing the delivery process, financing and overall results is an important part of health system analysis (see for examplean analysis on the effects of introducing national health insurance in Taiwan33 and Thailand.34 By setting more concrete overall goals and incremental targets, with clear definitions for success and/or failure, Kazakhstan's government would be able to continuously track progress and bottlenecks in the reform process. Reporting positive changes can help to attain a broad acceptance of health policy initiatives in the country as reported from Georgia.35
Recent international and regional developments (economic slowdown in neighbouring Russia/China and conflict in Ukraine) as well as the drop in global oil and gas prices36 have had a major influence on the country's economy. The GDP has fallen from US$231 billion in 2013 to US$218 billion in 2014,20 and this has led Kazakhstan to revise its growth targets downward as well as reviewing the state budget in order to achieve fiscal consolidation—this is a time when a government's commitment to long-term and people-centred healthcare reform is needed most. A well thought-of monitoring framework and data-driven decision-making could aid in getting the most out of limited resources.
Findings from the Kazakhstan case study, with a proposed monitoring framework tailored to the country’s needs, could also potentially be relevant for many other middle-income countries, as it shows that with an appropriate framework, data could be put together in a policy-relevant manner, gaps could more easily be identified and a quantitative basis could be provided for measuring and assessing the impact of health policy reform. However, indicators can only describe change, not explain change. Consequently, an evaluation approach should aim to learn from the reforms as they are implemented, and feed this back into policy development.
References
Footnotes
Handling editor Seye Abimbola
Acknowledgements KO and TC developed the idea for this paper and wrote the first draft. ST, MS and AN reviewed the paper and provided additional input/evidence. KO, TC and ER revised the paper and put it into its current format. All authors then finally reviewed and approved the paper.
Funding This paper was supported by Oxford Policy Management under the World Bank contract ‘Development and institutional sustainability of health policy analysis, provider payment system and strengthening single payer capacity; Kazakhstan; 2014–2015; Contract No: KHSTTIRP-A1/CS-02’.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.