Article Text
PDF
Abstract
Accurate and timely health information is an essential foundation for strengthening health systems. Data for decision making (DDM) is a training curriculum designed to enhance capacity of health department staff to capture and use high-quality data to address priority health issues. In 2013, the Pacific Public Health Surveillance Network adapted and piloted the DDM curriculum as an ‘at work, from work, for work’ field epidemiology training programme component for low-income and middle-income Pacific Island jurisdictions. Based on lessons learned from the pilot, we made several innovations, including delivery on-site at each district (rather than bringing trainees to a central location), conducting pre-DDM consultations and ongoing contact with health leaders across the programme, taking more care in selecting trainees and enrolling a larger cohort of students from within each health department. The decentralised programme was delivered in-country at four sites (both at national and state levels) in the Federated States of Micronesia. Following delivery, we performed an external evaluation of the programme to assess student outcomes, benefits to the health department and general programme effectiveness. Of the 48 trainees who completed all four classroom modules, 40 trainees participated in the evaluation. Thirty-two of these trainees completed the programme’s capstone field project. Eighteen of these projects directly contributed to changes in legislation, revised programme budgets, changes in programme strategy to augment outreach and to target disease and risk factor ‘hot spots’.
- epidemiology
- public health
- health systems
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
Field epidemiology training is an excellent way to build health information-related skills among health workers in low-income and middle-income countries. However, training programmes in themselves do not necessarily bring about improvement in public health practice, programmes, systems and policies.
Innovations in training programme design were successful in producing substantial numbers of graduates at both national and state levels in the Federated States of Micronesia, and in producing field projects leading to changes in public health practice, programmes and policies related to national priorities.
Explicit attention to engagement of health leaders and work teams, integration of training programmes with larger public health capacity building efforts, and on-site delivery of training at both national and state levels can maximise impact of field epidemiology training.
Introduction
In-service field epidemiology training activities are widely used to address gaps in the epidemiology workforce in lower-income countries. These programmes have documented success in producing trained workers who are capable of producing analysis worthy of publication in peer-reviewed journals.1 2 While a few have improved detection of disease outbreaks and response, most have not documented changes in organisational structures, policies and processes.3–6 Potential reasons for limited impact include the lack of ownership of the training programmes by health leaders; resistance to applying ‘student projects’ to the serious work of health departments; the difficulties in selecting the right candidates and field projects which are both feasible and of high priority; the tendency for multitasked candidates to be pulled away from course work to address other tasks; the shortage of expert, motivated project mentors and the lack of career pathways for graduates.4 7–11 These challenges are magnified for countries in the Pacific, owing to their geographic isolation, lack of economies of scale and limited human resource pools.
During 2013 and 2015, the Pacific Public Health Surveillance Network (PPHSN), a partnership of Pacific island ministries of health and key Pacific technical agencies, modified an existing data for decision making (DDM) curriculum to broaden the focus from outbreaks to general use of health information and to encourage an ‘at-work, for-work, from-work’ emphasis with interactive hands-on projects relevant to trainees’ workplaces.12–14 The revised DDM consists of four core courses plus a capstone field epidemiology project that is developed across the entire programme:
Introduction to Field Epidemiology.
Public Health Surveillance.
Outbreak Investigation & Management.
Computing for Public Health Practice.
Field Epidemiology Project.
The week-long modules are given at 2–3 month intervals, except for the Field Project, which spans the full programme over 6–12 months. The curriculum is credited by Fiji National University (FNU) as a postgraduate certificate in field epidemiology, and was piloted by epidemiologists from PPHSN partner agencies with a cohort of 15 health staff drawn from six Pacific countries who were brought together for each module.15 While the new training materials were successful in meeting learning objectives, several weaknesses were evident. In particular, very few students completed their capstone field projects and those that were completed had very little impact on public health action. In addition, some trainees lacked basic needed academic skills and having only one or two trainees from each country limited impact; costs from travel of trainees to a central location were also high.15
In response, several changes were made, aimed at increasing engagement of health leaders in trainees’ home health agencies, assuring usefulness of trainee field projects and selecting motivated and able candidates. These changes were applied to a new round of DDM training in the Federated States of Micronesia (FSM). The FSM has 105 000 people inhabiting 607 islands grouped into four states (Chuuk, Kosrae, Pohnpei and Yap) spread across 2700 km in the western Pacific. The states deliver direct health services while the national department handles coordination, national-level surveillance and standards setting. DDM in the FSM had two goals: (1) to improve the application of health data at national and state levels and (2) to equip a critical mass of health staff with mid-level epidemiology skills. The FSM DDM programme structure is shown in figure 1. Following delivery in the FSM, an external evaluator (one of the authors, MP) was contracted to evaluate the programme, including the programme context, inputs, processes and products and student reactions, learning behaviour and outcomes. A combination of questionnaires, review of course delivery reports and in-person, semistructured interviews with trainees, supervisors and directors were used, reflecting a systems perspective of training that helps assess value both in terms both of benefit to trainees and impact on health agencies.16 We report the results of the evaluation here.
DDM programme timeline. DDM, data for decision making.
What we did
First, the programme was decentralised so that travelling instructors visited each of each of four FSM states (with trainees from the national department joining those from Pohnpei state) to conduct face-to-face sessions rather than bringing trainees from multiple sites to a central location. This decreased travel costs, and permitted a larger number of candidates (10–15) from each site to participate.
Second, a planning consultation visit was made by a DDM instructor to each site before delivery to provide orientation of stakeholders, identify suitable candidates and projects, and finalise logistics. A focus of this consultation was to identify opportunities to address weaknesses in the use of health information in the agency via the field projects, and to set the stage for ongoing involvement by health leaders to put the field epi project products to use. A memorandum of agreement incorporating these elements was signed between the PPHSN partner agency and each state health director and national secretary.
Third, ongoing engagement with health agency leaders was strengthened by holding in-person meetings between instructors and each agency’s leadership team at the end of each face-to-face session to provide feedback, and to solicit suggestions to assure that field projects were proceeding in a way that was useful. Leaders were also invited to attend presentation of project work by trainees at the end of each face-to-face session.
Forth, epidemiologists working with PPHSN partner agencies in the region were qualified as instructors at FNU, supplemented at times by visiting epidemiologists from other agencies working in-country (there were no epidemiologists resident in the FSM). This further reduced DDM programme costs, provided strong mentorship for trainees, and provided enhanced opportunities for synergy between DDM and other capacity-building initiatives.
Finally, mentoring of trainees for field epi projects was strengthened. In the FSM, at least two instructors were assigned for each intensive, week-long class and approximately half of each class was devoted to one-on-one work on field projects. Tasks for completion by trainees during the periods between instructor visits were defined and instructors continued to support field projects by distance during these times. Worksite supervisors were enlisted to assist with mentoring, which was much easier to arrange with instructors delivering the programme on-site. An additional week-long field epi project workshop was added to the programme to assure completion, quality and implementation of projects.
What did we find?
Seventy trainees (53 female, 17 male) enrolled in the programme across the four cohorts, constituting a mixed group of nurses 20 (29%), public health programme officers 24 (34%), data clerks 6 (9%), administrators 15 (21%) and physicians 5 (7%). Of the 70 enrollees, 48 (69%) completed all four course modules, 40 (57%) completed field projects, and 32 (46%) completed all course modules and field project. Forty who completed at least four of the five programme activities were available for interview.
In the trainees’ course evaluations, 90% or more rated each module as ‘relevant’ or ‘highly relevant’ to their work, while 80% or more rated each module as ‘difficult’ or ‘very difficult’ (physicians and senior managers were the majority of those who rated the programme as less than difficult). Some trainees, particularly data clerks, found it stressful to cover so much didactic material in addition to hands-on project work during the face-to-face modules.
Many trainees appreciated not having to travel to course modules and instructors found it easier to mentor projects when data sources and trainees’ team members were close at hand. Having instructors rather than trainees travel also averted a great deal of logistical complexity and expense. The programme’s academic accreditation was praised by trainees for opening opportunities for career advancement and access to more advanced degree programmes.
The effectiveness of the precourse consultation to ensure an enabling learning environment varied across jurisdictions. Four trainees who did not complete projects cited lack of their supervisor’s encouragement and the volume of new work assignments as reasons for not completing DDM.
At two sites (Yap, and Kosrae) facilitators held an additional 1-week classroom session to provide dedicated time for in-person guidance to trainees and for completion of the capstone field project. Trainees found this ‘field project workshop’ to be very helpful; the proportion completing the field project was higher at these sites: 25/28 (89%) for Kosrae and Yap vs 7/12 (58%) for the Chuuk, Pohnpei and FSM National Departments.
Trainees felt that classroom training was effective in supporting them to design, plan, and complete projects. In the words of one trainee: ‘The classroom is very interactive. With DDM, we listen, we ask questions, we work through scenarios, we understand. I now have the courage to translate data into words and to explain this to my superiors and to the Governor so that they can make the right policies.’ Field projects stimulated actions including passage of laws, outreach campaigns to disease ‘hot spots’, and initiation of surveillance, disease registries and performance monitoring systems (table 1).
FSM field epidemiology project examples
None of the trainees had previous experience in preparing a dataset, analysing and reporting results in a systematic manner to peers, supervisors or others. Data entry was described as a ‘boring routine’ that did not generate curiosity about the use of data locally. None were aware of standard operating procedures for managing data. An immediate outcome was that all trainees developed standard operating procedures for collating, checking, and analysing data for their projects. In the words of one participant: ‘The DDM changed me a lot, it was an eye-opener to see the data and understand what’s happening in our areas, and how to collect, analyze and report the data.’
Trainees generally expressed confidence in communicating the results of their field project in professional settings, including some who presented at national and regional meetings. Such confidence is most likely the result of the training methods of the modified programme that incorporate learning theories. Methods included reinforcement and goal setting theories where trainees are provided repeated opportunities to share their work and receive feedback from classmates and facilitators. Social learning and adult learning theories guided the methods of classroom introduction allowing students to develop the skills to critique other trainees’ presentations. Health leaders also noticed a change in participating staff. In the words of a Director: ‘In the past, when I talked about data to my staff, I used to feel like I’m talking a language from Mars. But after the DDM, my staff understood what I asked them to do and they did a great job in analyzing and presenting the data because now they understand what they are doing…… I can now give them more work to do. They have learned how to bring about change in their work.’
What did we learn?
Much attention has been paid to development of standardised activities, leaning objectives and curricula.17 18 Similar attention should be given to targeting student training projects that address real needs in the health system, and optimise an environment to enable the implementation and impact of student’s work. One of the keys to doing this is to involve health leaders at trainees’ worksites very closely in the programme through formal consultations and MOUs with senior health department leaders at the outset, enlisting trainee’s direct supervisors and checking in with leaders and supervisors at each face-to-face session. Since our experience in the FSM, to further strengthen field projects we now also require each trainee to share a project update and solicit input from their work units following each DDM module, to explain how they are applying their data work to health interventions, and to document this process in course ‘logbooks’. However, adding these measures to improve worksite engagement with projects added to the heavy burden of didactic plus hands-on work during face-to-face modules (as indicated by the large majority of trainees who rated the sessions as ‘difficult’ to ‘very difficult’). To address this problem we have since added Web-based study materials for each DDM module so that they can be given in ‘flipped classroom’ style so that epidemiology concepts are introduced on-line in advance of each face-to-face session, allowing more time to be spent on hands-on work and worksite engagement during the face-to-face sessions. Providing an extra week of protected time for the field project, a measure that we adopted at two sites during DDM in the FSM, also helped to ‘decompress’ the other face-to-face sessions and improved rates of completion and programme impact.
In-country delivery of DDM at state and national levels is more effective than centralised delivery to regional cohorts. Compared with the student outcomes and project impacts of the initial centralised pilot, the in-country delivery helped to produce a ‘critical mass’ of data-literate staff in a jurisdiction, fosters close involvement of health leaders, simplifies programme logistics and reduces costs. Furthermore, we found that with adequate candidate screening involving health leaders, a single programme can accommodate a variety of health workers, including those from various levels (junior to senior) and from public health, healthcare and administrative units. While senior staff tended to produce projects with the greatest impact, data clerks and staff nurses developed better practices in their areas of responsibility and frequently experienced a big boost in enthusiasm for use of data. An exception was the few candidates who did not have the basic math and literacy skills needed to succeed in DDM. Developing a less ambitious, short-course training package for these staff may be useful.
Finally, we confirmed that offering academic credit for DDM motivates trainees. Postgraduate certificates were awarded to successful trainees. Student performance in DDM also helped to identify candidates for the more in-depth (and masters-degree level) Pacific field epidemiology training programme that is now in progress.
Conclusions
DDM was effective in strengthening priority components of national and subnational health information systems. Programme elements that contributed to success include a ‘learning by doing’ emphasis, decentralised delivery at the workplace in multiple locations, formal predelivery consultations with health leaders, allowing lots of protected time for development of field projects, combining repetitive input for projects from both health leaders and instructors, and delivery by field epidemiologists working in the region and for academic credit.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
References
Footnotes
Handling editor Seye Abimbola
Contributors All of the listed authors contributed substantially to the design, analysis or interpretation of the work, critically reviewed the manuscript and approved the final version.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.