Partnerships

In 2014, the Bill & Melinda Gates Foundation (BMGF) provided funds over the course of the 5-year grant to support the development and implementation of the RI Module, while the government of Nigeria provided leadership on the NHMIS through FMoH, and on the RI Module through NPHCDA. These two agencies also facilitated advocacy visits at the national and state levels. The US Centers for Disease Control and Prevention (CDC) provided technical leadership as well as financial support. The African Field Epidemiology Network (AFENET) and its National Stop Transmission of Polio (NSTOP) programme were responsible for the implementation of the RI Module in all 36 states and the Federal Capital Territory, as well as technical support to states and the national level for RI data analysis and use. Other technical partners included the WHO Nigeria Country Office, the UNICEF Country Office, the Clinton Health Access Initiative and the Health Information System Programme of Nigeria.

DHIS2 RI Module and dashboard

The DHIS2 RI Module is an online platform to enter, store and analyse RI data from all HFs in Nigeria that offer these services. The RI dashboard incorporates 20 AFRIN indicators to monitor the EPI performance and process (figure 1). Its purpose is to strengthen the infrastructure for RI data collection, visualise RI data for action, develop workforce capacity in health information systems at all levels, improve data quality and use at all levels and support innovation and learning on health information systems.9–13 Data inputted into the DHIS2 RI module originate at HF level, and are collected on four paper-based collection forms. These forms were introduced in 2014, and include:

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Visualisation of four AFRIN indicators on the DHIS2 RI dashboard (September 2019). AFRIN, Accountability Framework for RI in Nigeria; DHIS2, District Health Information System, V.2; LGAs, Local Government Areas; RI, routine immunisation.

• NHMIS summary form (2013 version), which collects information on number and type of antigen doses administered, as recorded in the tally sheets and the HF monthly vaccination summary.

• NHMIS supplementary form, which collects information on the number of RI sessions conducted, supportive supervision visits conducted, RI funding availability and information on newly introduced vaccines after 2013 (eg, inactivated poliovirus, rotavirus).

• Vaccine management summary form, which collects information on the utilisation of antigens, diluents, devices and other vaccination materials.

• Reaching every ward microplan summary form, which collects information on planned RI sessions and estimations of target population size.

The LGA receives the paper-based forms each month from each HF that offer RI services. The local immunisation officer (LIO) enters the information from the previous month on the DHIS2 data entry interface by the 15th of the following month. HFs do not enter their data on the platform because most do not have access to internet or computers, or the technical capacity to perform this task. Once entered, the raw data and the updated indicators immediately appear on the module’s dashboard.

There are 21 indicators on the RI dashboard, each reported using a graph, table or map. The indicators are divided in the following seven categories:

1. Data reporting

   • Reporting rates (public and private HFs)

   • Reporting rates (all HFs)

   • Proportion of HFs with timely reporting

   • Proportion of HFs reporting on time (all HFs)

2. Coverage and drop-out rates

   • Monthly and annual coverage rate, by antigen

   • Monthly coverage rate of selected antigens (aggrega ted)

   • Annualized coverage rate for selected antigens

   • Monthly and annual drop-out rates for Pentavalent 1-3 and BCG-Measles

   • Monthly dropout rate (Penta1/Penta 3) (aggregated)

   • Status of monthly Penta drop-out rate

   • OPV 3 vaccination coverage by LGA

3. RI vaccination sessions

   • Proportion of planned fixed and outreach sessions that were implemented in the past one month

   • LGAs holding at least 90% of fixed sessions in the past one month

   • LGAs holding at least 90% of outreach sessions in the past one month

4. Supervision

   • Proportion HFs that received at least one supervisory visit

5. Vaccine logistics

   • Proportion of HFs with vaccine supply stock-out reported in the past one month

   • Commodity stock balance

   • Vaccine stock balance

6. Cold chain functionality

   • Proportion of vaccine refrigerators functional

7. RI funding disbursed to health facilities

   • Proportion of HFs receiving adequate RI funding

   • HF allocation: Expected over actual for the last 6 months

The national-level officers review these data each month to provide feedback to the lower levels and design corrective actions. The health officials at the state and LGA levels—who include LIOs, cold chain officers, monitoring and evaluation officers, partners and political authorities—are required to review and analyse RI data on a monthly basis to identify gaps in RI services as well as emerging trends in RI utilisation (see figures 1 and 2). This information is intended to be immediately used to correct and improve RI performance in HFs, LGAs and states. HF staff receive information about their performance during these visits from the levels above. Most of these operations depend on the technical and financial support of partners in the field.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Implementation phases of the DHIS2 RI module and dashboard in each state over a 2-year period. DHIS2, District Health Information System, V.2; RI, routine immunisation.

Pilot

The implementation of the RI module and dashboard on Nigeria’s DHIS2 platform was piloted in November 2014 in Kano state. The aim was to assess feasibility of implementing the RI Module at the state level, test the newly developed infrastructure and tools for data collection and analysis, and document challenges that could affect the implementation to other states. The first activity was to conduct advocacy visits to state-level authorities to promote how the introduction of the DHIS2 system could improve the completeness and accuracy of RI records. AFENET/NSTOP staff developed the training curriculum and instructed select state-level and LGA-level healthcare officers on data aggregation and entry into the DHIS2 interface, and on data analysis, interpretation and use for decision making. Between November 2014 and January 2015, the LGAs were scheduled in phases to receive the 2-week training. At the end of the pilot, NSTOP staff documented two lessons learnt: (1) deploy an AFENET/NSTOP consultant to each state for 2 years to support DHIS2 operations and train government staff and (2) include a computer literacy session in the curriculum. These lessons were applied in all subsequent state-level trainings and roll-out programmes.

Scale-up

The RI module/dashboard was rolled out to the remaining 35 states of Nigeria and the Federal Capital Territory. States had different implementation start dates—beginning in November 2014 in Kano and finishing in Jigawa in December 2017. The launch in each state took 2 years and progressed across four phases: preimplementation, technical launch, active phase and graduation (see figure 2). The goal was to have all states graduated and running the DHIS2 RI Module/dashboard independently by November 2019.

1. Preimplementation: NPHCDA national-level staff, with CDC and AFENET, conducted advocacy visits to state-level authorities. The purpose of these visits was to ensure that the RI Module would receive political, financial and logistics support throughout the life of the project. The LIO was the officer designated for the monthly RI data entry process. Each state received a tailored RI dashboard, as well as training session for state-level and LGA-level officers.

2. Technical launch: The technical launch took place during the first 6 months of the module’s implementation at the state level. Updated RI data collection tools and one laptop were distributed to each LGA. NSTOP recruited one implementation officer (IO) per state, and deployed him/her at the start of the state’s implementation process to support RI module operations. The IOs worked within the State Primary Health Care Development Agency (SPHCDA) or the State Ministry of Health (SMoH) to implement supportive supervisory visits to improve RI data accuracy and completeness. Also, the IO trained the DHIS2 Focal Person (housed in the SPHCDA or SMoH) to take over all operations related to the RI Module and dashboard.

3. Active implementation phase: The active phase took place between months 6 and 18 of the implementation period. The IO conducted supportive supervisory visits to monitor the implementation of the RI module and dashboard, and provide on-the-job mentoring. Data validation rules, enhanced data visualisation options and an updated list of HFs were programmed on the DHIS2 platform to minimise the number of data entry errors. States used the RI dashboard to develop standardised monthly RI reports and interpret its results. A national-level monthly RI report was presented to NPHCDA and the National Emergency Routine Immunisation Coordination Centre (NERICC) for follow-up action.

4. Graduation: These activities took place during the last 6 months of the state-level implementation period. The national-level team visited the state twice to ensure a smooth transition. At the end of the 2-year period, the IO was withdrawn from the state and the DHIS2 Focal Person started operating independently.

5. Postgraduation: Activities during this phase ensured that state-level gains were maintained after SPHCDA took over all financial support and human resources for management of the RI module. A key component was NSTOP’s advocacy to government authorities (especially the states’ Executive Secretaries) to include DHIS2 funding in the state’s budget. NPHCDA’s Department of Planning, Research and Statistics (PRS) continues to provide technical support to the DHIS2 Focal Persons.

State-level support

Authorities in most states supported DHIS2 operations during and after the implementation period. For example, 22 states provide monthly internet bundles of 3.5 GB for RI data transmission, and 33 states conduct supportive supervision and on-the-job training during their monthly visits to HFs and LGAs.14 Seven states consistently provide funds to DHIS2 operations in their yearly budget. These funds are provided through a budget line for DHIS2 activities, the use of Save One Million Lives, and the financial support of partners. These funds ensure laptops replacement and maintenance, and supportive supervisory visits at the LGA and HF levels. Also, 30 states provide funding for printing NHMIS forms, while the remaining seven states rely on forms provided by the national level.

Patient and public involvement

Patients were not involved in any component of this project. The information collected through the DHIS2 RI module were the administrative data routinely recorded by the government of Nigeria as part of the national EPI programme. No additional information was sought or recorded.

National level

Monthly completeness reporting rates improved dramatically over 4 years, from November 2014 to July 2019 (see figure 3). More than 80% of LGAs reported on all four RI data sources. The overall completeness measure is stable at 80%–90% since July 2018.

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Completeness rate of the monthly reporting for the four sources of RI data in Nigeria, from the launch of the DHIS2 RI module project in November 2014 to July 2019*. *The data do not represent the 774 LGAs at one time. Rather, information from the LGAs was added as each state moved into the active phase. The graph reports information on all LGAs starting in December 2017. DHIS2, District Health Information System, V.2; HF, health facility; LGAs, local government areas; NHMIS, National Health Management Information System; RI, routine immunisation.

State and LGA levels

All states report RI data through the Module, identified one DHIS2 Focal Person, and review the RI dashboard each month. Online supplementary appendix 1 reports the completeness rate of reporting for each state, both during the IO’s deployment and after his/her withdrawal. No state reported a steep drop in completeness rate after the IO left, and some states continue to improve.

Implementation challenges and lessons learnt

NPHCDA and AFENET/NSTOP encountered numerous challenges to the implementation process of the RI module. Below is a list of issues and the corresponding lessons learnt:

1. Seven states have yet to budget funds for the maintenance of the RI module, and no state fully funds all aspects of the RI module. Activities like laptop maintenance, NHMIS form printing and LGA-level RI review meetings cannot be carried out consistently because of this deficit.

2. AFENET/NSTOP could not maintain the rigorous implementation schedule without the support of government authorities. However, once the IO withdrew from a state, some HFs and LGAs stopped reporting their monthly RI data regularly. Data managers from both NPHCDA and NERICC communicate regularly with the states to encourage timely reporting.

3. High staff turnover, leading to poor DHIS2 management skills at the LGA level. AFENET/NSTOP organises supervisory visits for data quality since 2017. These visits are opportunities for refresher trainings. NPHCDA and partners are considering expanding their field visits to all LGAs.

4. Subpar accuracy of RI data reporting on both paper-based and electronic tools. Errors are due to transcription or summation mistakes at the HF level.14 Another serious concern is the lack of paper tools for data collection. NPHCDA and AFENET/NSTOP programmed multiple validity checks and compulsory data elements in the RI module, developed indicators to identify implausible values and identify outliers. WHO introduced additional data quality indicators on the DHIS2 platform in January 2019. These tools reduced the number of errors for most states (see figure 4). The customisation of the RI dashboard, along with the introduction of a user-friendly interface, enhanced the module’s relevance to states, and promoted the use of RI data to improve the quality of RI services.

5. Lack of reliable internet connectivity. Many LGA officers use internet cafes or work in the evening. NPHCDA introduced a daily SMS reporting system in the 18 states to ensure that RI data could be transmitted to the national level. Soon, Gavi will sponsor the introduction of electronic data collection at the HF level across Nigeria. Data transmission will rely on internet connectivity, SMS texting or a combination, depending on the location of the HF.

6. Management of HF lists that offer RI services, which are seldom updated but are essential to calculate accurate vaccination coverage rates. FMoH is working to establish standard operating procedures for updating and managing the HF list that is uploaded onto the DHIS2. The list of HFs has already been updated for 18 states, and the Global Fund has provided financing to complete the work in the remaining 19 states.

• Download figure
• Open in new tab
• Download powerpoint

Figure 4

Example of RI data quality improvement on the RI module. Three has been a steady reduction in the number of data entry errors* since January 2019. *There should be no data entry for antigens hepatitis B, doses 1 and 2 because they are not included in the Nigerian immunisation schedule. RI, routine immunisation.

Finally, for countries who wish to replicate this implementation model, we recommend that the introduction plan for a module-based public health data collection project include the same elements as our introduction process:

• Advocacy and support to the development of government policy and HIS infrastructure.

• Gap analysis and planning to guide implementation.

• Key stakeholders’ involvement at every step of the implementation process.

• Continuous improvement of the plan through data review.

• Utilisation of multiple funding sources.

Limitations

The main limitation was the lack of reliable Government funding for DHIS2. AFENET/NSTOP provided technical and financial support in the field, but its participation in this project ended in February 2020. WHO already started to reduce its footprint in the LGAs as part of its Polio Endgame strategy. To mitigate this withdrawal, AFENET/NSTOP used the last year of the project to transfer technical knowledge and skills to the Department of Planning, Research and Statistics (PRS) of NPHCDA. As of April 2020, tts data managers can manage the RI module independently. Gavi and the Global Fund continue to provide financial support for the next milestone of data quality improvement, and other partners continue to provide technical support.

Next steps

Training programmes are shifting daily RI module operations from AFENET to PRS-NPHCDA. NERICC continues to receive the monthly RI data and reports for its own operations and planning. PRS-NPHCDA, NERICC, AFENET/NSTOP, CDC and other stakeholders will address RI data quality issues on the platform.