Hybrid Survey methodology

As a first step, NCD Monitoring and Surveillance plans were developed and endorsed in each USAPI to outline local NCD surveillance. In order to select NCD indicators for monitoring, existing NCD surveillance frameworks were reviewed.13 28

A 1 day NCD Hybrid Survey planning workshop was conducted in-country that involved local stakeholders interested in collecting and using NCD data. The goal of this workshop was to develop a Hybrid Survey project plan, project budget and survey instrument.

To develop a survey instrument, standard instruments with validated questions were reviewed such as BRFSS, NHANES and STEPS. Questions and measurements were selected that would fulfil local data needs and external reporting requirements. Core questions and measurements based on the PIHOA NCD surveillance framework were also maintained throughout the region to allow for regional comparisons. As needed, questions were adapted to be made more culturally appropriate, and some indicators were modified from global reporting standard to make them more practical for the region. These instruments were eventually piloted by staff, revised as needed and translated into native languages by a team of health and non-health community members and underwent rigorous review.

After the planning workshop, the project plan was presented to local partners and external support partners such as PIHOA, CDC, the Pacific Community (SPC) and WHO to determine how all budget items and technical support could be covered, thus further refining the project plan. The final project plan covered items such as sample size, sampling procedures, data collection methods, consent, surveyor recruitment, surveyor training, surveyor supervision, data processing, data cleaning, data analysis and dissemination.

To determine samples in each jurisdiction, households were first randomly selected from best available household listings. In some USAPI, remote outer islands had to be excluded due to small population size and lack of accessibility. After the household sample was selected, one adult (18 and older) from each household was randomly selected using Kish methodology.29

Approximately 6 months after the planning workshop, a required 4–5 day training workshop was held in-country for all surveyors. This workshop covered items such as sampling procedures, interview techniques, question-by-question review, physical and biochemical measurement protocols, electronic data collection and management skills on a tablet and participant health feedback. All items covered during the training were included in a comprehensive training guide used during the training workshop that was given to all surveyors. These training guides were adapted for each USAPI, but the core components remained the same for consistency across the region. External support partners and health staff from other jurisdictions who had already conducted a Hybrid Survey led the training workshop. This peer-to-peer training was an effort to further build capacity in the region.

An additional part of the survey preparation involved promotion of the survey and community education. To successfully get messages into the community, it was critical to involve local partners such as local, traditional and religious leaders in this process to ensure community buy-in.

Data collection began immediately following the training workshop and typically took about 6 months. Approximately 30 local surveyors were used in each USAPI, and generally two local survey coordinators provided leadership and oversight. All USAPI but CNMI (who used paper forms) used tablets to electronically collect and upload survey data. All surveys were conducted using a face-to-face survey questionnaire along with physical and biochemical measurements. Physical measurements conducted included height, weight and three blood pressure measurements. Biochemical measurements conducted included fasting blood glucose (in all USAPI except CNMI that used a random blood sugar) and total cholesterol measured on a point-of-care device.

All data collected were uploaded to a secure server, then downloaded and cleaned to develop a final data set and data dictionary. About 6 months after data collection was completed, a final report was produced with technical support from PIHOA. This report was then provided to local health departments for review and endorsement by local health leadership. Then, approximately 1 month after report endorsement, a 2-day dissemination workshop was held in-country with local stakeholders to present the data and train these individuals on how the data can be used to set health priorities and select certain strategies.

Strengths, weaknesses and lessons learnt were documented after each survey based on qualitative evaluations with the survey team and shared within the USAPI to modify methodology in other jurisdictions. Additionally, these evaluations will be considered when planning for the second round of data collection in all USAPI.

It should be noted that methodology differed slightly in Guam due to the fact that Guam already had a functional BRFSS. Therefore, in Guam, state-added questions were included at the end of the phone survey that asked participants to provide contact information in order to be called back for a follow-up survey. These individuals were then contacted, and an appointment was made to collect the participant’s height, weight, blood pressure, fasting blood glucose and cholesterol. These data were merged with the BRFSS phone survey data in order to estimate prevalence of chronic conditions and disease.

Impacts of the Hybrid Survey model on surveillance

In 2015 (baseline for the development of the Hybrid Survey model), three USAPI jurisdictions (Guam, Palau and Yap) out of nine (counting FSM states individually) (33.3%) had completed an adult population-based survey in the past 5 years. This includes Guam, whose survey relied only on self-report and lacked physical and biochemical measurements. Four (44.4%) of the USAPI jurisdictions (Chuuk, CNMI, Kosrae and RMI) had no adult NCD prevalence data at all in 2015. Additionally, only two (22.2%) USAPI jurisdictions (Guam and Yap) had data sets that they were able to access for their own analysis and only four (44.4%) USAPI jurisdictions (Guam, Palau, Pohnpei and Yap) had at least two comparable surveys to be able to assess trends over time (table 1).

In 2020, all nine USAPI jurisdictions had, or were in the process of collecting current adult NCD prevalence data. On completion of the Hybrid Survey in Yap, all USAPI jurisdictions will have objective NCD prevalence data based on physical and biochemical measurement rather than self-report of height/weight and hypertension/diabetes status that can under-report chronic disease. Additionally, eight out of the nine USAPI jurisdictions (all but Chuuk) (88.9%) now have data sets that they can access for local analysis, and five (American Samoa, Guam, Palau, Pohnpei and Yap) (55.6%) have at least two comparable surveys to evaluate trends (table 1). Additionally, turnaround times for results were greatly improved, though not quantified in this report.

Lessons Learnt

After 5 years of adopting the Hybrid Survey model, more USAPI have current adult NCD data with physical and biochemical measurements, are able to access their survey data sets for local analysis, have the ability to analyse trends and have faster turnaround times for data reporting after survey completion. This model has proved to be an effective and potentially sustainable way to collect adult NCD risk factor and disease prevalence data by allowing the USAPI jurisdictions to take control of their own NCD surveillance by first developing an NCD monitoring and surveillance plan, then designing their own adult population-based survey that works best given available resources. By using existing survey instruments and validated questions from multiple support partners, the USAPI can collect data that meets reporting requirements from various external partners, as well as meet local data needs using one survey.

In addition to the Hybrid Survey model being an effective method for improving NCD surveillance, it has also been an effective strategy for building epidemiological capacity which has been previously identified as a weakness in the region.20 Although support partners provide technical support, local staff members coordinate the survey and collect their own data. The staff members involved in these surveys have developed valuable skillsets to include sampling, interview techniques, data collection and management using tablets, physical and biochemical measurement techniques and increased overall health knowledge that have been used in other surveys within their own jurisdiction. Additionally, survey coordinators who have completed their own jurisdiction’s survey are then used as trainers in other locations during the survey training workshop. This model has developed a cadre of experienced survey coordinators in the region who are able to share resources and provide peer-to-peer training and support. This type of peer-to-peer education has proven to be effective within Pacific Islander communities when it comes to other issues such as diabetes management.30

Another important feature of this model is the timeliness of data availability and reporting, and data ownership and access to data sets for local analysis. Past adult population-based surveys were collected using paper forms, and the data entry process was often delayed. Additionally, when outside partners collected data previously, there was often long lag times in data reporting where a report was generally developed five or more years after data were collected, making the data outdated by the time that they could be used. The Hybrid Survey collected data using tablets, so the data could be uploaded and viewed in real-time, therefore eliminating the lengthy process of data entry. After data collection was complete, a survey report was developed in 3–6 months. Additionally, the cleaned data set and dictionary were given to the jurisdictions for use locally. Through timely data processing and report development, as well as local data ownership, local health departments were able to have tangible reward after the long, resource-intensive process of data collection.

After the Hybrid reports were prepared, 2 day data dissemination workshops were held to share the findings and train stakeholders and community members on how to use these data to identify priorities and inform public health action. These dissemination workshops were a critical part of this model, as oftentimes data reports from the region were previously never brought to the community, and if they were, most community members are intimidated by using data, or had never used data to set priorities and drive action. After these dissemination workshops, the Hybrid Survey data were used to review and revise national NCD strategic plans, monitor and evaluate progress on NCD action items and advocate for certain programmes or policies within the community. These dissemination workshops could potentially be useful after other large surveys in the region.

The biggest strength of the Hybrid Survey model is the collaboration, both within the jurisdiction and between the support partners. In small, resource-limited settings, population-based surveys can be challenging and consume many human and financial resources.31 Therefore, it makes sense to integrate various NCD-related and other health indicators from different departments locally and from multiple external partners into one comprehensive survey. This allows these small island jurisdictions to maximise available resources and drive their own NCD surveillance efforts. Changing the model from external partners conducting or recommending surveys locally, to local health departments modifying their own survey and reaching out to external partners for support has proven to be an effective model in the USAPI and has the potential to be used in other small, resource-limited areas such as other Pacific Island jurisdictions, as well as rural areas that often lack local data.

There are of course limitations to this Hybrid Survey model. Success may depend on a jurisdiction’s need and ability to reshape the NCD surveillance system. It is critical that an NCD surveillance plan be developed and agreed on for an NCD Hybrid Survey to be successful. Additionally, health department leadership support for this model is critical to ensure appropriate resources can be allocated.

There were also unforeseen challenges when implementing these surveys such as outbreaks and natural disasters that are common in the region. Therefore, the Hybrid Survey planning and implementation needed to be flexible. Also, there were issues of surveyor integrity that arose in some jurisdictions including falsification of data and taking convenient participants instead of the randomly selected individual. These were issues that needed to be carefully monitored by survey coordinators, and rigorous checks were put in place in some locations. These issues were closely monitored and limited as much as possible.

NCD surveillance data have been found to be limited in most low-income and middle-income countries largely due to lack of capacity to collect these data, despite increasing NCD burdens in these nations.32 In higher income countries, there are challenges around collecting and reporting data from rural and remote areas. Oftentimes NCD data from rural and remote areas are not accessible due to small samples, or estimates need to be used for these areas.33 34 Additionally, the Oceania region commonly gets consolidated with the southeast Asia and east Asia regions in global studies, making it difficult to highlight the specific health concerns of this region.35 Therefore, it is essential to have accurate, locally collected and reported health data in these small island nations. The next step for this work will be to publish the results from these USAPI Hybrid Surveys more widely.