Article Text
PDF
Abstract
As health systems practitioners and researchers increasingly turn towards systems thinking approaches and work on building interorganisational networks, they have demonstrated increasing interest in network analysis for investigating relationships and interactions between system actors, both at the individual and organisational levels. Despite the potential of network-based approaches to improve health system efficiency, effectiveness and responsiveness, both the theoretical and practical guidance on designing and evaluating network-building strategies is underdeveloped within the field. While there are multiple tools and resources to help users collect, manage and analyse network data, there is much less guidance on the practical applications of this information. One apparent gap is the limited application of longitudinal organisational network analysis, in which data are collected from the same organisational actors repeatedly over multiple time points. This yields insights into the dynamic nature of networks, including how the network structure and interactions change over time. Given that networks are rarely static, the addition of the time dimension has the potential to substantially enhance the analytical value of network analysis and contribute to more nuanced guidance for interested practitioners and policymakers. In this article, the authors draw on their experiences in conducting longitudinal network analysis of interorganisational relationships in the USA and India to comment on the opportunities and challenges of the methodology within the field of health systems research. We also provide suggestions as to how some of these challenges may be addressed or mitigated.
- public health
- health systems
- health systems evaluation
- other study design
Data availability statement
As a reflection on other studies, there are no data in this piece of work.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Summary box
System science and complex adaptive systems theories, frameworks and methods are gaining interest because of their aim to accommodate the dynamic and evolving relationships between health systems entities and variables.
Network analysis methods used are often static, representing a snapshot of relationships at a single point in time, which limits their potential for drawing inferences about network-based interventions.
Longitudinal network analysis (LNA) can enhance our understanding of how to effectively implement interorganisational collaborations, which in turn are essential for achieving the Sustainable Development Goals and Universal Health Coverage.
Drawing from field applications in India and the USA, this paper highlights key methodological challenges and opportunities for application of LNA; these relate to data collection, management and analysis, including interpreting and visualising results.
Introduction
Due to the inherent complexity of intervening in health systems, researchers and practitioners are seeking to apply new methods to investigate questions that cannot be answered using the traditional armamentarium of public health.1 System science and complex adaptive systems theories, frameworks and methods are gaining interest because of their aim to accommodate the dynamic and evolving relationships between health systems entities and variables.2 3
Concurrent with this trend, there is a growing interest among health and social service organisations in network-based strategies and interventions,4–8 including for transmission networks, social networks and organisational networks.9 Network analysis is a natural complement to such efforts, with substantial potential for application in formative research, ongoing monitoring of network development and evaluation of the effectiveness of network-based approaches. While network analysis has been widely used in public health to study transmission networks and social networks, the analysis of organisational networks has emerged only more recently, despite its widespread use in the fields of business and political science.10,9
Network analysis is a systems science method whose aim is to measure the nature of relationships between social actors, whether individuals or organisations, which in turn influence actors’ behavioural opportunities, constraints and decisions.11 Early application of organisational network analysis demonstrated the potential benefits of cooperation between entities in a network structure, including the pooling of resources (eg, financial or physical capital, technical skills, information, etc.), increased efficiency or effectiveness in achieving a shared objective or bringing together diverse stakeholders that may not have otherwise come together in a planned or coordinated manner.12 13 Organisational network analysis has also been used to assess network structure and/or coordination of services,14 15 to identify opportunities for integration of services,16 17 to refine internetwork and intranetwork resource exchange18 19 and planning in multisectoral environments,20 to monitor and evaluate capacity building21 22 and to study policy advocacy.23 Despite the encouraging findings from this research, the application of network analysis to organisational networks seems to be lagging behind that for networks of individuals. Though multiple practitioner-oriented publications highlight the potential benefits of using network analysis to build organisational networks,24–26 this guidance often papers over the complexity of network building: the primary aim is often to increase the number of network members and the number of relationships among them in order to achieve shared programme goals, often without acknowledging or questioning the implicit assumption that more is better. There are multiple ways in which in this ‘more is better’ assumption can break down, however, potentially resulting in: increased interorganisational competition;27 28 time and resource investment with little benefit to members;27 worsening benefit–cost ratio or reduced efficiency after reaching a certain network size;13 28 concerns about information confidentiality;28 network opposition and professional protectionism;28 duplication of efforts;13 ambiguity or uncertainty around accountability mechanisms;29 and coercion or manipulation of weaker network members by more powerful ones.28 29
The literature measuring organisational networks over time is also underdeveloped. In this paper, we refer to this approach as longitudinal network analysis (LNA) of interorganisational relationships. From an evaluation perspective, LNA may be used to assess the relationships between changes in network structure and desired outcomes, moving closer to causal inference than would be possible using a cross-sectional snapshot of a network at a single time point. This, in theory, is useful for identifying and refining network strengthening strategies that are best suited for various objectives. Operationally, LNA can be used to monitor the extent to which implementation of a networking strategy actually matches what was intended, helping to guide decisions related to network initiation, management and governance. Underscoring the value of this approach, Walker and Stohl30 used LNA to demonstrate the emergent, dynamic nature of intranetwork linkages related to resource sharing and communication about tasks over the lifespan of two collaborative engineering design projects.30 Applying the method within the health sector, Keeling et al31 demonstrated that LNA could be used to improve management decisions in local health departments in the USA and to detect organisational adaptation over time, including improved outbreak mitigation, reduction of task redundancy and improvements in specific population health outcomes.31 Thomas et al applied LNA to demonstrate the value of an organisational network approach for improving coordination of HIV and family planning referrals to reduce unmet needs among people living with HIV in Ethiopia.16
Drawing from the authors’ own experience of applying LNA in partnership with non-profit organisations in the USA and India, this paper highlights key potential benefits and challenges of LNA for health systems research and practice. The ‘Field applications’ section summarises the context, purpose and methodological details. The ‘Findings’ section outlines the authors’ observations from these field applications, with an emphasis on practical and methodological challenges. Finally, the ‘Discussion’ section offers suggestions for how some of these challenges may be mitigated.
Field applications
The insights highlighted in this paper are derived largely from two field applications of LNA between 2015 and 2017, including in the USA and India. The first application involved collaboration with a non-profit organisation to guide a multistate, network-building strategy for increasing access among low-income communities to healthy, fresh fruits and vegetables through local farmers markets in several states of the Northeastern USA.32 Two of the authors, LP and DG, partnered with the non-profit organisation to help characterise the structure of the organisational networks in two states in which it was providing programmatic support—New Hampshire and Vermont—as well as changes in the structure over a 6-month period. The studies employed a mixed-methods exploratory sequential design. For each time period, data were collected through a self-administered digital network survey over a 2–3 month time frame, from October to December 2015 and May to June 2016.
The second application was through a partnership with an India-based non-profit foundation implementing an integrated rural health and development project in Uttar Pradesh state of northern India.33 CH partnered with foundation staff to characterise the intersectoral collaboration between the organisations working on maternal and child health (MCH) and/or water and sanitation (WASH) before and immediately after the implementation of the project (2015–2017). Data from the two time points were collected using face-to-face semistructured interviews with 31 key informants from 24 organisations working on MCH and/or WASH in Kachhauna, Uttar Pradesh, India between June and September 2017. Network measures including density, centrality and homophily were calculated to characterise the networks and a qualitative analysis was undertaken to better understand the strengths and weaknesses of the intersectoral collaboration from the perspective of key informants. Details from both field applications are summarised in table 1.
Field applications of longitudinal network analysis in the USA and India
In both field applications, LNA was applied as a formative assessment to inform programme implementation efforts by enhancing the partner organisation’s understanding of the levels and types of interaction between network actors. For the US case, the focus of LNA was to characterise specific types of interactions between farmers markets in one network and between regional supporting organisations in the other. The authors also aimed to quantify ‘network capacity’ to achieve programme objectives by gathering organisations’ perceptions and experiences with the network. The data from two time points revealed seasonal variation of network measures, thus providing insight into the dynamic—and potentially cyclical—nature of key interactions, while also underscoring the importance of taking seasons into account when monitoring the growth or evolution of the network.
For the India case, the primary impetus for LNA was to identify the key organisations focused on improving MCH and WASH services in Kachhauna and to understand the nature and extent of the coordination between them before and after the implementation of the project. The non-profit then used this information to identify opportunities to enhance multisectoral collaboration. Since the initial round of data collection took place in the early stages of project outreach to establish organisational partnerships, it was also intended as a baseline for measuring progress towards fostering collaboration between actors and sectors (eg, health, water, sanitation) over time.
Challenges
Both applications highlighted methodological and logistical challenges relevant to the use of LNA in other contexts. These were identified and discussed through a series of internal reflection meetings among study team members after the completion of both field applications. Below, we organise these challenges into seven broad, non-mutually exclusive, categories. We reflect on the relevance of these challenges, and also on the opportunities they could provide to stimulate innovation in health systems research.
Identifying which network measure(s) to use
While there are commonly accepted interpretations of certain network measures, both for individual actors (eg, ‘degree centrality’, or the overall connectedness to other actors; ‘betweenness centrality’, which represents gatekeeping power/influence) and for networks overall (eg, ‘density’, or overall connectedness of the network based on the proportion of all possible connections that are actually present), there is less guidance about how to determine which types of ties to focus on (eg, routine communication, resource sharing, formalised partnership, etc.). The relevance and importance of each type of tie may vary based on the research question, context, actor characteristics or stage of network development/maturity. These limitations are particularly true when the unit of analysis is an organisation, as many of these network measures apply most directly to networks of individuals. Additionally, from the perspective of those seeking to use network-based approaches to achieve, enhance or sustain programmatic outcomes, there is no single way to measure the somewhat abstract and context-dependent concept of ‘network capacity’, which is often conceptualised as an intermediate step on the causal pathway to improved health or social outcomes. All of these factors represent potential barriers to the applied use of LNA for health systems research because they rely heavily on those conducting the analysis to carefully consider and construct a logical bridge between abstract network measures and tangible, program-relevant indicators. This makes it difficult for people to clearly articulate what types of network characteristics to target as well as the types of measures to help them track progress over time.
Dealing with data missingness
Gaps in data points are particularly problematic for network analysis, since one missing respondent at a particular time point in a group of N entities results in a loss of N-1 data points and, consequently, inferences based on a partial snapshot of the network (see figure 1, created with NetDraw).34 This is especially challenging when each given relationship or interaction between entities is defined by two separate data points (ie, data from both Organisation A and Organisation B about the nature, frequency or other characteristic of a particular interaction between A and B). When analysing these types of ‘confirmed ties’ (ie, an interaction confirmed by both parties), if one data point between a pair of nodes is missing, the other data point is effectively missing as well, since both are required to characterise the interaction. To compound the issue, it may not be clear what assumptions to use for data imputation since individual nodes are not necessarily assumed to behave like other nodes with similar characteristics. Different patterns of missingness across time points further complicate efforts to identify meaningful patterns or trends over time.35
One-way (unconfirmed) reported linkages (‘any links) at time 2 for network B in the US-based case application, colour-coded by actor response versus non-response. Of the 52 actors in the network at time 2 for network B in the US-based case example, only 21 (40%) responded; of these, only 11 actors also responded at time 1. Though respondents report their interactions with non-respondents, no information is available from non-respondents to confirm those interactions or to report on interactions with other actors. This substantially limits the ability to characterise the network with confirmed ties at each individual time point and the issue is often compounded as the number of time points increases with longitudinal analysis.
Threats to validity and reliability of measurements
When administered in survey or questionnaire format, it is possible that data will be collected from a small number of individuals in each organisation (potentially just a single person) and it is assumed that the responses from these people represent the perspective or experience of their entire unit or organisation. This introduces the possibility of several potential issues, including selecting the wrong person as the respondent or the wrong person being assigned to respond by organisational management; respondents misrepresenting the interactions of the organisation, whether intentionally or unintentionally (eg, due to recall bias); different individuals from a given organisation providing responses at different time points. Any of these issues could result in misleading findings.
Unclear interpretation of results
Networks can undergo changes in relationships, centrality, density and other measures, but it may be difficult to derive a clear, meaningful interpretation of these changes, particularly in terms of interpreting the numerical magnitude of a given change. Whereas in static network snapshots, organisations can be easily grouped or ranked based on various network attributes—and can thus be summarised in ways that programme managers and policymakers may find relevant, such as identifying organisations serving as key resource providers or information brokers, for instance—the interpretation can become more opaque when incorporating the time dimension.
Two contributing factors in particular emerged from our field applications. The first is that the numerical values of network measures (both at the node-level and network-level) vary simply based on the set of entities for which data are available at a particular time point, making it difficult to disentangle true changes from distortions due to missing data. While it may be possible to impute data points for some types of network analyses, this can be problematic because: (a) imputation models may fail to converge if the missing data exceed 15%–20% of all data points, often resulting in a restricted analysis focusing on the observed subsample of the network; and (b) it is often inappropriate to assume that the behaviour of missing respondents will be similar to those who responded.35 The second is that there may be no clear target or picture of the ideal network structure. Without a specific desired endpoint for ‘network building’, it is difficult to provide nuanced evaluative judgements about the current network structure or changes over time.
Recommendations
While the potential value of LNA in health system research is clear, non-trivial challenges that may be limiting the application of this methodological approach in the field exist. The recommendations outlined below may help accelerate the adoption and adaptation of LNA for health systems research and practice.
Expand the application of network theory to health systems interventions
Although there is a substantial body of literature on network theory applied to a variety of other fields (including sociology, social anthropology, economics, social psychology and education),11 it is relatively underdeveloped within health systems research.8 By network theory, we are referring specifically to the use of graph-theoretic (ie, network map) representation and quantitative structural metrics to analyse agents (ie, as nodes or vertices) and relationships between them (eg, as connections or edges) within a system or network; this contrasts with organisational theory, which is a broader umbrella of various frameworks and approaches to investigating relationship dynamics within and between organisations and for which there is a much vaster literature in health systems research. Additional theoretical work is needed to help organisations design and test more nuanced strategies for using network-building as a means of influencing system ‘software’ (ie, the relationships, ideas, interests, norms and values of interdependent actors)36 to achieve organisational objectives and improve health outcomes. Specifically, there is a need for conceptual guidance about which types of network structures and attributes are best suited for various network objectives and about patterns of structural change as networks evolve.
Focus on a few priority network ties
Given the propensity for network analyses to require a large amount of data, combined with the time and resource costs of data collection as well as the aforementioned data management challenges, many researchers would benefit from limiting the scope of their network analysis on a few, priority ties. These should be identified based on a specific research question or questions, accompanying theoretical framework, and practical considerations of data quality and availability. Additional guidance is also needed to help researchers select the right respondent(s) to represent each organisation and decide how to measure relevant ties for each research question (ie, which interaction(s) to target).
Define and collect relevant outcomes data
Related to the above suggestion, it is important to incorporate relevant measures of proximal or intermediate outcomes (whether positive or negative) that are anticipated to result from changes in network structure or interactions (ie, as per the theoretical framework and existing empirical evidence in the literature). Doing so can help to provide information about how implementation changes the relationships in a particular context, can help monitor changes in context as exemplified by stakeholder relationships and can contribute to an understanding of how negative or positive outcomes arise out of stakeholder relationships. During implementation, it could help spot emergent behaviours, as well as entry points that can be considered as part of adaptive intervention designs.
Incorporate network data collection into organisational reporting
Since ad hoc data collection specifically for the purpose of a network analysis is likely to increase the reporting burden on respondent organisations (thus increasing the likelihood of missing or incomplete responses), this process should be incorporated into existing, routine reporting systems to the extent possible. This is likely to be most feasible when the additional required data are focused and few (as suggested above) and when there is already an explicit governance structure, a clear communication mechanism between the network members and a core actor responsible for both network building/coordination and facilitating ongoing data collection about the network. Depending on the implementation scale and timeframe, the frequency of checking organisational relationships can be adjusted to meet a particular activity’s unique needs for information. Recent advances in user-friendly software, such as the PARTNER platform (The website also contains examples of case applications of the software; as of the writing of this article, a feature for ‘longitudinal tracking’ of networks is ‘coming soon’. PARTNER Platform. Visible Network Labs. Accessed 28 May 2021. Available from: https://visiblenetworklabs.com/partner-platform/), are simplifying the process of data collection, management and analysis, thus lowering the barrier to entry for organisations interested in network analysis.
Incentivise complete, timely reporting of network data
As a way to reduce missingness, ensure that there is some real-time or immediate incentive for network respondents to report data for the network analysis. Even before the network analysis is complete, respondent organisations should be provided with timely, relevant, meaningful information related to the data they provided. While there are myriad ways to accomplish this, the approach the authors used was to create pithy, program-relevant, non-academic briefs as an interim alternative to the academic products of the analysis.
Incorporate qualitative data in a mixed-methods design
The challenge of interpreting findings appropriately underscores the need for collecting qualitative data over time to complement quantitative network measures and sociograms. At the outset of network analysis, qualitative methods may be instrumental in better understanding the context in which the network exists, as well as in determining which quantitative network measures are the most relevant and why. During the analysis, qualitative data are beneficial for understanding changes in the context and answering key ‘how’ and ‘why’ questions related to changes in network structure and dynamics over time. This is due to a number of reasons, including heterogeneity of organisational actors, data limitations (eg, missingness and reliability issues) and the high contextual dependence of many health systems issues and questions. As articulated clearly by Creswell et al37 38 in regard to mixed methods, researchers should determine in advance specifically how each of the methods, separately and together, contributes to answering the research question(s).
Understanding how organisational relationships change over time is important for guiding network-based approaches to implementation research, health systems policy and practice, from coordination and integration of care to planning and management of resources. We hope that in the future, more researchers and practitioners will apply LNA to gain an insight into emergent behaviour of organisations and how to best leverage and intervene within networks and document their experiences so as to continue advancing its theory and practice.
Conclusion
Network analysis can help illuminate critical interactions between organisational actors intervening in health systems policy and practice, which are often inadequately captured in other types of analyses. Tracking such interactions longitudinally is key to measuring and enhancing the effectiveness of network-based approaches but can be difficult to implement in practice. Future work in this direction will benefit from theoretical work linking specific types of network structures and attributes to organization- and network-level objectives; consideration of such linkages by those designing and those evaluating network-based approaches; and addressing several non-trivial data challenges, including through the recommendations offered in this paper.
Data availability statement
As a reflection on other studies, there are no data in this piece of work.
Ethics statements
Acknowledgments
The study team would like to gratefully acknowledge Wholesome Wave and the farmers markets and regional hubs in Vermont and New Hampshire that responded to the two rounds of data collection in October to December 2015 and May to July 2016 as part of the main research study.
References
Footnotes
Handling editor Valery Ridde
Contributors DG contributed to data analysis and writing for both field applications and led the conceptualisation and writing of this manuscript (original draft, review and editing). LP led the design, data collection, project administration, analysis and writing for the US-based field application and contributed to the conceptualisation and writing (review and editing) of this manuscript. CH led the design, data collection, project administration, analysis and writing for the India-based field application and contributed to the writing (review and editing) of this manuscript.
Funding The US-based project described was supported by Grant Number U54HD070725 from the Eunice Kennedy Shriver National Institute of Child Health & Human Development (NICHD). The project is cofunded by the NICHD and the Office of Behavioral and Social Sciences Research (OBSSR). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NICHD or OBSSR. The authors also gratefully acknowledge the HCL Samuday Project in India, which was supported by Grant Number 124005 from the HCL Foundation, as well as the key informants who contributed to that study. We also gratefully acknowledge our Indian coinvestigators who coauthored the substantive research paper specific to that case, which was published in PLOS One in 2019 (https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0219786). Though the present paper brings together reflections from the author teams after the research projects were concluded and after our coinvestigators moved on to other jobs, projects and institutions, their work on the original study helped inform discussions for this paper.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.