Metadata: the transparency matrix

Reporting of each global health transaction should be associated with metadata that make explicit the position of that particular transaction. I build on the ideas of Seye Abimbola17 18 and others19 in highlighting a seven-dimensional vector space (table 1) likely to be most consequential for reflexivity.

As a carefully crafted ‘data structure’, this instrument contains both quantitative and qualitative inputs (and is therefore open to interpretation and evaluation from varied standpoints). These transparency matrices should be freely available and linked to each publication as metadata. The ability to extract each publication’s identity vector as well as narrative reflection can help appropriately situate each writing within the broader landscape of relevant knowledge. As the world increasingly becomes data-driven and machine-learning evaluated, it is essential to anticipate issues of bias and fairness, and proactively devise ways to counter existing and entrenched biases in data collection methods.20 21 Logging transparent data, in deliberately designed data structures, is likely to help address harmful and ethically dubious recommendations and conclusions based on biased data.21 22

Metrics are a contested tool in global health.23 Often wrongly assumed to be value-neutral, numerical metrics are in reality a product of underlying norms, values and assumptions.24 25 Distortion during data collection24 and the ‘arithmetic gymnastics’25 sometimes used to fit data to a metric, often obscure ‘on-ground’ reality.26 27 In not being end-user responsive, the data used to derive metrics are prone to being ‘gamed’.28 Further distortions are introduced during visualisation and storytelling.24 The soft metrics described here are designed with a visualisation-first and narrative-first point of view. While there is a quantitative index (identity vector) linked to the narrative (table 1), the tool is open to interpretation, stressing a data-first rather than a tool-first approach. The visual tools have redundancy built in. For instance, ontology is exposed from both a discrete/graph (figure 1) and continuous/cascade (figure 2) perspective. Reflection about intersectionality is also built-in. Most importantly, I present these tools/metrics as an examplar for reflexivity rather than a prescriptive template.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Directed Graph of Ontology: Shown here is an example of how a directed ontological graph could look like. Ontologies (the nature of entities in a transaction and relationships between them) can be visually examined using a directed graph. Shapes indicate the hierarchical status and colour-coding indicates the level of inclusion/exclusion. Ontologies can make explicit the validity of assumptions about actors and relationships, identify bottlenecks and indicate excluded intersectional identities. This graphic can be modified to elucidate the relationships between epistemological entities (concepts) and how different levels of concepts validate/nullify ensuing entities and which intersectional identities are rendered invalid in a given scheme of reasoning.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

The Cascading Cones Chart: Cone apices represent individual stakeholders. The base of each cone represents the sphere of direct influence of the given stakeholder in the current transaction. The height (or depth) of each cone represents the levels across which that influence extends. The cylinder represents the end-users of the current transaction. The interface of the cones at the bottom of the cascade with the cylinder represents the intersectional identities touched by the given transaction. The projection of this interface onto the ellipse at the bottom shows population sections (or union of intersectional identities) covered by the given transaction (white ellipses) as well as the blind spots or intersectional identities missed (black region).

Ontology, intersectionality and cascading cones of influence

Ontology refers to social entities and their relationships and how relationships and entities ‘create’ each other. No research, implementation or policy transaction should be considered valid unless it can be situated appropriately in the landscape of power relations. Power ontologies directly affect the pose, gaze, voice, position and lens that frame a given transaction in global health.

Intersectionality seeks to describe social identities as products of differential situation inside a power landscape.29 For instance, the intersectional identity of a person who is a ‘woman AND domestic worker AND living on three dollars a day AND living in a village near a large city in southern Pakistan’ is very different from that of a ‘woman AND farm worker AND on subsistence living AND situated in a ‘remote’ rural area in northwestern Pakistan’. These two individuals may have very different life experiences and health and social needs, although conventionally, in a global health context, they would often be lumped into a single category of ‘poor working women in rural Pakistan’. Intersectionality can offer a valid and legitimate basis for defining the true scope of a ‘global’ health research project. Instead of making universal generalisations, the domain of applicability can be better defined by transparently reflecting about the universe of all the potential intersectional identities within the scope of work and the ‘slice’ of this universe that is being evaluated/studied (table 1).

Ontologies can be examined and represented by a directed graph or flowchart (figure 1) where nodes represent stakeholder entities and end-user intersectional identities. The arrows represent the direction of influence in the context of each transaction. Visualisation of the ontology can make the underlying context and power relations explicit and guide in identifying valid conclusions and recommendations. Ontology can also be visualised in a cascading cones chart (CCC) (figure 2).

CCC can also help to more intuitively understand the intersectionality of end-users at the interface with power structures as well as the intersectional identities overlooked or missed by a given transaction. Thinking through the CCC can further shed light on the level(s) at which the transaction interface should be situated as some intersectional identities within the population of interest may be obscured by limitations or biases particular to the intermediaries in the cascade of influence (figure 2).

Publishing methodologies from alternate epistemic standpoints

If progress is to be made towards epistemic justice in global health, we need to make room for alternate ways of knowing and evaluating at the highest echelons of credibility.18 Mainstream journals publishing academic research with their editorial and peer review structures often function as gatekeepers for what qualifies as ‘admissible’, ‘high-quality’ research and knowledge. However, in judging validity and value through a one-dimensional Eurocentric epistemic lens,30 such gatekeepers commit and perpetuate epistemic injustice against knowledge-producers expounding from alternate ways of knowing and indigenous epistemologies.18 31 This leads to testimonial injustice32 namely exclusion of testimony that originates from alternate interpretations of how the world works.

It is essential that leading journals publishing impactful global health research make room for original research articles that explore the world from alternate epistemes. Gradual formalisation and mainstreaming of alternate ways of knowing and analysis can start with publishing ‘methodology’ or epistemology papers conceptualised and originating from their relevant locales. Bringing these methodologies to the fore can help demystify their workings for an interested practitioner in other contexts, resulting in broader dissemination, awareness and cross-fertilisation of ideas and knowledges.

In addition to requiring artifacts as metadata for context as well as logs of reflexivity, journals publishing global health research should also lower barriers to publish knowledges emanating from alternate ways of knowing and being. Dedicated space should be created in publications for these parallel logics and epistemologies and a greater number of peer-reviewers belonging to traditionally disadvantaged backgrounds should be approached, engaged and recognised. Such humility and open-mindedness are only possible when we start shifting our thinking from “hierarchies” of knowledge and evidence to “landscapes” of contextually relevant knowledge and from cliques of influence and institutional prestige and privilege to “flattened” social networks and egalitarian ontologies. The visual and narrative tools described in this work can help illuminate some of these cognitive transitions.

Randomised controlled trials (RCTs) are often considered the highest and most pure level of scientific evidence in health and social sciences.33 34 Their strength derives from randomisation which controls confounding. However this ‘overcontrolling’ often makes these studies idealised (mis)representations of reality-in-context, by artificially dissociating them from the contextual causal substrate (‘usual care’ settings) in which the studied interventions are to be implemented.33

Many variants of RCTs such as pragmatic or adaptive designs have been recommended but they each suffer from several limitations, statistical loopholes and similar objections about which version of reality they represent.33 34 The costs associated with conducting any sizeable RCT have led to over-representation of the interests of sponsors and investigators hailing from North America, Western Europe and East Asia and shifted the power of knowledge creation decisively in favour of entities from these regions.33 As we wade into the age of pervasive and ever-proliferating real-world data and aspire for a future of personalised/individualised health for all, we may well be forced to privilege ‘real-world evidence’ over and beyond RCTs.33 35

In order to make sense of the high volume and velocity of real-world data one often needs to rely on complex algorithmic pipelines, or machine learning techniques. Yet with millions or billions of learnable parameters many of these technologies often leave little room for explaining how the algorithmic ‘oracle’ draws its inferences.36 37 It is interesting to reflect on how global human health decision-makers have greater faith in inductive bias of artificially intelligent ‘black-boxes’ of limited explainability38 39 than ‘explainably intelligent’ fellow humans with inductive biases and explainable epistemes fine-tuned over millennia of evolution-in-context.

Moving from statistical to causal inference and using a calculus of causal diagrams for reasoning40 may be a step in the direction of greater transparency, accessibility and inclusivity. Requiring each author to fill a Transparency Matrix (table 1) and publishing this as metadata with each study will likely encourage reflexivity in study design and implementation, including for RCTs when such trials are deemed necessary.

CCC (figure 2) and visual representations of ontologies (figure 1) can also be adapted to reflect and reason about sequential assumptions and correctly identify the intersectional situations where inferences may be justifiably applicable. Using simple visual tools enables a greater diversity of people to participate, weigh in and be heard. This is especially so as many indigenous ways of knowing and reasoning rely on visual artefacts and narratives to make sense of the world.41 42