The role of consortia in reforming and strengthening organisations’ research systems and cultures: real-life learning

Research consortia are able to influence improvements in research systems and cultures in their member organisations, but in practice they and their funders are not maximising this influence. They are hampered by a lack of evidence and are generally not incentivised to invest in research systems unless strongly encouraged by their organisations’ leaders or mandated by the funders. Even if funders do require them to strengthen organisational systems, cultures and environments, the resources provided to do this are often relatively small compared with the funds available for their primary research. The importance of taking a systems approach has been widely recognised because capacity development is ineffective if targeted at only one level—it needs to extend beyond individuals to organisations and the (inter)national level. Not taking a systems approach means that capacity development activities are fragmented, local institutions are not able to train and retain their researchers,1 and there is a lack of national bodies able to coordinate research and use findings for policy and practice.7

Although some tools to help organisations identify strengths and gaps in their research systems have been published,12 13 there is insufficient evidence about how to strengthen research systems systematically and effectively, and how to measure any changes in outcomes and impact.14 In particular, evidence about how this strengthening can be achieved by research consortia is scarce.15–17 Unless the outputs and outcomes of research system strengthening can be measured, consortia leaders and funders may find it difficult to allocate resources to these activities since they both rely on project evaluation indicators to demonstrate performance.18

For over a decade, our Centre for Capacity Research (CCR) has been producing evidence about how to improve the efficiency, effectiveness and inclusiveness of organisations’ research systems—especially within international research consortia primarily involving partners in the UK and sub-Saharan Africa. Our CCR researchers are multidisciplinary and, uniquely, have been embedded within large multinational research programmes at the interface between the programme management team and the consortia. They generate evidence that helps to improve programmes’ effectiveness in strengthening research capacity within the programmes’ lifetime while also contributing to the development of a critical mass of global knowledge and expertise.

The purpose of this paper is to collate practical examples of how large multinational research consortia influenced and strengthened member organisations’ research systems. Illustrative examples of actions taken, which were drawn from a range of programmes, are provided in italicised boxes. We present learning and recommendations for consortium leaders and their members, and for their funders and programme managers, about practical actions they can take to contribute more effectively to positive changes in research environments and culture among their member organisations. Our recommendations are based on our reflections about evidence generated from many large consortia-based research programmes which focused on research topics including health, natural sciences, conservation agriculture and vector control. These programmes all had a mandate from the funders to strengthen research capacity in their LMIC partner organisations.

The programmes and their participating consortia had all been selected by the funders following open competitions. The consortia all involved organisations (mostly academic) in LMICs (predominantly sub-Saharan Africa) with variable levels of management and research capacity. Further details about the programmes from which evidence was generated are provided in box 1. This evidence was collected using multiple methods (eg, surveys, observations, interviews, document reviews) and from the perspectives of different cadres in the research workforce (eg, researchers, managers, directors, funders, technicians, evaluators).

Our authors were involved with these consortia for their whole duration either as researchers embedded in the programmes to explore aspects of research capacity strengthening or as research managers (MS, NB) who worked closely with the research teams. Ongoing qualitative and quantitative information on research systems was collected from across the programmes with additional in-depth investigations of specific aspects of research systems drawn from selected consortia. Our researchers synthesised the evidence and presented it back to consortia members for their feedback throughout the programme (eg, through presentations and seminars at annual meetings, reports and publications). More details about the programmes, the methods used to obtain information about research systems and associated publications can be found at CCR|Liverpool School of Tropical Medicine (lstmed.ac.uk). The researchers involved in each programme met regularly to reflect on and summarise findings and five (representing all of the programmes) came together at workshops in 2021–2022 to compare findings concerning research systems, culture and environment across all the programmes. Through these discussions, we identified examples of practical actions that consortia had used to strengthen organisations’ research systems and cultures and challenges they encountered. Four complementary groups of examples emerged, acting on different levels of the research system: (a) individuals’ knowledge and skills; (b) capacity strengthening ethos; (c) organisations’ visibility and prestige; and (d) inclusive and responsive management practices. Our recommendations are based on our findings and these examples.

Individuals’ knowledge and skills

Capacity strengthening ethos: sharing, encouraging and networking

Consortia were often able to enhance postgraduate training programmes in their partner organisations; for example, by providing mentors, supervisors or technical experts from within and beyond their own organisation for PhD students. In addition to creating new relationships across organisations, this motivated students and improved their performance and confidence, contributing to a more constructive research environment.

Several consortia facilitated the sharing of equipment, training and expertise among their members (eg, on laboratory techniques or project financial management) and supported exchange visits (including between LMIC organisations) for all those involved in the consortium. In addition to providing technical skills and exposure to good practices concerning research cultures elsewhere, this inclusive approach clearly signalled that all members were valued and acknowledged.

Some consortia had budget allocations to support capacity strengthening for organisations’ priorities rather than for the specific needs of the research programme, for example, to develop guidelines or improve internet provision.

Occasionally, if consortia had more than one partner in the same country or city, there were opportunities to share office space or even jobs. Where common needs were identified across several consortia, access to central support was provided through the programme. The process of sharing resources among organisations helped to make the strengths of organisations explicit and also enhanced the technical capability and systems of weaker organisations within the consortia.

Some consortia incorporated activities that would enable income generation, upgrading and expansion of their member organisations’ services, thereby fostering sustainability. This was particularly apparent in consortia that involved laboratory testing (eg, for water and soil analyses), because if these tests were externally certified to be of internationally recognised quality, they could market their services at commercial rates.

Consortia leaders also had to overcome difficulties in personal relationships within and between organisations, for example, between: supervisors and PhD students; experienced and less experienced researchers; researchers with different disciplinary backgrounds; and researchers and other professional staff such as technicians and managers. To address these problems, consortia leaders took time to familiarise themselves with the cultural, social and political context of their member organisations. They provided opportunities for confidential discussions, shared policies for dealing with bullying and harassment, and made sure that everyone understood that they had a responsibility for safeguarding.19 20

Enhancing organisations’ visibility and prestige

Many consortia capitalised on their size and networks, and the visibility associated with a large research programme, to have greater influence on research uptake than would have been possible by each organisation acting alone. Organisations’ knowledge of the local context helped consortia make sure there was a demand for the research before starting, and ‘influencers’ within their organisations provided access to national decision-makers who could guide and use the research.

Some consortia employed specialists in policy engagement and research dissemination to promote impact—expertise that was often not available within individual organisations.

Organisations that have a reputation for impactful research are more likely to be sustainable, so several consortia adopted strategies (eg, ‘communities’, meetings) to maintain organisations’ enhanced networks once funding had ended.

Some consortia fostered new interorganisational collaborations (eg, for proposal writing or training PhD students) involving activities beyond those of the original programme.

Engagement in inclusive and responsive consortium management practices

Managing the interface between the consortia research systems and those of their members’ organisations was a challenge for consortium leaders because they had to navigate multiple bureaucracies. They found ways to avoid setting up consortium-specific staff or systems in parallel to those in their partner organisations, aiming instead to build on existing systems as far as possible. They also introduced strategies to bridge the interface between the consortia’s systems and those of their member institutions, for example, by embedding consortia staff in member organisations and having equitable representation in consortia leadership teams.

Consortia leaders initiated early discussions with each of their member organisations to make sure they fully understood and were engaged with their research activities and objectives: this was particularly important for new members of the consortium and for those who had not been involved in these types of consortia before.

Consortia often put a lot of effort into understanding their partner organisations’ needs, and strengths and weaknesses, and to support areas that organisations found challenging. This early identification of weaknesses—often in data management and financial management of research—enabled consortia to put extra support in place, usually from within the consortia themselves.

Some consortia invited their member organisations to start up meetings to make decisions about whether to centralise the management of finances and research data in the lead organisations or to decentralise it to member organisations. This prompted organisations to reflect on their own research systems. These open discussions were important because some organisations welcomed centralised management of funds as an efficient way to procure research consumables and equipment, whereas others thought that centralisation showed a lack of trust in their organisations’ capabilities. Joint written agreements were developed between consortia and organisations concerning, for example, financial responsibilities, data management, authorship and intellectual property.

Opportunities for feedback about management of the consortia, and for constructive and safe discussions were important for enabling all those involved in the consortium to air their personal concerns (eg, on workloads or inequities) to the management team. These feedback and communication processes tended to be refined and adapted as the programme and the relationships across the consortium matured. In some cases, this ‘learning’ about how to do better in terms of research capacity strengthening and in responding to feedback was formalised as the responsibility of a dedicated team.

Although initially such discussions could be difficult, ultimately, having clarity and formal agreements around roles and responsibilities, and constructive feedback channels to a responsive management team, strengthened organisations’ research systems and engendered trust and respect between the consortia, the organisations and programme managers.

Measuring changes in research capacity is challenging since these may only become sustainable several years after the grant has ended and cannot easily be attributed directly to an individual programme. However, consortia did provide examples of how they evidenced the outcomes of their capacity strengthening efforts, which included improvements in infrastructure, grant income and organisations’ systems and processes.

Recommendations

Research consortia can play a major role in strengthening the research systems and promoting conducive research cultures in their member organisations. Ultimately, this not only enhances the quality of all the organisations’ research, but it also ‘leads to increased organisational resilience’ and the consolidation of research capacity.3 Currently, there is still a low level of investment in organisations’ research systems, especially in human resources such as professional services and research support staff, and this often leads to insufficient project management capacity across LMIC partners, especially in project execution, procurement and monitoring and evaluation.21

Many of the consortium-related benefits for organisations’ research systems can be realised through strategic investment in individuals,22 because a conducive environment will attract and retain a high-calibre research workforce. Consortia leaders therefore need to know how to achieve the dual benefits of developing individuals and aligning the development of individuals to organisations’ needs. This means being inclusive by providing equitable opportunities, not just for academics, but for all those involved in the research process (eg, research professional staff including librarians, accountants, IT technicians and those responsible for research governance).22

Although all the consortia we examined had a mandate from the funders to strengthen research capacity, they were given little guidance by the funders about how this could be best achieved. Furthermore, consortia leaders were primarily selected for their research expertise rather than their knowledge of good practice in strengthening research systems and could have benefited from more guidance and resources at the start. Consortia can open up training opportunities for staff in their member organisations who are not directly supported by the consortium and they can facilitate cascading of new skills and knowledge.22 Funders should encourage and monitor these activities since they are valuable and cost-effective ways of enhancing organisations’ research capacity.18

This ethos of inclusivity should also extend to valuing the contributions of all disciplines represented across consortia organisations. Multidisciplinary research is essential to tackle complex health, climate and other priority issues, and member organisations’ research systems and cultures need to be able to integrate and equitably support multiple disciplines. Consortia provide a good platform for bringing together diverse expertise but need to negotiate different power relations across their member organisations.23–26 It takes considerable—and frequently underestimated—time and resources to break down barriers and to create trusting and respectful relationships among disciplines.27 28 Unless this process is managed well and early, disunity can build up29 and consortium members may become stressed and demotivated, which may jeopardise the functioning of the whole consortium.18 30 31

Empowering individuals with knowledge and skills in management and research leadership benefits the consortium and also contributes to strengthening member organisations’ research systems.32 Consortia can promote empowerment by involving members in the active running of the programme18 and by capacitating them to take on increasing responsibilities within the consortium and in their organisations over the consortium’s lifetime. This requires a focus, not only on technical training and formal processes, but also on ‘soft’ skills such as communication and being respectful and equitable.18 It also requires mechanisms that allow consortium members to safely provide feedback and air concerns to optimise the capacity strengthening gains.18 22 33

Participating in and managing programmes that have dual ‘research’ and ‘capacity strengthening’ goals take substantial time for consortia and their member organisations, which is often underestimated. They need to balance efficiency (eg, a centralised management structure) and effectiveness (eg, decentralisation with strengthening of processes in partner organisations) of programme delivery.34–39 These two concepts—efficiency and effectiveness—could be complementary if consortia receive clear guidance from funders about their commitment to improving organisations’ research systems.40 41 This commitment should be reflected in funders’ requirements at all stages of the research process, from designing schemes and commissioning projects, to monitoring and evaluation.38 Funders’ selection panels should have expertise in, and weight their scores appropriately towards, strategies for strengthening organisations’ research systems.42

The more engaged a partner organisation is in consortia management, leadership and decision-making, the greater the benefit to both consortium and partner, and the more research capacities and systems will be strengthened.18 However, there has been almost no research into how to do this.17 An evidence-informed framework based on a theory of change43 has recently been published to help consortia with decision-making so they can maximise research capacity gains and returns on investments.18 44 However, without more knowledge of what works in different contexts and what does not, many opportunities to strengthen organisations’ research environments may be overlooked and, therefore, not included in consortia’s budgets and activities. Generating such knowledge is difficult because gains in research capacity may not become apparent until a few years after the end of a project, and by then attribution of any changes to an individual project may be unclear. Despite this, if consortia could better demonstrate their role, even in part, in system changes, this would strengthen the evidence required to make a case for greater focus and investments.

To achieve a stepwise change in research systems strengthening, funders need to send a clear message to consortium leaders that an organisation’s research systems and processes, and their research outputs, should both be ‘excellent’ and complementary,45 and based on the principles of ethical and equitable partnerships.33 46 47 Without such explicit statements, consortium leaders will likely continue to prioritise research outputs over embedding sustainable improvements in their organisations’ research systems.18 Funders need to recognise that outcomes for system-level research capacity strengthening efforts are not always quantifiable or produced in the short term, and they should be open to and work towards developing a wide range of evaluation indicators for assessing consortia performance. Much more evidence is needed about how to evaluate the research capacity strengthening components of consortia-based programmes. This includes better, validated tools and indicators and evaluation processes that are constructive for funders and grantees, and not overly burdensome. This means that funders need to identify knowledge and practice gaps and commission research to fill these gaps.

Our recommendations (box 2) are based on real-life evidence from several large consortia-based research programmes and, if implemented, they could contribute substantially to improving the design and operation of consortia-based programmes so they can more effectively achieve their research capacity strengthening outcomes. The transferability of our recommendations is enhanced because they are based on evidence collected through a variety of research methods, from programmes covering diverse research topics and from the multiple perspectives of different members of the research workforce including researchers, research managers, institutional directors, research funders, laboratory technicians and programme evaluators. However, the majority of organisations involved were academic and based in sub-Saharan Africa so the applicability of our findings beyond these contexts may be limited. It is possible that there may have been a bias among participants to over-report positive examples and so we may have missed covering some of the less successful aspects in our recommendations. However, we sought to mitigate this by ensuring that participants all understood that our learning research team was independent of the grant managers and research funders, and that the purpose of our involvement was to use their (anonymised) contributions to make positive ongoing improvements to their programme.