Article Text
PDF
Abstract
Background Too few pharmacists receive formal training on substandard and falsified (SF) medical products. Strengthening knowledge across pharmacists is considered a moral and ethical duty of academia, that is, to build the health systems’ capacities to combat this global health threat these poor-quality products represent. This study therefore aimed to evaluate whether a dedicated educational course for undergraduate pharmacy students can improve their knowledge on these products.
Methods A survey was conducted at three sub-Saharan universities. Knowledge was assessed through scores on a 20-point questionnaire with questions related to the course content. Scores were compared before and after the course, and a linear mixed-effects model analysis was used to analyse score differences. Students were furthermore asked for feedback and self-assessment. In addition, teachers were interviewed on the context of the course introduction. These data were analysed descriptively.
Results Among 335 out of 355 students who completed the survey (n=41/53 in Cameroon, n=244/252 in Senegal and n=50/50 in Tanzania), knowledge of SF medical products was enhanced, with increase in all countries, overall, by 3.5 (95% CI 3.1 to 3.9) score points. Students improved in all offered modules in each country. Students confirmed their improvement through self-assessment.
The course was well received among students and teachers. Barriers included time constraints and access to practical means (equipment availability, room allocation, internet accessibility and affordability). These barriers can be overcome by key enablers such as the support from university leadership and early involvement of the university in the course design.
Conclusions The course improved students’ knowledge on SF medical products. These findings encourage further full implementation of this course in existing curricula beyond the pilot and can inform possible future scale-up. This has a potential for reinforcing the capacity of health systems to protect communities from SF medicines, by empowering all pharmacist across the health systems to intervene.
- Health education and promotion
- Health systems
- Public Health
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. All data relevant to the study are included in the article or uploaded as supplementary information. Contact the corresponding author if you wish to receive more information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Substandard and falsified (SF) medicinal products are a serious threat to public health and are especially found in those countries that have under-resourced regulatory authorities.
Although pharmacists can play an essential role in protecting the public against SF medicines, too few are trained well enough to fulfil this responsibility.
WHAT THIS STUDY ADDS
A jointly developed educational course on SF medicinal products for undergraduate pharmacy students showed an improvement of knowledge in three pilot countries: Cameroon, Senegal and Tanzania.
Teachers that were involved in the implementation of the course indicated that the readily available teaching materials were considered very helpful.
A barrier for the successful implementation was the availability of sufficient time for preparation and the deployment of the course.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This study can inform other universities that wish to implement a dedicated course on SF medicines in their pharmacy curriculum.
Empowering all pharmacist across the health systems to address the socially and ethically unacceptable issue of SF medical products has a potential for reinforcing health systems’ capacity and safeguarding public health.
Introduction
Substandard and falsified (SF) medical products are a threat to public health.1 2 In addition to the obvious harm caused to patients exposed to these poor-quality medical products, they also contribute to antimicrobial resistance, undermine immunisation programmes, erode confidence in healthcare professionals, medicines and health systems, and waste precious financial resources through prolonged treatment.3–6 They are morally unacceptable because they are not ordinary consumer goods and patients have the right to expect that the products they receive are safe and efficacious.
Countries with under-resourced regulatory authorities tend to be most vulnerable to SF medical products reaching patients.7–9 Between September 2012 and February 2022, the WHO Global Surveillance and Monitoring System has received 3169 case reports from 139 countries.8 10 Thirty-eight per cent of reports originate from the African region.7 Most serious incidents of SF medicines are often reported by healthcare professionals, in particular pharmacists.8 11 However, too few pharmacists receive formal training on this issue during their undergraduate education.12–14
Pharmacists are important players at the interface between the health system and the patient (end-users) community. They constitute the last defence/checkpoint between patients and their treatment and thus have both ethical and professional responsibility to minimise any health risks. In order to address the training gap and advance the essential role of pharmacists in pharmaceutical and health systems, a course on SF medical products for undergraduate pharmacy students was developed as part of a pilot project funded by the European Commission, designed and implemented by the WHO and the International Pharmaceutical Federation (FIP) in 2019–2021 in close collaboration with the pilot universities.15 16 This course was then deployed as a new component into the curriculum for pharmacists in the sub-Saharan African region. It includes detailed information relating to the root causes of SF medical products, the products most at risk, early warning signals of their presence in the supply chain, how to avoid, detect and report SF medical products, and advise patients and consumers.15
The main objective of this study was to assess the change in knowledge about the topic of SF medical products in undergraduate pharmacy students who participated in this course, which was delivered at three selected pilot pharmacy schools in sub-Saharan Africa (Cameroon, Senegal and Tanzania). The study also looked at self-assessment of the students’ knowledge prior to and after the course, and whether they found the course useful and worthy to recommend to other students. In addition, teachers were interviewed on the context of the course introduction and motivation of the students as well as main enablers and barriers of its implementation.
Methods
Course characteristics
The course was developed by an informal expert group, composed by pharmacy experts from academia and practice from FIP, pilot universities, project partners of the Commonwealth Pharmacists Association (CPA) and La Conférence Internationale des Ordres de Pharmaciens Francophones (CIOPF), local African experts with regulatory background and WHO, who sponsored the pilot project. It was developed through a consultative process; several meetings with multiple rounds of review and consultation were held until a consensus was reached. The consultation took into account multicultural dimension. The course is available in both English and French language. It is designed to cover comprehensive information about SF medical products in order to teach pharmacy students how to avoid, detect and report SF medical products, and how to advise affected patients. The course is composed of six modules on SF medical products: module A on general introduction; module B on identification of medical products at risk; module C on prevention (from entering the supply chain); module D on detection strategies; module E on reporting; and module F on intervention after coming in contact with them.15 As this was a generic course, the pilot universities in Cameroon, Senegal and Tanzania were responsible for considering necessity of adjustment of the content, using the ‘adopt and adapt principle’15 to contextualise the content and for flexibly incorporating the course into universities’ existing curriculum based on their needs.15
The teaching materials included case studies. The module on detection (module D) also included practical lessons, with universities teaching visual inspection (all) and analytical inspection with qualitative and semiquantitative investigations of medicines (Senegal and Tanzania).
The course was ready for deployment as of 2021. It was deployed face-to-face despite challenges and delays related to the COVID-19 pandemic. Each university deployed all six modules. An overview of the characteristics of the participating universities is included in table 1.
Characteristics of participating universities
Study sample
A total of 355 students participated (n=53 in Cameroon, n=252 in Senegal and n=50 in Tanzania) in this study. The pilot universities were chosen based on recommendations from the WHO and FIP, based on academic leadership and willingness to carry out the project. Two other universities who were also part of this pilot but did not yet deploy the course were excluded from the study. The students were at least in the third year of their study programme. The participation of the students in the evaluation was voluntary, and they were excluded from the analysis if they did not enter any information in the questionnaire.
Interviews with teachers
To understand how the course was deployed, a follow-up interview was conducted with the participating teachers from each university. These teachers were all directly involved in the implementation of the course after following a teaching (Train-the-Trainer) course themselves. A semistructured interview was used to assess enablers and barriers in deployment of the course and motivation of the students. An interview guide was developed with a list of ten open-ended questions. The list of the open-ended questions that were used for the semistructured interview and shared with the teachers beforehand in the interview guide can be found in the online supplemental appendix.
Supplemental material
Student survey
In the pre-assessment, students were asked in an open question to write out their prior knowledge about SF medical products. The answers were coded based on their correspondence to the course’s six modules.
In the post-assessment, students were asked (1) to what extent their knowledge on SF medical products had improved after participation in the course, (2) to indicate in which areas (linked to each of the six modules) their knowledge had improved, (3) to what extent this course will be helpful in their professional life and (4) if they would recommend this course to all pharmacy students and/or pharmacists. Answers were analysed by using descriptive analysis.
Knowledge assessment
No suitable questionnaire or assessment was found in the available literature that could be used for knowledge assessment in the context of this study. Therefore, an assessment questionnaire was developed and validated by the expert group who developed the course. The assessment comprised of 20 multiple choice questions related to the learning objectives of the curriculum. Each correctly answered question yielded one point. No points were deducted for incorrectly answered questions. Assessment responses from the students were received via computers or mobile devices with a web-based software tool, QuestionPro. None of the questions were mandatory. Fully uncompleted assessments, however, were excluded from the analysis.
The same assessment questions were used in the pre-test and post-test to ensure the compatibility of the results. The students were asked to fill out an assessment questionnaire before the course and after the course. The study approach is visualised in the online supplemental appendix.
Data analysis
The effects of the course on assessment scores were examined via linear mixed-effects model analysis. This method was chosen to account for both within-person and across-person variability. The statistical model contains both fixed effects and random effects. In addition, it is an appropriate method of analysis when not all data can be matched. Some participants only completed the pre-test or post-test leading to missing data, and linear mixed-effects model analysis allows to use all the data. One model was built with the assessment score as dependent variable and with fixed effects of time (pre-course or post-course). The country and subject number (student number) were considered as random effects. The validity of the statistical model was evaluated by using residuals to verify the normality assumption. The two-tailed significance level was set at α=0.05. All statistical analyses were performed with IBM SPSS Statistics, V.25 and Microsoft Excel V.2016.
The questionnaire aimed to cover all modular areas, but the emphasis was not given to an equal distribution; therefore, some questions were linked to more modules than others. Module E on reporting focuses on a national situation and is therefore not suitable for a uniform questionnaire, thus was not part of the assessment. Initially, it was not the intention to assess improvement per module, but a post hoc descriptive analysis was undertaken to explore this. The assessment scores were stratified by module and the total obtained score (percentages) per module was determined, both before (pre-test) and after (post-test) the course. The percentages were determined by dividing the obtained total score per module over the theoretical maximum score (number of questions per module multiplied by the number of students who completed the assessment) multiplied by 100%.
Data from the student survey and interview responses from the semistructured interviews were clustered into four themes (enablers, barriers, motivation of students and important environmental factors) with important quotes highlighted.
Results
Pre-assessment versus post-assessment knowledge scores
A total number of 335 students (response rate 94.4%) filled out the questionnaire. Overall, as displayed in table 2, student knowledge scores improved by 3.5 points (95% CI 3.1 to 3.9) out of 20 points after they took the course. When stratified by country, students showed a statistically significant improvement of 3.5 points (95% CI 2.2 to 4.8) in Cameroon, 3.4 points (95% CI 3.0 to 3.8) in Senegal and 5.3 points (95% CI 2.9 to 7.7) in Tanzania (figure 1).
Assessment score pre-test versus post-test.
Deployment of SF course at different universities
Students improved in each measured module across all countries (figure 2). In Senegal, the biggest improvement (15.6%) was observed in module C, covering information about the prevention of SF medical products from entering the supply chain. In Cameroon, the largest improvements (14.0%) were observed in module D on detection and in Tanzania (43.2%) in module B related to the identification of medical products at risk.
Percentages of questions answered correctly, stratified by modular theme in Cameroon, Senegal and Tanzania. ‘Introduction’ corresponds to module A and n=5 questions; ‘Identification’ corresponds to module B and n=3 questions; ‘Prevention’ corresponds to module C and n=1 questions; ‘Detection’ corresponds to module D and n=4 questions; ‘Intervention’ corresponds to module F and n=7 questions.
Self-assessment by students
A total number of 246 students described their prior knowledge of SF medical products (Cameroon n=51, Senegal n=167, Tanzania n=28, response rate 69.3%). Among them, 36 students (14.6%) indicated no previous knowledge. If any knowledge was indicated, students in all three countries indicated that they mostly had some prior knowledge of the general aspects about SF medical products (52.6% in Senegal; 42.9% Cameroon; 60.0% Tanzania). Prior knowledge per module per country can be found in the online supplemental appendix.
Students overall declared a similar improvement in all modules, approximately around 20% per module. The online supplemental appendix lists country-specific improvements per module. Most of the students indicated that their knowledge improved because of participating in the course (figure 3). Also, the large majority found the course useful for their professional life and would recommend the course to all pharmacy students and/or pharmacists around the globe. No large differences were observed between countries.
Students' attitudes towards knowledge improvement, future recommendation to others and usefulness of the course.
Interview sessions with teachers
Interviews with the teachers helped to understand the context and key enablers and barriers of course deployment. These are listed in table 2 with selected quotes included.
Discussion
This jointly developed educational course on SF medical products for undergraduate pharmacy students improved their knowledge on these poor-quality medical products. Knowledge increased overall and for each individual module in all countries, both as assessed through the questionnaire and according to student self-assessment. The course was well received among students and faculty, and enablers and barriers of deployment were identified. This may encourage further full implementation of this course in existing curricula beyond the pilot and inform possible future scale-up. It also indicates that prevention and detection of SF medical products should move from being a top-down exercise hold by specialists to being integrated into the pharmaceutical system through training of practitioners in a bottom-up approach.
The course was building on rather poor (self-reported) prior knowledge of students. Nearly one-sixth of the responding students reported no previous knowledge, while others reported some knowledge, mainly related to general aspects about SF medical products. This does not come as a surprise given earlier observations from WHO and others.3 8 12 While of high importance, current pharmacy school curricula largely lack systematic teaching on SF medical products.13 When introducing a new topic in a healthcare curriculum, the implementation typically undergoes different phases, from the exploration phase where the topic is being explored via early-adopting programmes (such as this pilot programme) to the initiation phase during which adoption across healthcare educational programmes is enacted. This phase will require continuous feedback to adjust appropriate educational delivery mechanisms. After this follows a standardisation phase in which knowledge is spread throughout the profession.17
To inform such process, it is important to identify key barriers and enablers of course implementation.18 One of the key barriers observed was insufficient time. In all countries, teachers reported that they would welcome more hours to be either dedicated to this course or to the preparation that preceded the deployment. Insufficient time in pharmacy or medical-related curricula is a well-known barrier to a new course introduction.13 19–21 This barrier can be overcome by support from university leadership,12 which was observed in Tanzania in the present study where the course development coincided with overall university curriculum revision, which resulted in more support for this course as well. Furthermore, the course would benefit from removing barriers of access to practical means, and more resources to provide for equipment, sufficient teaching space, internet accessibility and affordability, etc.
On the enablers side, the readily available teaching materials were considered very helpful resources for the teachers to facilitate the delivery. As previous studies found, availability of ready-to-adopt course materials is an important enabler.13 Participation of universities in the course development, providing input and insights of what is likely to be the needs of their students and overall ownership and commitment in the pilot project was a crucial enabler of success as has been reported previously.22
The course was developed as a modular one. When looking at individual themes, even though the universities eventually decided not to carry out the course per module and instead run the modules interchangeably, students improved in all measured modules in each country. The results therefore indicate the students improved in various areas of the course content. This was also reflected in the overall improvement across all modules in the student’s self-assessment. This indicates that the course was effective in reaching its objective of increasing student’s knowledge on SF medical products. Yet, this is not reason to stand still. As Dizon et al showed, even if students (momentarily) improve their knowledge through training, they still need to continue developing such knowledge and skills.23 The importance of continuous (lifelong) learning is consistently highlighted.24–26 This calls for efforts to integrate training on SF medical products into continuous education for pharmacists.
Strengths and limitations
This is the first study bringing insight into the education on SF medical products and deployment of the comprehensive and dedicated undergraduate pharmacy course in sub-Saharan Africa, but there are several limitations to note. First, not all participating students completed both the pre-assessment and post-assessment questions due to practical issues. Therefore, a linear mixed-effects model was used to be able to include all the available data.
Second, the number of questions was not equally distributed across modules—balancing between optimising the length (and respective time burden on students) with a total of 20 questions only, and full representation of the course content. Moreover, each university had the possibility to adjust the teaching content to their needs. Therefore, knowledge questions related mainly to common aspects across modules.
Third, as the same assessment questions were used in the pre-test and post-test, students could theoretically have memorised the questions in favour of the post-test scores. However, the assessment answers were not shared after completing the pre-test, making this assumption less plausible. Fourth, although the course has led to a statistically significant increase in knowledge, it is uncertain how sustainable the knowledge acquired is. Also, it is unclear how long this increase will remain, which is subject for further research. It was also not the aim of the study to compare the countries given the differences in the implementation context. Although there were differences across schools, it is difficult to draw any conclusions.
Despite the limitations cited, the study presents supporting evidence for the success in this pilot course implementation with the aim to close the gap in SF medical products education.
Future directions
The findings of this study can inform future pilots and institutionalisation of the training. As soon as the two other universities who were initially involved at the pilot project but have not yet introduced the course (Nigeria and Uganda) will deploy the course, findings can be compared across the whole pilot. This would also allow to investigate further the bilingual (francophone and anglophone) and cultural aspect of the curriculum, as this has been a unique feature of this course. The results could be compared against insight studies on awareness/risks/access in the similar socioeconomic segments of the general population in each country. Recent studies conducted by the WHO in Uganda and Nigeria can serve as a starting point.
Future insights could be given into long-term knowledge acquisition (namely, whether the score would change in 2–5 years). Furthermore, it could be investigated whether the students used the acquired knowledge in their practice-oriented professional careers. Continuous professional development for the (wider) workforce also needs consideration.
While this course was implemented in sub-Saharan Africa, clear and consistent themes emerged that can be applied in expanding this course to other geographical regions—a step towards providing pharmacists with the knowledge to contribute to minimising of the threat to the populations around the globe. Adaptations of this course (language, scope, etc) could then be deemed elsewhere.
Conclusions
The undergraduate pharmacy students who participated in the dedicated educational course improved their knowledge of SF medical products. Knowledge improved in all countries across all measured modules, both as assessed through an assessment and according to student self-assessment. The course was well received among students and teachers, who found the course useful and would recommend the course further. Enablers and barriers to deployment were identified, which can inform further full implementation of this course in existing curricula beyond the pilot and possible future scale-up to other pharmacy schools. Empowering all pharmacists across the health systems—instead of specialist training only—to address the socially and ethically unacceptable issue of SF medical products has a potential for reinforcing health systems’ capacity and safeguarding public health.
Data availability statement
All data relevant to the study are included in the article or uploaded as supplementary information. All data relevant to the study are included in the article or uploaded as supplementary information. Contact the corresponding author if you wish to receive more information.
Ethics statements
Patient consent for publication
Ethics approval
The study was conducted under Dutch legislation that indicates ethical approval is not required for this type of study. The study was conducted according to the principles of the Directive 95/46/EC General Data Protection Regulation (GDPR). Informed, specific and explicit consent was freely given by the students who participated in the study. Furthermore, informed, specific and explicit consent was freely given by the teachers who were interviewed, and the interview guide was shared at least one week beforehand. Participation by students and teachers was completely voluntary and could be terminated at any moment. The data were handled anonymously and none of the findings is traceable to the individual participant. Data are stored on a secure location and only accessible to the core research team. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors would like to thank Dr Ir Irma Meijerman, Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands, for her input to study design, Ms Svetlana Belitser, statistician at the Utrecht Institute for Pharmaceutical Sciences (UIPS), Utrecht University, Utrecht, The Netherlands, for her insights in the statistical analysis and Dr Catherine Duggan, the CEO of the International Pharmaceutical Federation (FIP) for the final review of this article. Given this research was conducted from international partnerships, a structured authors’ reflexivity statement was added in a supplementary appendix to the paper. It summarises the measures authors put in place in their efforts to promote equitable authorship in this publication.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Seye Abimbola
Twitter @ZKusynova, @eliangiringa1
Contributors All authors were involved in conceptualising the study. ZK and YB collected and analysed the data with the local help of GE-L, EK and SOS. ZK and YB drafted the manuscript and all authors reviewed the paper. AM-T served as the overall guarantor for the study.
Funding The development and implementation of the course (six university modules based on a competency framework) was financially supported by the European Commission and the WHO.
Disclaimer The views expressed in this article are the personal views of the authors and must not be understood or quoted as being made on behalf of any of the affiliated organisations.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.