Outcome and condition sets, indicators and calibration for qualitative comparative analysis
ToC outcomes | Condition | Indicators | Data collection | Calibration |
Long-term outcome | Primary care service utilisation for mental illness | Monthly mental health patients as a percentage of monthly clinic patients averaged over the quarter | Facility profile | Monthly mental health patients as a percentage of monthly clinic patients was averaged over the quarter. Calibrated in STATA using a ‘direct’ transformation to calibrate levels of deviation from anchors, in terms of log odds. Anchors: 0 (fully out of the set), 1 (mid point) and 3 (fully in the set). |
Short-term and medium-term outcomes | Medication supply | Essential medications were available at the facility in the last month | Facility profile | Coded the availability of all medications outlined in the MHCP: all of the time (1), most of the time (0.75), sometimes (0.5), rarely (0.25) and not available (0). An average score was obtained for all medication. |
Supervision (facility) | All facility staff receive supervision on a regular basis as defined by the MHCP and guidelines | Facility profile | Number of monthly supervision visits provided at facility level was calculated as a proportion of the number recommended in the MHCP. This was averaged over the quarter for each type of supervision. The data was naturally calibrated into 0 (fully out of the set), 0.333 (one supervision session every 3 months, more out than in the set). 0.666 (two supervision session every 3 months, more in that out of the set), 1 (fully in the set). Following factor analysis, the data were averaged to create a mean facility supervision score. | |
Supervision (community) | All community staff receive supervision on a regular basis as defined by the MHCP and guidelines | Facility profile | Number of monthly supervision visits provided at community level was calculated as a proportion of the number recommended in the MHCP. This was averaged over the quarter for each type of supervision. The data were naturally calibrated into 0 (fully out of the set), 0.333 (one supervision session every 3 months, more out than in the set), 0.666 (two supervision sessions every 3 months, more in that out of the set), 1 (fully in the set). Following factor analysis, the data were averaged to create a mean community supervision score. | |
Trained human resources (facility) | Adequate numbers of trained human resources as per the MHCP are available at facility level | Facility profile; process data | Number of trained staff was calculated as a proportion of the government-recommended staff numbers for the cadre of worker. This was initially done for prescribers and non-prescribers but averaged following a factor analysis. Calibrated in STATA using a ‘direct’ transformation to calibrate levels of deviation from anchors, in terms of log odds. Anchors: 0 (fully out of the set), 0.6 (mid point) and 1 (fully in the set). | |
Trained human resources (community) | Adequate numbers of trained human resources as per the MHCP are available at community level | Facility profile | Number of trained FCHVs was calculated as a proportion of the government-recommended staff numbers for the cadre of worker. We did not disaggregate between FCHVs trained in detection and referral and those with additional training in home-based care. | |
Facility psychosocial interventions | Facility psychosocial interventions available | Facility profile | Average of binary variables indicating the availability (1) or absence (0) of basic and advanced psychosocial interventions for all four mental illnesses outlined in the MHCP. | |
mhGAP | mhGAP available | Facility profile | Average of binary variables for availability of mhGAP for all four mental illnesses outlined in the MHCP. | |
Referral to tertiary care | Referral to tertiary services available | Facility profile | Binary variable on availability of referral services in the last month. | |
Community identification | Increased number of cases detected and referred by community health workers | Facility profile | Average number of community identification detection tool referral forms received at the facility per quarter. Calibrated in STATA by ranking the variables and then standardising them between 0 and 1. Resultant fuzzy value is more out than in the set if there is less than 1 CIDT per month. | |
Community awareness and stigma reduction | Proportion of people reached by community awareness programmes | Facility profile | The cumulative proportion of people who had attended community sensitisation sessions was calculated using the cumulative number of people attending the sessions over the population of the area around each health facility. The indicator was calibrated in STATA using a ‘direct’ transformation to calibrate levels of deviation from anchors in terms of log odds. Anchors: 0 (fully out of the set), 0.01 (mid point) and 0.04 (fully in the set). |
CIDT, community informant detection tool; FCHV, female community health volunteer; MHCP, mental healthcare plan; mhGAP, Mental Health Gap Action Programme.