Impacts of performance-based financing on health system performance: evidence from the Democratic Republic of Congo | BMC Medicine

Setting

With an estimated maternal mortality rate of 473 per 100,000 live births [16] and an under-5 mortality of 81 per 1000 births (World Development Indicators 2022), the DRC remains among the countries with the poorest health indicators globally. Seventy percent of the population, currently estimated at 96 million, lives below the international poverty line of 2.15 USD a day (World Development Indicators 2022). Even though a relatively high share of women receives at least some antenatal care (82%) and delivers in health facilities (82%), major gaps in other areas such as vaccination coverage and quality of health services remain [17,18,19,20,21].

In terms of the health system, the country’s 26 provinces are divided into 516 health zones that typically comprise a single first-level referral hospital and 12–20 health centers; each health center is responsible for a catchment area of about 10,000. The current health expenditure per capita was 21 USD in 2020, only about half of the 39 USD low-income country average. Central government health spending amounts to only 16% of the total health expenditure (Global Health Expenditure Database). It covers a small share of health workers’ income and most health facilities do not routinely receive financing or other resources. As a result, health facilities, whether public or private, heavily rely on user fees to remunerate their staff, procure supplies, and cover other operating costs [22].

The study presented here was conducted in 58 health zones across six provinces covered by a World Bank-financed project. The provinces are Kwango, Kwilu, and Mai-Ndombe in the west of DRC and Haut Katanga, Haut Lomami, and Lualaba in the southeast. Additional information on the zone selection is presented in Additional file 1.

The performance-based financing scheme

The DRC PBF program has been implemented by the Ministry of Public Health since 2016 with financing from the World Bank as part of the larger Health System Strengthening Project. The program offered contracted health facilities quarterly payments conditional on the volumes of primarily reproductive, maternal, and child health services. A complete list of services, fee scales, and additional information on the program design are presented in Additional file 1: Table S1-S2.

In addition to the quantity-based incentives, facilities also received quality-based payments based on a detailed quality checklist. As shown in the Additional file 1: Tables S3-S4, the checklists contained a range of indicators related to both structural and process quality [23] and were completed primarily through review of documents and registries, and verification of the availability of different supplies. The quality bonus was proportional to the quarterly rewarded for the number of incentivized services provided (quantity bonus). Facilities that scored less than 50% on the quality checklist did not receive any quality bonus. Health centers could receive a maximum bonus of 25% of quantity bonus, while hospitals could get a maximum bonus of 40% of the total quantity-based transfers made.

The financing received through the intervention, equaling approximately $1.6 USD per capita per year, did not replace any source of funding previously received by the facilities. However, the program introduced rules for how contracted facilities use their revenues, whether received as PBF payments or through any other source. Facilities were allowed to spend a maximum of 50% of each quarter’s revenue on personal bonuses for staff. The amount received by each staff member depended mostly on individual quarterly performance evaluations based on an evaluation tool developed by the project and to a lesser extent on grade, title, and seniority. The rest of the budget needed to be allocated based on a quarterly business plan, which facility managers developed using another structured tool developed for the project. Since 2020, facilities were required to spend a minimum of 20% of the PBF payments on medications and other consumables.

A detailed data reporting verification and counter-verification system, involving both reviews of facility registers, and tracking of users at the community level, was set up to minimize the risk of erroneous or fraudulent reporting. In addition, health zone teams were incentivized to conduct routine supervision and coaching of health facilities, including assistance in the elaboration of the quarterly business plans.

Experimental design

Out of the 58 evaluation health zones, 30 were selected for the PBF program through public randomization ceremonies conducted in each province in the presence of representatives from all health zones. The randomization was blocked by province. Additional file 1: Figure S1 shows the spatial location of the zones and the randomization results.

In the control health zones, facilities received quarterly transfers equaling the average transfer made to facilities in the PBF zones in the same province, adjusted for the population in the catchment areas and equity classification. There was no verification of service volumes reported by facilities in the control zones, and there was no assessment of their quality with the quality checklist. These facilities also did not need to comply with the program rules regarding spending their revenues and allocation of staff bonuses. However, control facilities also had to spend a minimum of 20% of their transfers on medications since 2020.

Facilities in both treatment arms received an initial investment transfer at the beginning of the project’s implementation as well as a one-off shipment of medications and family planning products. The unconditional financing began at the same time as the PBF program was launched in each province: the last quarter of 2016 in the provinces Kwango, Kwilu, and Mai-Ndombe and in the third quarter of 2017 in the other three provinces.

Data collection

To allow a comprehensive assessment of program impact, two major rounds of data collection at the facility and household level were conducted: a first round in 2015 (baseline), and a second round in 2021 (endline). In the first step, five health centers were randomly selected together with the main referral hospital within each health zone for an in-depth health facility assessment. The health facility assessments included a general facility survey completed by the facility-in-charge; interviews with health providers; direct observations of under-5 outpatient consultations, ANC visits, family planning consultations, and deliveries; and patient exit interviews. This facility-based data was complemented with data from interviews with a representative set of households in the catchment area of each facility. To identify households, the research team first identified all villages (or neighborhoods) in the catchment area of each facility and then randomly selected one village for the survey. Within each selected village, all households were listed by the survey team. During the baseline, 10 women with current or recent pregnancies were selected for an interview in each village. During the follow-up, 27 women between the ages of 15 and 49 were selected per village for an interview, including women without recent pregnancies. Figure 1 summarizes the overall study design and Additional file 1 provides additional information on the data collection, sampling framework, and power calculations.

Study design. The results presented in the paper are only of the follow-up survey data. In Additional file 1, we show an analysis of the baseline data to test for balance between the control and the PBF groups

Theoretical framework

Figure 2 presents the theory of change underlining the expected changes in the PBF arm relative to the control arm receiving the unconditional transfers. The figure also illustrates the mechanisms through which PBF is hypothesized to improve outcomes relate to all six building blocks of the World Health Organization (WHO) Health System Framework [24]. In response to the conditionality of the financial incentives, together with the enhanced supervision and verification, facilities are expected to improve the quality of care, which should attract more patients over time. Following the Donabedian framework [23], improvement can occur in terms of structural or process quality. Structural quality mostly captures service readiness while process quality captures the technical (clinical) and non-technical aspects of patient-provider interactions. Facilities are also expected to reduce user fees to further increase demand for their services. The supervision and verification, together with the management tools (quarterly business plan and staff evaluation tool) are expected to improve facility management; this could further contribute to quality improvements through improved decisions about resource allocations and coaching and monitoring of staff. Observing improved quality of care and reduced user fees, households are expected to use more services while paying less per service and to have higher satisfaction with health facilities.

It is theoretically ambiguous how the PBF program would affect staff satisfaction, given that by construction unconditional facility financing was set up to ensure equal financial resources would be available to facilities in both groups. Changes in the within-facility distribution of staff remuneration were of course possible. Intrinsically motivated providers could also react positively to the better structural quality as well as the more structured supervisions that can enable improvements in competencies. On the other hand, higher patient volumes and enhanced monitoring and reporting requirements could increase provider workload and reduce their job satisfaction.

Variables and outcome measures

We divided our outcome measures into seven domains included in the theory of change presented in Fig. 2. We considered three domains of service quality: structural quality, technical process quality, and non-technical process quality. Non-technical process quality combined measures of respectful care reported by enumerators observing consultations and deliveries, and users’ satisfaction reported during exit interviews. For service fees, we analyzed both the official service fees reported by facilities and payments reported by users during the exit interviews. We also analyzed binary measures related to fee policies and users’ satisfaction with service affordability. Providers’ satisfaction was measured through a series of work satisfaction questions directly asked to providers. Service coverage measures were based on household survey self-reports. In total, we analyzed 55 indicators in 8 groups, with many indicators calculated as indexes summarizing an even larger set of indicators. Detailed definitions of all indicators analyzed as well as their data sources are presented in Additional file 1.

Statistical analysis

We used multivariate linear regression models to estimate the impact of the PBF treatment on each outcome. All models controlled for the randomization block (province). For outcomes related to directly observed consultations, we controlled for facility, provider, and user characteristics. Standard errors were clustered at the health zone level – the level at which the PBF program was allocated. Additional information on the covariates and power calculation is presented in Additional file 1. Given the large number of outcomes analyzed, we used random-effects meta-analysis to generate a pooled average estimate within each domain. The main assumption underlying the random effects meta-analysis approach is that the true causal effects of an intervention are distributed normally around a central mean effect, which we believe is a reasonable assumption within each health systems domain. All statistical analysis was conducted using the Stata SE 16.0 software package.