Impact of a primary healthcare quality improvement program on diabetes in Canada: evaluation of the Quality Improvement and Innovation Partnership (QIIP)

QIIP program description

From 2008 to 2010, QIIP used the Institute for Healthcare Improvement’s Breakthrough Series (IHI-BTS) Model7 to implement a QI learning collaborative (LC) program with teams of FPs, allied health professionals, and administrative staff from FHTs and community health centers (CHCs) across Ontario. IHI-BTS uses an adult learning model that typically brings together teams to learn from each other and experts in topic areas targeted for improvements.7 The overall purpose of QIIP was to educate, train and enable PHC teams to improve patient care and outcomes in three areas: (1) chronic disease management—program target: type 2 diabetes mellitus (T2DM) management; (2) disease prevention—program target: colorectal cancer (CRC) screening; and (3) office access and efficiency—program targets: ‘advanced access’ to healthcare and team functioning.

The QIIP-LC program was delivered over three waves; each wave was standardized in content and consisted of prework, three 2-day learning sessions (separated by action periods) and one summative congress. This ensured all participants received the same program. In total, each wave of the program lasted between 15.5 and 17 months. Learning sessions provided opportunities for teams to learn QI methodology such as Plan-Do-Study-Act cycles8 and Ontario’s Chronic Disease Prevention and Management framework.9 Learning sessions also provided a rich opportunity to network and learn from each other. The QIIP-LC program included support from QI coaches (QIC) whose role was to facilitate and mentor participants throughout the program and support electronic medical record optimization.10 A series of communication strategies or channels included a virtual office, list serve and monthly teleconferences between PHC teams and their QICs. Participation in QIIP was voluntary and open to PHC organizations across Ontario. More details about the QIIP-LC program characteristics are available in the program logic model (online supplementary file S1) and published in Harris et al.11

Design and participants

A retrospective, cluster, matched-control prechart and postchart audit was conducted for this component of the evaluation. PHC teams were randomly selected from a sampling frame based on the proportional distribution of participating teams: (1) model of care (academic, FHT and CHC); (2) wave of LC; (3) geographical region (Local Health Integration Network (LHIN) boundaries); and (4) practice setting (rural/urban). LHINS are geographical areas responsible for the regional administration of public healthcare services in the province of Ontario, Canada. Created 1 April 2007, they are mandated with planning, integrating, and distributing provincial funding as well as engaging with their local communities.13

One randomly selected physician per team (N=34) and matched control (N=34) were recruited. Control physicians were identified based on the sampling framework of their matched QIIP-LC physician using a pragmatic priority approach selecting controls. When identifying controls matched to a QIIP-LC physician within an FHT, ideally controls were ranked according to distance (prioritizing control physicians geographically further away to reduce contamination). For CHCs, controls were identified from alternate CHC practices, as most CHCs in Ontario operate in one practice location. In the event that an appropriate control physician did not exist, physicians from alternate FHT practices or community practices were recruited.

Physicians were included in the evaluation if they were in active clinical practice at least 1 year prior to the program start and had a minimum of 20 patients with T2DM in their practice. Physicians meeting eligibility criteria generated a list of patients with T2DM using ICD-9 250 billing code (International Classification of Diseases, Ninth Revision). Deceased patients and those residing in a nursing homes were excluded.

External, trained auditors were assigned to a physician. Auditors were not informed whether physicians were part of the QIIP-LC program or control group. Standard data abstraction forms were used. Auditors randomly selected patients from the list and screened for eligibility. Eligible patients were: (1) ≥18 years of age; (2) diagnosed with T2DM; and (3) diagnosed prior to the start of the audit timeframe. Eligible charts were reviewed for prespecified T2DM clinical process and outcome data 12 months prior to the LC (baseline), during the LC (intervention range: 15–17.5 months), and 12 months after the LC (post). Auditors were instructed to review charts until 12 eligible patients per physician were reached. Not all physicians had 12 eligible patients to contribute; therefore, the first dataset totaling 809 patients was used for analysis on all diabetes outcome measures. For analysis of glycemic outcomes, a second dataset was constructed of patients above study target A1c (≥7.3%). Not all physicians had five patients (from the original 12) above study target A1c; therefore, auditing continued until a minimum of 5 at baseline was met. The second dataset had a total of 310 patients. Data collection dates for control physicians were based on the time frame of their matched QIIP physician.

Primary outcome measures

Two primary outcome measures were calculated to evaluate the effectiveness of the QIIP-LC program on T2DM management. The primary clinical outcome was the A1c of patients above study target, while the primary process outcome was the proportion of patients with an annual foot exam. Investigators defined A1c to be above target for this study as 7.3% rather than the Canadian Diabetes Association Clinical Practice Guideline (CPG) of 7%14 in order to identify patients in whom glycemic treatment intensification was more likely.

Secondary outcome measures

Secondary clinical measures included glycemic outcomes and management (A1c value, A1c at CPG target, oral hypoglycemic agent (OHA)/insulin medication prescription), hypertension outcomes and management (systolic/diastolic blood pressure (BP) values, BP at CPG target and antihypertensive medication prescription) and lipid outcomes and management (low-density lipoprotein (LDL) value, LDL at CPG target and statin medication prescription). Intensification of glycemic, hypertension and lipid management were also included as secondary clinical outcome measures. Intensified glycemic management was characterized by adding an OHA, increasing the total daily dose of an OHA, adding insulin and/or increasing the total daily dose of an insulin. Intensified hypertension management was characterized by adding an antihypertensive and/or increasing the total daily dose of an antihypertensive. Intensified lipid management was characterized by adding a statin, increasing the total daily dose of a statin and/or switching statin medication to atorvastatin or rosuvastatin (higher potency statins).

Secondary process measures were the completion rate of screening for other diabetes-related complications (BP, lipid profile, albumin:creatinine ratio, glomerular filtration rate, serum creatinine, foot exam, eye exam, peripheral neuropathy exam, ECG exam, waist circumference documented and body mass index (BMI) documented). Visits to the PHC team were documented by date and provider type to determine the total number of visits to all HCPs. Documentation of diabetes counseling and education (exercise, weight, diet, smoking cessation, hypoglycemic events and adjustment of treatment plans) and self-management goals were collected by date. Specialist referrals were documented by date and type.

Sample size

Sample size calculations were conducted taking into account the sample size requirements for each primary outcome and adjusting for clustering and loss to follow-up. The final number of physicians required per group was 33.11

Analysis

Analysis was performed on all outcomes using SAS V.9.2.15 Three time periods were constructed for each outcome variable: (1) 12 months prior to the LC (baseline); (2) during the LC (during); and (3) 12 months after the LC (post). The generalized linear model (Proc Genmod in SAS) was used to compare change in outcome measures over time (baseline–during–post) between the QIIP-LC physicians (hereafter reported as QIIP group) and the control group, accounting for clustering within the physician’s practice and controlling for baseline measures.

Process/dichotomous outcome variables (eg, foot exam) were considered complete if documented in patient charts at least once during each time period. For continuous outcome variables (eg, A1c value), the most recent documented value during each time period was used. Missing values were populated with data carried forward from the previous time period. Other continuous outcome variables such as the number of visits and number of HCPs seen were calculated from the data set.

For each time period, the most recently prescribed medications and the total daily dose for each medication was used to construct treatment intensification variables by medication category (oral antihyperglycemic agents, insulin, antihypertensives and statins). Treatment intensification was determined by comparing baseline medication to the medication regimen during the LC or 12 months post-LC.

Primary outcome results

Mean A1c was significantly lower in the QIIP group during the 12-month program (p=0.01); however, these improvements were not sustained post-LC (p=0.10, table 3). No difference between groups was observed for foot exams (p=0.15, table 4).

Secondary outcome results

No significant changes in other glycemic outcomes and management were observed between groups (in the subset of patients above study target (A1c ≥7.3%)) (table 3). Similarly, no changes in vascular protection from baseline were observed for hypertension outcomes and management (BP management and medication, table 3), while lipid outcomes and management highlighted that more patients in the QIIP group achieving CPG target LDL cholesterol (≤2.0) over time compared with the control group (p=0.03). No other significant differences in lipid management between groups were found. Table 4 presents the screening for diabetes-related complications. Lipid profile testing (p=0.02), eye examinations (p=0.03), peripheral neuropathy exams (p=0.01), and documented BMI (p=0.04) were significantly higher for the QIIP group. Patients in the QIIP group had significantly more visits to HCPs from baseline compared with the control group (p=0.001, table 5). There were no differences between groups for the number of HCP seen (p=0.39) or the number of patients who saw more than one HCP per visit on average (p=0.89) (an indicator of team-based patient care). QIIP group referrals to a specialist decreased from 31.5% (pre) to 28.8% (post), while control group referrals increased slightly from 28.5% (pre) to 30.0% (post); however, these differences were not statistically significant, p=0.08. The proportion of patients who had documented counseling/education for diabetes (weight, diet, exercise, hypoglycemic events, treatment plan, smoking cessation, or self-management goals) over the study period was examined, with no significant differences between groups (p=0.43).

Strengths and limitations

The QIIP-LC program evaluation was designed to address the growing consensus in the literature for more rigorous study designs of QI interventions.33–38 This evaluation expands the literature on QI evaluation methodology with its use of a control group, a rare study design in research and evaluation of QI programs. To maximize the effectiveness of the control and minimize the risk of contamination, control physicians were recruited using a pragmatic priority approach selecting controls ranked according to distance. Furthermore, collection of clinical chart audit data by independent audit rather than self-report heightened the strength of the findings, and the stratified random selection of participants, pre–post, multimeasure design added strength to this rigorous methodology. QIIP PHC teams were challenged to focus on more than one QI area at a time in a busy clinic setting,11 and it is possible that diabetes was not the focus of teams assessed by chart audit for T2DM outcomes. QIIP had a number of program targets (T2DM, CRC screening, advanced access and team functioning), and it would be beneficial for future evaluations to have the capacity to assess the degree to which participants focused on each target area of the intervention.

Complex QI initiatives are challenging to evaluate, and the QIIP-LC program was no exception. The retrospective nature of the evaluation may have limited the controlled design, and use of the last observation carried forward technique to impute missing values in the dataset may have resulted in a biased estimate of the intervention, or underestimated the variability of the results. Future analyses would benefit from statistical techniques designed to understand the magnitude of this bias and complement existing methods for dealing with missing data. Furthermore, participation bias existed on two levels. Clinical teams volunteered to participate in the program suggesting their interest in improving their practice. This bias was amplified by virtue of informed consent to participate in the evaluation. Participants who refused may have done so based on their lack of interest in QI interventions or their perceived lack of change or improvements. The impact of the QIIP-LC program may have been diluted by a number of simultaneous Ontario MOHLTC provincial initiatives and mandates targeting improvements in diabetes management in primary care.39–42 Design and implementation of QI programs would benefit from complimentary, rather than overlapping initiatives to optimize the impact of these programs. Finally, physicians and their respective teams were encouraged through the QIIP-LC to develop sustainability plans to integrate successful concepts into the practice and to share lessons learnt in the program to colleagues both within and external to the collaborative. Sustainability and spread of lessons learnt were key program components of the QIIP-LC program, and improvements in both QIIP and control groups could be an indication that changes initiated by QIIP-LC participants were integrated into the whole practice organization and thus, control practices within the same FHT may have adopted these new learnings.