Research design and methods
An external evaluation of the QIIP-LC program was conducted using a rigorous mixed-method, multimeasure design. The evaluation included: (1) development of a logic model (see online supplementary file S1) and assessment oriented process evaluation; (2) a retrospective, cluster, matched-control, prechart and postchart audit on the management of T2DM and rate of CRC screening; (3) a controlled post-only survey of practices participating in the chart audit on advanced access to healthcare; (4) semistructured, post-only, in-depth telephone interviews; (5) post-only web-based participant survey; and (6) health administrative data analysis with the Institute for Clinical Evaluative Sciences.11
Primary outcome results11 and the impact of QIIP on team functioning12 are published elsewhere. This paper presents the results of the chart audit related to the impact of the QIIP-LC program on T2DM management, including: (1) glycemic outcomes and management; (2) vascular protection (hypertension and dyslipidemia) and (3) screening for DM-related complications. This paper also presents the chart audit findings related to healthcare utilization (healthcare professional (HCP) visits, specialist referral for diabetes care) and diabetes counseling, education and self-management goal setting.
Evaluation of the QIIP-LC was approved by the Research Ethics Board at Western University and Queen’s University. A waiver of patient consent for the chart audit was granted under the Ontario Personal Health Information Protection Act from each Ethics Review Board.
Design and participants
A retrospective, cluster, matched-control prechart and postchart audit was conducted for this component of the evaluation. PHC teams were randomly selected from a sampling frame based on the proportional distribution of participating teams: (1) model of care (academic, FHT and CHC); (2) wave of LC; (3) geographical region (Local Health Integration Network (LHIN) boundaries); and (4) practice setting (rural/urban). LHINS are geographical areas responsible for the regional administration of public healthcare services in the province of Ontario, Canada. Created 1 April 2007, they are mandated with planning, integrating, and distributing provincial funding as well as engaging with their local communities.13
One randomly selected physician per team (N=34) and matched control (N=34) were recruited. Control physicians were identified based on the sampling framework of their matched QIIP-LC physician using a pragmatic priority approach selecting controls. When identifying controls matched to a QIIP-LC physician within an FHT, ideally controls were ranked according to distance (prioritizing control physicians geographically further away to reduce contamination). For CHCs, controls were identified from alternate CHC practices, as most CHCs in Ontario operate in one practice location. In the event that an appropriate control physician did not exist, physicians from alternate FHT practices or community practices were recruited.
Physicians were included in the evaluation if they were in active clinical practice at least 1 year prior to the program start and had a minimum of 20 patients with T2DM in their practice. Physicians meeting eligibility criteria generated a list of patients with T2DM using ICD-9 250 billing code (International Classification of Diseases, Ninth Revision). Deceased patients and those residing in a nursing homes were excluded.
External, trained auditors were assigned to a physician. Auditors were not informed whether physicians were part of the QIIP-LC program or control group. Standard data abstraction forms were used. Auditors randomly selected patients from the list and screened for eligibility. Eligible patients were: (1) ≥18 years of age; (2) diagnosed with T2DM; and (3) diagnosed prior to the start of the audit timeframe. Eligible charts were reviewed for prespecified T2DM clinical process and outcome data 12 months prior to the LC (baseline), during the LC (intervention range: 15–17.5 months), and 12 months after the LC (post). Auditors were instructed to review charts until 12 eligible patients per physician were reached. Not all physicians had 12 eligible patients to contribute; therefore, the first dataset totaling 809 patients was used for analysis on all diabetes outcome measures. For analysis of glycemic outcomes, a second dataset was constructed of patients above study target A1c (≥7.3%). Not all physicians had five patients (from the original 12) above study target A1c; therefore, auditing continued until a minimum of 5 at baseline was met. The second dataset had a total of 310 patients. Data collection dates for control physicians were based on the time frame of their matched QIIP physician.
Primary outcome measures
Two primary outcome measures were calculated to evaluate the effectiveness of the QIIP-LC program on T2DM management. The primary clinical outcome was the A1c of patients above study target, while the primary process outcome was the proportion of patients with an annual foot exam. Investigators defined A1c to be above target for this study as 7.3% rather than the Canadian Diabetes Association Clinical Practice Guideline (CPG) of 7%14 in order to identify patients in whom glycemic treatment intensification was more likely.
Secondary outcome measures
Secondary clinical measures included glycemic outcomes and management (A1c value, A1c at CPG target, oral hypoglycemic agent (OHA)/insulin medication prescription), hypertension outcomes and management (systolic/diastolic blood pressure (BP) values, BP at CPG target and antihypertensive medication prescription) and lipid outcomes and management (low-density lipoprotein (LDL) value, LDL at CPG target and statin medication prescription). Intensification of glycemic, hypertension and lipid management were also included as secondary clinical outcome measures. Intensified glycemic management was characterized by adding an OHA, increasing the total daily dose of an OHA, adding insulin and/or increasing the total daily dose of an insulin. Intensified hypertension management was characterized by adding an antihypertensive and/or increasing the total daily dose of an antihypertensive. Intensified lipid management was characterized by adding a statin, increasing the total daily dose of a statin and/or switching statin medication to atorvastatin or rosuvastatin (higher potency statins).
Secondary process measures were the completion rate of screening for other diabetes-related complications (BP, lipid profile, albumin:creatinine ratio, glomerular filtration rate, serum creatinine, foot exam, eye exam, peripheral neuropathy exam, ECG exam, waist circumference documented and body mass index (BMI) documented). Visits to the PHC team were documented by date and provider type to determine the total number of visits to all HCPs. Documentation of diabetes counseling and education (exercise, weight, diet, smoking cessation, hypoglycemic events and adjustment of treatment plans) and self-management goals were collected by date. Specialist referrals were documented by date and type.
Sample size
Sample size calculations were conducted taking into account the sample size requirements for each primary outcome and adjusting for clustering and loss to follow-up. The final number of physicians required per group was 33.11
Analysis
Analysis was performed on all outcomes using SAS V.9.2.15 Three time periods were constructed for each outcome variable: (1) 12 months prior to the LC (baseline); (2) during the LC (during); and (3) 12 months after the LC (post). The generalized linear model (Proc Genmod in SAS) was used to compare change in outcome measures over time (baseline–during–post) between the QIIP-LC physicians (hereafter reported as QIIP group) and the control group, accounting for clustering within the physician’s practice and controlling for baseline measures.
Process/dichotomous outcome variables (eg, foot exam) were considered complete if documented in patient charts at least once during each time period. For continuous outcome variables (eg, A1c value), the most recent documented value during each time period was used. Missing values were populated with data carried forward from the previous time period. Other continuous outcome variables such as the number of visits and number of HCPs seen were calculated from the data set.
For each time period, the most recently prescribed medications and the total daily dose for each medication was used to construct treatment intensification variables by medication category (oral antihyperglycemic agents, insulin, antihypertensives and statins). Treatment intensification was determined by comparing baseline medication to the medication regimen during the LC or 12 months post-LC.