The agreement of patient-reported versus observed medication adherence in type 2 diabetes mellitus (T2DM)

Introduction

Poor medication adherence is a public health threat that increases the risk of disability and death1 and disproportionately affects those with chronic illness,2 poor access to healthcare,3 and low health literacy.4 The cost of non-adherence in the USA falls between $100 billion and $289 billion annually.5 ,6 Efforts to improve adherence require valid and reliable measures that can be easily integrated into the existing workflow of real-world clinical settings. Using a self-reported medication assessment tool such as the Morisky Medication Adherence Scale (MMAS)7 has shown conflicting results in many chronic illness populations, including heart failure,8–10 hypertension,11 ,12 mental illness,13 and diabetes.14–16 In the USA, as the prevalence of type 2 diabetes mellitus (T2DM) rises to nearly 10% of the population,17 improving adherence to medications that affect glycemic control will be increasingly important to preventing debilitating complications and death. More evidence is needed to establish the validity and clinical utility of self-reported measures of medication adherence for patients with diabetes.

Research design and methods

In this study, we examined the level of agreement between patient-reported medication adherence using the MMAS and directly observed assessment of pills and injectables by nurse practitioners (NPs) during home visits. In addition, we sought to determine the ability of each of these two measures to discriminate for higher versus lower levels of blood glucose (glycated hemoglobin, HbA1c >7.5).

The cohort comprised high-risk patients (n=430) from four geographically diverse counties participating in the Southeastern Diabetes Initiative (SEDI): Durham County, North Carolina, USA; Cabarrus County, North Carolina, USA; Quitman County, Mississippi, USA; and Mingo County, West Virginia, USA. The study design and intervention of SEDI have been previously described.18 Participants were adults with a diagnosis of T2DM and defined as high risk for serious adverse events or death. The criteria used to define ‘high-risk’ for this study included baseline indicators of diabetes complications such as microvascular and macrovascular damage, renal and cardiovascular comorbidities, and social factors available in the electronic health record, including insurance and marital status. These factors were integrated with geographic information and used to create a geographic health information system (GHIS), which combined patient-level and community-level risks and was used to generate a ‘risk algorithm’.18 Using this algorithm-generated score, we identified the top 10% of the population as ‘highest-risk’ and invited these patients to enrol in the intensive intervention as represented here (n=430). Exclusion criteria included an inability to make health-related decisions, a terminal illness with a life expectancy of 6 months or less, a diagnosis of T1DM, or gestational diabetes or pregnancy.

In each county, multidisciplinary healthcare teams—including a physician, NP, dietitian, pharmacist, licensed clinical social worker, and community health assistant—provided care. Participants provided written informed consent to participate in the clinical intervention, consisting of a home visit that included a physical assessment, review of medications, and completion of patient-reported outcome surveys for medication adherence, nutrition, physical activity, diabetes care self-efficacy, and health behaviors. At each home visit, NPs administered the MMAS—a validated, 8-question survey that assesses self-reported medication adherence.7 NPs determined medication adherence using pill counts, medication bottle dates, and direct observation of insulin administration. Through these measures, NPs gave a categorical score to reflect adherence. Patients were assigned medication adherence scores of 0–20%; 21–80%, or >80%, reflecting the proportion of actual medications taken. Adherence >80% of the time was considered adherent, a method previously described.10 ,19

We used Cohen's κ coefficient to determine the level of inter-rater agreement between the two measures. We evaluated the association between adherence and blood glucose using a Wilcoxon rank-sum test. Finally, we calculated the c-statistic to assess the ability of each measure to discriminate between high and low blood glucose.

Results

Of 536 patients enrolled in the clinical intervention, 430 had complete data for baseline MMAS scores, observed adherence, and HbA1c values and were therefore eligible for analysis. There were no systematic differences between participants and those excluded for missing data. Descriptive results are shown in table 1. About half of the patients were categorized as adherent, though self-reported adherence (n=261, 61%) was lower than directly observed adherence (n=338, 79%) (table 2). The level of agreement between the two adherence measures was fair (κ=0.24; 95% CI 0.15 to 0.33; p<0.0001). The median number of medications was 6.0 (IQR 7.0). In addition, oral medications were associated with higher rates of adherence as compared with subcutaneous injections (χ²=6.88; p<0.009).

HbA1c indicated uncontrolled blood glucose (HbA1c >7.5) in a majority of the cohort (n=350, 86%) (table 3). For both methods of assessment, higher adherence was significantly associated with lower HbA1c levels (p<0.001); yet, the ability of each measure to discriminate between lower and higher blood glucose was weak (c-statistics=0.63 and=0.61, respectively).

Discussion

In this large cohort of patients with high-risk diabetes across four geographically diverse counties of the Southeastern USA, the agreement between self-reported medication adherence and directly observed counts of pills, insulin pens, and medication bottles was only fair. Several possible conclusions may be drawn.

First, patients' perceptions of their medication-taking behavior may be inaccurate, confounded by the myriad of medications for which dosing instructions vary with life events, such as food intake, activity, or time of day. In this study, for example, 61% (n=261) of patients reported consistently taking medications as prescribed. Yet, of those, 31 people were actually considered non-adherent based on direct observation. Conversely, 40% (n=169) of patients reported poor adherence. Yet, of those, 110 people were doing better than they thought and were assessed as adherent by an NP. For many, this misperception of good versus poor adherence may simply be due to the high proportion of medications that are prescribed to be taken ‘as needed’ or in sliding scale doses. In previously reported studies, discrepancies in self-reported adherence have been attributed to the complexity of the medication regimen20 ,21 and the inherent difficulty in recognizing what adherence is.

Another interpretation of the results may be that in high-risk patients with T2DM, the MMAS may lack sensitivity and may be a poor indicator of actual medication use. Previous work with the MMAS in other populations has shown a strong correlation between self-reported adherence and actual adherence. In hypertension, for example, the MMAS has demonstrated a strong correlation between self-reported adherence and subsequent blood pressure control.7 ,12 Though early studies support the MMAS as being sensitive to actual changes in medication adherence, more recent studies have disputed these findings.11 ,22 ,23

In diabetes, numerous studies have evaluated self-reported medication adherence using the MMAS, some showing a positive relationship between adherence and HbA1c.24 ,25 In many of these studies, however, participant demographics differed widely from those in SEDI, with fewer comorbid illnesses, higher rates of insurance coverage, more frequent single-dose regimens, and higher levels of education and health literacy.16 In SEDI, factors that classified patients as high risk included recent hospitalizations, substance use, tobacco use, and multiple comorbidities—including coronary artery disease, hypertension, heart failure, or chronic kidney disease—all of which require complex medication regimens. As a result, patients may report that they are ‘getting enough medications’ daily, skewing self-reported results26 and suggesting that improvement in diagnostic measures is needed, particularly in illnesses with multiple comorbidities.

A second finding of this study was low discrimination. The c-statistics for both measures were similar, 0.63 for self-reported MMAS and 0.61 for direct observation. Though higher adherence scores were significantly associated with lower HbA1c values for both measures, neither was able to discriminate well between lower and higher levels of serum glucose control as indicated by HbA1c. Similar findings have been previously reported.11 ,16 One possibility is that other factors, beyond medication usage, are driving A1c in this high-risk population.

Regardless of the underlying reasons for lack of agreement between self-report and observed counts, every effort must be made to discover where, in this high-risk population, the breakdown occurs. Despite only fair agreement between self-report and observed medication counts, using the MMAS as a screening tool could improve identification of those highest risk patients in need of follow-up. The MMAS offers a quick survey that can be used in inpatient and outpatient as well as individual and group settings, and is useful for identifying those who need further evaluation and support for medication management. The individual reasons for non-adherence were not included as part of the analysis, as our focus was to evaluate and objectively report the agreement between patient-reported adherence and the observed assessment of a healthcare provider who was visualizing and counting bottles, vials, and pills. Though reasons for medication-taking behavior are critical in an intervention, this analysis was designed to provide evidence for the use of preintervention baseline data for determining the value of patients' self-reported medication adherence. Using these results substantiates the value of a baseline adherence assessment, and opens a forum for patients and providers to then earnestly consider reasons for various medication-taking behaviors and take these into account as part of the subsequent plan of care.

Limitations to this study include the following: a relatively long intake visit and placement of patient-reported outcome surveys, including the MMAS, at the end of the visit when patients may have been fatigued could affect results; patients took the self-reported survey while the healthcare member was present, which may have imposed an inadvertent Hawthorne effect; and a relatively high proportion of the cohort was unable to self-administer the survey due to blindness or very low health literacy. Though NPs read the survey for patients in the latter category, the results may have been skewed.

Conclusion

With 29 million diagnosed and undiagnosed people with diabetes,17 providing a brief but accurate assessment of medication adherence is a requisite tool for clinical practice. This study suggests that in chronic illnesses with multiple comorbidities in which a complex medication regimen is required, identifying more sensitive and discriminatory measures of medication adherence is necessary.