Article Text
PDF
Abstract
Introduction To estimate adherence to clinical practice guidelines in selected settings at a population level for Australian children with type 1 diabetes mellitus.
Research design and methods Medical records of children with type 1 diabetes mellitus aged 0–15 years in 2012–2013 were targeted for sampling across inpatient, emergency department and community visits with specialist pediatricians in regional and metropolitan areas and tertiary pediatric hospitals in three states where approximately 60% of Australian children reside. Clinical recommendations extracted from two clinical practice guidelines were used to audit adherence. Results were aggregated across types of care (diagnosis, routine care, emergency care).
Results Surveyors conducted 6346 indicator assessments from an audit of 539 healthcare visits by 251 children. Average adherence across all indicators was estimated at 79.9% (95% CI 69.5 to 88.0). Children with type 1 diabetes mellitus have higher rates of behavioral and psychological disorders, but only a third of children (37.9%; 95% CI 11.7 to 70.7) with suboptimal glycemic control (eg, hemoglobin A1c >10% or 86 mmol/mol) were screened for psychological disorders using a validated tool; this was the only indicator with <50% estimated adherence. Adherence by care type was: 86.1% for diagnosis (95% CI 76.7 to 92.7); 78.8% for routine care (95% CI 65.4 to 88.9) and 83.9% for emergency care (95% CI 78.4 to 88.5).
Conclusions Most indicators for care of children with type 1 diabetes mellitus were adhered to. However, there remains room to improve adherence to guidelines for optimization of practice consistency and minimization of future disease burden.
- clinical practice guidelines
- children's quality of care
- pediatric type 1 diabetes
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Significance of this study
What is already known about this subject?
Clinical practice guidelines have been developed to help identify and minimize risk factors for complications of type 1 diabetes mellitus, and ensure that treatment targets are met.
What are the new findings?
In a population-level survey of three Australian states, children with type 1 diabetes mellitus aged 0–15 years in 2012–2013, care was in keeping with clinical practice guidelines on average 80% of the time.
This did not differ significantly by acuity, location or care setting.
However, screening for psychological disorders in children with suboptimal glycemic control only occurred in a third of children (37.9%; 95% CI 11.7 to 70.7).
How might these results change the focus of research or clinical practice?
Clinical care is generally not affected by acuity, location or care setting; however, there remains room for increasing adherence to guidelines around screening for psychological disorders.
Introduction
Type 1 diabetes mellitus (T1DM) is an increasingly common and chronic illness that often begins in childhood.1 It is associated with substantially increased morbidity and mortality risks. Epidemiological data suggest a reduction in lifespan of 8–13 years, which has been primarily attributed to the cardiovascular disease associated with the condition.2 3 Australia has a high incidence of T1DM, with >10 000 children affected nationally.4 As such, T1DM has been classified as a national health priority by the Australian government.5
To reduce the long-term burden of illness, it is important to identify and minimize risk factors for complications of T1DM, and ensure that treatment targets are met.6 Despite the significant health impacts of T1DM, worldwide data suggest that risk factors are not minimized and targets are not met. Clinical practice guidelines (CPGs) have been developed to help optimize and standardize the delivery of evidence-based care of children with T1DM across all healthcare settings, which has proven health benefits.7 Historically, compliance with guidelines has been suboptimal and, more recently, efforts have been made to close this gap.8–14
The CareTrack Kids (CTK) study assessed adherence to CPGs for 17 conditions in Australian children aged 0–15 years, in 2012 and 2013, including T1DM.15 Here, we present and discuss the proportion of children with T1DM that received care in line with CPGs at indicator level, in hospitals and from pediatricians in private practice in the community.
Methods
The CTK methods have been described in previous publications.15–17 The authors describe the relevant aspects of this analysis of T1DM results.
Development of indicators
We defined a clinical indicator as a measurable component of a standard or guideline, with explicit criteria for inclusion, exclusion, time frame and practice setting. Indicators were derived by modification and application of the RAND-UCLA Delphi method. A systematic search for local CPGs related to care of children with diabetes was conducted. Two CPGs related to T1DM were found and 233 candidate recommendations were extracted according to a specified protocol.17 We initially screened recommendations for eligibility and excluded recommendations based on four criteria: (1) weak strength of wording (eg, ‘may’ and ‘could’); (2) low likelihood of the information being documented in medical records; (3) guiding statements without recommended actions and (4) ‘structure-level’ recommendations (eg, requirements for healthcare professionals to be aware of local policies). Seventy-seven recommendations were screened out, leaving 156 for clinician review.
Two stages of clinician review were conducted. Internal review was undertaken using email by clinicians associated with the CTK study, two pediatricians and a general practitioner. Recommendations were excluded if they were assessed as having low acceptability, feasibility, appropriateness, impact or if the concept was covered in another recommendation. At the end of three rounds of internal review, a further 137 recommendations were removed, leaving 19 to be considered by external reviewers.
External reviewers were recruited via advertisements to the members of relevant medical colleges, and through local networks. Five pediatricians were recruited to undertake three rounds of external review, on a customized wiki site, using the same criteria as internal reviewers. Four recommendations were removed through this process, and the remaining 15 recommendations were broken into 35 distinct audit indicator questions (see online supplementary appendix 1 for full listing).
Supplemental material
Sample size and sampling process
The CTK study audited the medical records of children aged ≤15 years receiving care in 2012 and 2013. We targeted 400 medical records for T1DM and 6000 medical records for 16 other conditions across regional, metropolitan and tertiary healthcare settings. For every visit for T1DM care found in the 6400 medical records, a separate assessment of indicator adherence was made.
Details on the general sampling methods are provided in the original paper15; additional details specific to T1DM can be found in online supplementary appendix 2. Briefly, we sampled medical records at four healthcare settings: (1) hospital inpatients, (2) emergency departments (EDs), (3) general practices (GPs), providing primary care and (4) pediatricians in community-based settings. These settings were chosen within randomly selected health department administrative units (‘health districts’) in the Australian states of Queensland, New South Wales and South Australia; approximately 60% of Australian children live in these three states. Sampling of GPs for T1DM was restricted to a subset of the GPs; given the relative rarity of the condition and the restricted sampling, only 7 GPs were identified as providing T1DM services to children, with 10 children having 31 visits—as the representativeness of these results is unclear, we decided to restrict the current analysis to hospital-based and community-based specialist services. Figure 1 illustrates the breakdown of medical record reviews by state and healthcare provider type, for the three selected types of setting. Response rates in the main CTK study varied by setting: 92% (34 of 37) of eligible hospitals agreed to participate, but the estimated recruitment rates were 25% (20 of 80) for pediatricians (see online supplementary appendix 2).15
Supplemental material
Visits of children for diabetes care, by state and healthcare provider type*. *Total number of visits to emergency departments=269; total number of admissions to hospital=186; total number of private pediatrician consultations=84. Total number of diabetes assessments in: New South Wales=263; Queensland=209 and South Australia=67. Total number of visits assessed for care of diabetes in sampling frame=539.
Data collection
Data were collected by eight experienced pediatric nurses (surveyors), trained to assess eligibility for indicator assessment and compliance with CPGs. Indicators that were not relevant to a particular setting were automatically designated as not applicable; for each other indicator, surveyors determined whether the indicator was relevant to the visit and, if judged relevant, assessed compliance as ‘yes’ (care was compliant) or ‘no’ (relevant, but compliance not documented).
Analysis
Adherence at indicator level was estimated as the proportion of indicators relevant to the visit that were assessed as compliant (‘yes’). Indicators were aggregated by type of care, and adherence was also estimated for each type: ‘diagnosis’ assessed the presence of diagnostic confirmation and baseline investigations (indicators 1–4; n=4); ‘routine care’ assessed the quality of routine clinical care (indicators 5–12 and 28–30; n=11) and ‘emergency care’ assessed the emergency care of children presenting with diabetic ketoacidosis (indicators 13–27 and 31–35; n=20). Results by care type were separately estimated by stratum (tertiary hospitals, which have statewide responsibilities, metropolitan and regional geographical location) and by healthcare setting.
Sampling weights were constructed as specified in online supplementary appendix 2, to adjust for oversampling of states, health districts and healthcare settings. The weighted data were analyzed in SAS/STAT V.9.4 (SAS Institute, Cary, North Carolina, USA), using the SURVEYFREQ procedure. Variance was estimated by Taylor series linearization and the primary sampling unit (health district) was specified as the clustering unit. Stratification and, where appropriate, domain analyses were used (see online supplementary appendix 2). Exact 95% CIs were generated using the modified Clopper-Pearson method, except when the point estimate was 100% where the unmodified Clopper-Pearson method was used.18
Indicator and group estimates were both suppressed if there were <25 eligible visits. Adherence rates for different care types were not compared statistically because the results are not independent; in one visit, for example, both diagnostic and routine care indicators were assessed. Similarly, ED and inpatient results were not compared statistically as they are drawn from the same medical records. Within each care type, we compared pairwise differences in adherence rates for routine care between strata (metropolitan vs regional vs tertiary hospital) and within the routine care type we calculated differences between pediatricians and EDs, and between pediatricians and inpatients (there was insufficient pediatrician data in the diagnosis care type, and no pediatrician indicators in the emergency care type). Where calculated, statistical significance was estimated using the F-test approximation of the Rao-Scott χ2 test, which adjusts for the design effect.
Results
Information about the 251 children with one or more eligible assessments of adherence to CPG for T1DM is provided in table 1. Almost half the children were aged 12 years or older, with almost equal number of males and females. Each child had a median of two healthcare visits across the 2 years (range 1–14).
Characteristics of the 251 children with visits for diabetes
Of 19 635 possible recommendation assessments, 1897 (9.7%) were automatically filtered out as not being applicable to the selected healthcare setting, and 11 392 (58.0%) were designated as not applicable or otherwise ineligible, for example, not every healthcare visit was for diabetic ketoacidosis and so those indicators would not be assessed. The field team conducted 6346 eligible assessments grouped into 539 visits, at a median of 10 indicators per visit (range 1–26). Diabetes indicators were assessed in 9 pediatrician’s practices, 33 hospital EDs and 27 hospital inpatient settings.
Quality of care
The assessed adherence for each indicator is shown in table 2. The average adherence across all indicators and settings was 79.9% (95% CI 69.5 to 88.0).
Adherence by indicator
Adherence is not reported for 6 of the 35 indicators, as they were assessed in <25 visits. For the 29 indicators where adherence was reported, compliance ranged from 37.9% for indicator 5 ‘Children and adolescents diagnosed with type 1 diabetes who presented with suboptimal glycemic control (eg, HbA1c >10% or 86 mmol/mol) were assessed for co-occurrence of psychological disorders using a validated screening tool’, to 100% for indicators 14 ‘Children and adolescents with type 1 diabetes who presented with signs of DKA had their vital signs monitored’ and 25 ‘Children and adolescents with type 1 diabetes who presented with severe DKA (blood glucose >11 mmol/L, venous pH <7.1, bicarbonate <5 mmol/L) and hypoperfusion (delayed capillary return, tachycardia for age) had their fluid type adjusted according to ongoing sodium, potassium and glucose levels’. The IQR for individual indicator adherence was 71.6%–96.3%.
The only indicator that was adhered to in fewer than half of the visits was screening for psychological disorders in children presenting with suboptimal glycemic control (37.9%; 95% CI 11.7 to 70.7; indicator 5). There was also poor adherence to indicators requiring documentation of sick-day emergency plans, including the need for ketone measurement (50.8%; 95% CI 25.3 to 76.0; indicator 29) and details of 24 hours sources of clinical advice (56.8%; 95% CI 40.3 to 72.3; indicator 30). An indicator about education on carbohydrate counting had two-thirds adherence (67.7%; 95% CI 21.6 to 96.5; indicator 28).
For each of the three care types, table 3 shows the estimated adherence in Metropolitan and Regional geographical strata and separately in tertiary pediatric hospitals, which have state-wide roles. Within each care type, the estimated adherence did not statistically significantly differ between the three strata (p>0.50 for each pairwise comparison). Adherence to indicators relating to diagnosis was estimated at 86.1% (95% CI 76.7 to 92.7), 78.8% for routine care (95% CI 65.4 to 88.9) and 83.9% for emergency care (95% CI 78.4 to 88.5).
Adherence by care type and geographic/tertiary hospital strata
Similarly, table 4 presents the estimated adherence by healthcare setting for each of the three care types. There were no statistically significant differences between the estimated adherence by healthcare setting for routine care, the only care type where difference could be assessed: community pediatricians’ adherence did not differ from adherence in inpatient settings (80.0% vs 82.1%, p=0.82) or ED settings (80.0% vs 62.0%, p=0.18).
Adherence by care type and healthcare setting
Discussion
This study characterizes the adherence of healthcare provision to CPGs, at a population level, in children with T1DM, aged 0–15 years from 2012 to 2013, restricted to hospital settings and community-based pediatrician visits in three states where 60% of Australian children reside. We have previously reported that children presenting with T1DM, care was consistent with CPG indicators three-quarters of the time (75.8%; 95% CI 66.5 to 83.6).15 Restricted to hospital settings and community pediatricians, the overall estimate was 79.9% (95% CI 69.5 to 88.0). The current analysis demonstrates that, within each care type (diagnosis, routine care and emergency care), adherence was similar across metropolitan and rural geographies and tertiary hospitals. Care was also similar between community pediatricians and hospital settings for indicators relating to routine care. Although the overall rate of adherence was high, there were some important indicators with low adherence.
Children with T1DM have an increased prevalence of behavioral and psychological disorders.19–21 These disorders are predictive of worse long‐term outcomes and decreased quality of life.21 So, it was disappointing to see that only approximately a third of children with suboptimal glycemic control were screened for psychological disorders using a validated tool (indicator 5), although this should be interpreted cautiously as the CI shows a wide range. Few clinicians would challenge the importance of mental health, or the need for screening, but this study highlights it is one of the areas where there is a possible evidence-practice gap. Similarly, few would argue against the provision of sick-day emergency plans (indicators 29 and 30), but this too was an area where the selected professional groups underperformed, with roughly half having a documented plan that included either blood ketone measurement or details on 24 hours access to clinical advice. This group of indicators may be promising targets for local quality improvement activities.
Other indicators with lower adherence may reflect diversity in clinical practice. For example, an indicator about education on carbohydrate counting (indicator 28) had two-thirds adherence (67.7%), possibly reflecting the diversity of views on appropriate insulin management within the specialty.7
Previous studies have assessed the impact on glycemic outcomes for patients attending diabetes clinics in metropolitan versus rural areas but only at a state-based level.22–24 Our study shows that in Australia, in three states containing the majority of the national pediatric population, the care provided at diagnosis, during routine follow-up and in an emergency is comparable regardless of location. We also found that the quality of routine care provided did not differ significantly between pediatricians in the community in comparison to ED or inpatient settings. These are important findings for future healthcare planning, especially considering the geographical isolation of many Australian children and the associated difficulty in accessing tertiary healthcare.
Our study findings, although derived from an Australian setting, are similar to international data. This includes self-reported adherence,8 and prospective cohort studies11 25 from the USA and Canada which assessed adherence to CPGs in the management of children with diabetes. The agreement of findings derived from studies using different methodologies provides greater confidence in the robustness of the result. CTK assessed a broader range of indicators than most. For example, the study by Amed et al11 defined ‘optimal adherence’ as three diabetes-related physician visits/year, three HbA1c tests/year, one glucagon prescription dispensed/year and appropriate screening for diabetes-related comorbidity and complications. This contrasts with the 35 indicators ranging from diagnosis to routine and emergency care assessed in our study.
To our knowledge, only one study has attempted to improve adherence to T1DM practice guidelines in children.26 This novel study found positive effects of a computerized decision aid in a diabetes clinic to improve the rate of screening for diabetes-related complications.26 However, this study was limited by its retrospective evaluation and was only conducted in a single outpatient clinic. Given the increasing prevalence and burden of T1DM, it is surprising that further work in this area has not been performed.
The strength of our study lies in its extensive design in collating indicators from two sets of guidelines, combined with expert review, a comprehensive set of indicators used and large sample size in ED and inpatient settings. The assessment of adherence to CPG recommendations used a standardized, objective, peer-reviewed, quality-controlled format across a variety of healthcare settings and providers. This allowed assessment of the state of healthcare based on objective criteria without bias across multiple jurisdictions.
The study has several limitations. For example, guideline adherence is not always reflective of patient or healthcare provider perspectives on quality of care. A previous study has indicated that non-adherence may be attributed to a variety of reasons such as omission, systems issues or a conscious decision by the treating clinician not follow a particular recommendation.27 Data acquisition in our study was reliant on medical records documentation meaning the full extent of the care provided may not have been captured, potentially leading to an underestimation of actual adherence. Moreover, the study was designed to assess guideline adherence, so we unable to comment on the extent to which adherence correlates with clinical and patient-level outcomes.
Another limitation is that healthcare for pediatric diabetes management is often provided in hospital outpatient clinics. Our sampling strategy only identified 31 visits in 7 of the 84 GPs sampled, so we excluded these from consideration in the current study, and we did not formally sample visits to all hospital outpatient clinics. A post hoc assessment of six children attending outpatient clinic visits in one tertiary hospital in each participating state, using the same indicators and assessed by the same surveyors, revealed an overall estimated adherence rate of 84.7% for 469 routine care indicators, not dissimilar to routine care in tertiary hospitals overall (76.0% from 603 routine care indicators; 95% CI 52.1 to 91.9). Finally, clinical practice is constantly evolving and the clinical practice guidelines we used do not necessarily reflect the latest developments in the field. For example, indicator 23 recommends a 10 mL/kg bolus of 0.9% normal saline for children with severe DKA but recent evidence suggests that this advice may not be superior to alternative fluid regimens; nevertheless, experts recommended this indicator for care in 2012 and 2013.28 Finally, we note that guideline adherence may have changed since 2012–2013. While up-to-date feedback to clinicians would be preferable, health services research at this scale represents a significant logistical exercise resulting in unavoidable delays; the methods can be adapted for more rapid deployment at local level to provide clinicians with prompt feedback for quality assurance purposes.
Conclusion
A three-state sample of Australian children aged 0–15 years, in 2012 and 2013, with T1DM received care in line with CPG indicators an average of 80% of the time in hospital ED and inpatient setting and from community pediatricians. Importantly, compliance was similar across different care types, healthcare settings and providers. There remains room for improvement in adherence to CPG recommendations—in particular, screening for psychological disorders and provision of sick day management plans. This study acts as an important step towards promoting standardized care, improving outcomes and minimizing future disease burden.
References
Footnotes
Collaborators on behalf of the CareTrack Kids investigative team.
Contributors JB and PH had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Concept and design: JB, PH, CTC, SBD and RGMG. Acquisition, analysis or interpretation of data: JB, PH, GA and HPT. Drafting of the manuscript: JB, PH, RGMG, GA, HPT, CTC and SBD. Critical revision of the manuscript for important intellectual content: JB, PH, RGMG, GA, HPT, CTC and SBD. Statistical analysis: JB, PH, GA and HPT. Obtained funding: JB, PH and CTC. Administrative, technical or material support: JB, PH and GA. Supervision: JB, PH and CTC.
Funding The CareTrack Kids study was funded by an Australian National Health and Medical Research partnership grant (APP1065898), with contributions by the National Health and Medical Research Council, Bupa Health Foundation, Sydney Children’s Hospital Network, New South Wales Kids and Families, Children’s Health Queensland and the South Australian Department of Health (SA Health).
Map disclaimer The depiction of boundaries on the map(s) in this article do not imply the expression of any opinion whatsoever on the part of BMJ (or any member of its group) concerning the legal status of any country, territory, jurisdiction or area or of its authorities. The map(s) are provided without any warranty of any kind, either express or implied.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval The authors received primary ethics approval from relevant bodies including hospital networks and the Royal Australian College of General Practitioners (HREC/14/SCHN/113; HREC/14/QRCH/91; HREC/14/WCHN/68; NREEC 14–008), and site-specific approvals from 34 sites. Australian Human Research Ethics Committees can waive requirements for patient consent for external access to medical records if the study entails minimal risk to providers and patients; all relevant bodies provided this approval. Ethics approvals included reporting by healthcare setting for condition-level papers. Participants were protected from litigation by gaining statutory immunity for the CTK study as a quality assurance activity, from the Federal Minister for Health under Part VC of the Health Insurance Act 1973 (Commonwealth of Australia).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available from the authors on reasonable request.