You are here

  • Home
  • Archive
  • Volume 8, Issue 1
  • Addressing practical issues of predictive models translation into everyday practice and public health management: a combined model to predict the risk of type 2 diabetes improves incidence prediction and reduces the prevalence of missing risk predictions

Article Text

Article menu

Download PDFPDF

XML
Cardiovascular and Metabolic Risk
Addressing practical issues of predictive models translation into everyday practice and public health management: a combined model to predict the risk of type 2 diabetes improves incidence prediction and reduces the prevalence of missing risk predictions
  1. Martina Vettoretti1,
  2. Enrico Longato1,
  3. Alessandro Zandonà1,
  4. Yan Li2,
  5. José Antonio Pagán3,4,
  6. David Siscovick5,
  7. Mercedes R Carnethon6,
  8. Alain G Bertoni7,
  9. Andrea Facchinetti1,
  10. Barbara Di Camillo1
  1. 1Department of Information Engineering, School of Engineering, University of Padova, Padova, Italy
  2. 2Icahn School of Medicine at Mount Sinai, New York, New York, USA
  3. 3Department of Public Health Policy and Management, New York University, New York, New York, USA
  4. 4Center for Health Innovation, New York Academy of Medicine, New York, New York, USA
  5. 5Research, Evaluation & Policy, New York Academy of Medicine, New York, New York, USA
  6. 6Department of Preventive Medicine, Northwestern University, Chicago, Illinois, USA
  7. 7Division of Public Health Sciences, Wake Forest University Health Sciences, Winston-Salem, North Carolina, USA
  1. Correspondence to Dr Barbara Di Camillo; barbara.dicamillo{at}unipd.it

Abstract

Introduction Many predictive models for incident type 2 diabetes (T2D) exist, but these models are not used frequently for public health management. Barriers to their application include (1) the problem of model choice (some models are applicable only to certain ethnic groups), (2) missing input variables, and (3) the lack of calibration. While (1) and (2) drives to missing predictions, (3) causes inaccurate incidence predictions. In this paper, a combined T2D risk model for public health management that addresses these three issues is developed.

Research design and methods The combined T2D risk model combines eight existing predictive models by weighted average to overcome the problem of missing incidence predictions. Moreover, the combined model implements a simple recalibration strategy in which the risk scores are rescaled based on the T2D incidence in the target population. The performance of the combined model was compared with that of the eight existing models using data from two test datasets extracted from the Multi-Ethnic Study of Atherosclerosis (MESA; n=1031) and the English Longitudinal Study of Ageing (ELSA; n=4820). Metrics of discrimination, calibration, and missing incidence predictions were used for the assessment.

Results The combined T2D model performed well in terms of both discrimination (concordance index: 0.83 on MESA; 0.77 on ELSA) and calibration (expected to observed event ratio: 1.00 on MESA; 1.17 on ELSA), similarly to the best-performing existing models. However, while the existing models yielded a large percentage of missing predictions (17%–45% on MESA; 63%–64% on ELSA), this was negligible with the combined model (0% on MESA, 4% on ELSA).

Conclusions Leveraging on existing literature T2D predictive models, a simple approach based on risk score rescaling and averaging was shown to provide accurate and robust incidence predictions, overcoming the problem of recalibration and missing predictions in practical application of predictive models.

  • type 2 diabetes
  • prevention
  • risk factor modeling
  • modeling
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjdrc-2020-001223

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    View Full Text

    Footnotes

    • Presented at Part of this study was presented in abstract form at the 12th International Conference on Advanced Technologies & Treatments for Diabetes, Berlin, Germany, February 20–23, 2019.

    • Contributors MV conceived and designed the work, analyzed and interpreted the data, and drafted and critically revised the manuscript. EL and AZ analyzed and interpreted the data, and critically revised the manuscript. BDC conceived the work, interpreted the data, drafted, and critically revised the manuscript. AF, YL, JAP, DS, MRC, and AGB interpreted the data and critically revised the manuscript. BDC is the guarantor of this work and, as such, had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    • Funding The MESA study was supported by contracts HHSN268201500003I, N01-HC-95159, N01-HC-95160, N01-HC-95161, N01-HC-95162, N01-HC-95163, N01-HC-95164, N01-HC-95165, N01-HC-95166, N01-HC-95167, N01-HC-95168, and N01-HC-95169 from the National Heart, Lung, and Blood Institute, and by grants UL1-TR-000040, UL1-TR-001079, and UL1-TR-001420 from the National Center for Advancing Translational Sciences (NCATS). The ELSA study is funded by the National Institute on Aging (R01AG017644), and a consortium of UK government departments coordinated by the National Institute for Health Research. This research was funded by: the European Commission within the Horizon 2020 PULSE project, ID 727816, from January 11, 2016 to October 31, 2019; MIUR (Italian Minister for Education) under the initiative “Departments of Excellence” (Law 232/2016).

    • Competing interests None declared.

    • Patient consent for publication Not required.

    • Ethics approval Ethic approval was not required for this study because this study analysed datasets collected by previous studies (MESA and ELSA) which were available to the authors of this paper in anonymized form, without any link to the real subjects.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Data availability statement Data are available on reasonable request. Data are available from the website of the MESA study (https://www.mesa-nhlbi.org/Publications.aspx) and the ELSA study (https://www.elsa-project.ac.uk/accessing-elsa-data) on reasonable request.

    • Author note The ELSA was developed by a team of researchers based at the University College London, NatCen Social Research, and the Institute for Fiscal Studies. The data were collected by NatCen Social Research.