Have trouble viewing this email? Click here

Clinical Prognostic Model Validation

An ICTS Research Methods Short Course

Tuesday November 18, 2014, 2:30 pm – 6:00 pm, Nelson Lecture Hall - Irvine

Course Description: Prognostic research involves building multivariable prediction models to accurately estimate patient prognosis. Prognostic estimates can be used for (a) clinical decision making (e.g., selection of treatments, follow-up diagnostic tests); (b) advising patients of the likely outcomes of their disease or suitability/risk of medical procedures; (c) assessing the added prognostic value of novel biomarkers or technologies among others. A prognostic/predictive model, prior to its broad application, should be rigorously validated. Model validation provides a quantitative assessment of how well the model fits the data at hand and how well it is expected to perform on new patients or patients in other settings / institutions. This short course will present model validation procedures for the 3 main prognostic models used in practice: Cox regression, logistic regression and linear regression. Illustrative examples will be implemented in the statistical software R.

Learning Objectives: Course registrants will have the opportunity to learn and understand:

  • The fundamental prediction problem and avoid common pitfalls like overfitting and sources of bias
  • Basic design considerations, like the number of predictor variables to use
  • Multivariable prognostic models (linear, logistic and Cox regression) for different measures of patient outcomes
  • How to measure predictive accuracy, including indexes of discrimination and calibration (shrinkage factor)
  • Bootstrap model validation and other validation procedures (e.g., cross-validation, data-splitting)
  • Implementation of model validation in the R software package (library rms - regression modeling strategies)

Topics: The prognostic/prediction modeling problem • Overfitting • Review of multivariable prognostic models including linear, logistic and Cox regression for continuous, discrete and censored patient outcomes, respectively • Design considerations for predictive model building • Variable selection procedures • Measuring predictive accuracy - internal validation • Indexes of discrimination (ability to separate patients with different outcomes): concordance index c, Somer's D • Calibration (extent of bias) • Bootstrap estimation of discrimination and calibration, optimism • Common bias from inadequate model validation • Cross-validation and data-splitting • External validation • Implementation of model validation primarily in the R statistical software package / library rms (basic modeling in SAS will also be covered) • Examples to be discussed, includes models for predicting 30-day mortality, kidney transplant failure, and patient discharge to institutional care facility after undergoing pulmonary lobectomy for cancer

Prerequisites: a) Basic understanding of the use and interpretation of linear, logistic and Cox regression; b) Basic familiarity with statistical software packages (e.g., coding in SAS or R). If you do not meet these prerequisites, please complete the tutorials below; this will help you achieve the course learning objectives. Tutorials:
http://www.nature.com/bjc/journal/v89/n3/pdf/6601119a.pdf (Paper in BJC; focus on Cox model on p. 431-434)
http://www.ats.ucla.edu/stat/r/dae/logit.htm (Logistic regression in R)
http://www.cdc.gov/nchs/tutorials/nhanes/NHANESAnalyses/LogisticRegression/Info1.htm (Logistic regression)
http://www.ats.ucla.edu/stat/sas/faq/oratio.htm (Logistic regression interpretation)

Registration Fee: UCI faculty and staff: $75, UCI student: $50, non-UCI registrant: $150 (fee includes lecture notes and coffee break)

To register: http://www1.icts.uci.edu/ccgateway/biostat_cpmv.cfm

Contact: Erika Whitton • (714) 456-2308 • ewhitton@uci.edu


About the Course Instructor: Dr. Nguyen is Professor in the Department of Medicine and Director of the Biostatistics, Epidemiology & Research Design (BERD) Unit, UC Irvine Institute for Clinical and Translational Science. Dr. Nguyen’s research involves several areas of basic, clinical and translational science, ranging from genomics, targeted treatment trials for individuals with Fragile X Syndrome to characterization of outcome risk trajectories in patients on dialysis, such as cardiovascular events and infection-related hospitalization. Stemming from his collaboration on biomedical research studies, Dr. Nguyen develops innovative statistical methods/tools to further clinical and translational science research. Prior to joining UC Irvine in 2013, he was Professor in the Division of Biostatistics, Department of Public Health Sciences, Consortium Statistics Leader for the NeuroTherapeutics Research Institute, Director of the Data Coordinating Center for the Early Autism Risk Longitudinal Investigation (EARLI) Network, and Director of the Statistics Core of the Center for Children’s Environmental Health at UC Davis. He has published over 85 research papers in clinical applications and statistical methodologies.