Brett Klamer

Table of Contents

External Validation

External validation of a model is the process of assessing the generalizability or transportability of its predictions to an independent sample. The process of external validation may lead to rejection of a proposed model, or the proposed model may be accepted (validated) if the performance characteristics are deemed to be sufficient. Resampling methods are used for ‘internal validation’ of the model for data originating under the same setting. Though internal validation can be thought of as an estimate of external validation, it is not sufficient evidence of external validation. An independent dataset suitable for external validation generally has one or more of the following properties (Moons et al. (2012)):

A principled approach to external validation follows these steps:

  1. Collect a suitable independent sample of sufficient size.
  2. Create a descriptive summary table that compares the characteristics of the original sample vs. the external sample.
  3. Compare prediction performance estimates for the following scenarios
    1. Original apparent: The original model applied to the original data.
    2. Original internally validated: The cross-validitory based estimate from the original model applied to the original data.
    3. Original externally validated: The original model applied to the new data.
  4. Compare model parameter estimates and prediction performance estimates from
    1. The original model
    2. The original model selection algorithm applied to the new data only
    3. The original model selection algorithm applied to the combined data (original + new)
    4. Potentially an updated model selection algorithm applied to the combined data (original + new)
  5. Discuss differences for these outcomes, whether they might be due to population differences, overfitting, underfitting, differences in data capture, extrapolation, etc.

Royston and Altman (2013) provide specific suggestions for Cox models.


Moons, Karel G M, Andre Pascal Kengne, Diederick E Grobbee, Patrick Royston, Yvonne Vergouwe, Douglas G Altman, and Mark Woodward. 2012. “Risk Prediction Models: II. External Validation, Model Updating, and Impact Assessment.” Heart 98 (9): 691–98.
Royston, Patrick, and Douglas G Altman. 2013. “External Validation of a Cox Prognostic Model: Principles and Methods.” BMC Medical Research Methodology 13 (1): 33.
Published: 2022-02-20