Goodness-of-fit for continuous data

Published

April 25, 2025

Under construction

The graphs are broadly separated into two categories: prediction-based (basic) and simulation-based tools.

Prediction-based evaluation

Population-based graphs

Table 1: Various evaluation graphs in nonlinear mixed effect model aand proposal for a core set of evaluation graphs
Graphs What does it assess?
DV vs. PRED Trends may suggest a modification of structural model, RUV model or IIV model.
CWRES vs (TIME, PRED) Trends may suggest a modification of structural model, RUV model, or IIV model. Trends by conditioning on covariates suggest including covariates.
CWRES vs COV Trends suggest including covariates or changing the covariate model.

Individual-based graphs

Table 2: Various evaluation graphs in nonlinear mixed effect model aand proposal for a core set of evaluation graphs
Graphs What does it assess?
Individual fits: (DV, PRED, IPRED) vs TIME Expect evenly distributed observation around the individual predicted curve, not spot-on predictions (indication of overfit). This diagnostic is not useful for sparse data.
DV vs IPRED Only evaluates strutural model and RUV, not IIV.
IWRES vs (TIME, IPRED) Evaluates RUV. A cone-shaped graph of IWRES vs IPRED suggests a change in the error model.
ETAx vs ETAy Prefer random sampling of ETAs from posterior distribution. Correlation between EBE suggests including correlation between random effects unless data are sparse.
ETA vs COV Trends between EBE and covariates suggest including covariates or changing the covariate model.

Simulation-based evaluation

(pc)VPC: Trends may suggest a modification of the structural model, the residual error model, or the parameter variability model. Trends when conditioning on covariates suggest including covariates or changing the covariate model.

Explanation of pcVPC

A Prediction-Corrected Visual Predictive Check (pcVPC) addresses limitations of conventional VPCs by correcting for predictable sources of variability, allowing clearer detection of model misspecification.

More specifically, a pcVPC divides (normalizes) the dependent variable by the population prediction for each bin. This correction removes the influence of variability due to independent variables.

This is especially important when data includes large variations in covariates (e.g., dose), or when adaptive dosing strategies are applied.