Modeling Methods
Model-based analysis is more an art than a science. -Lewis B. Sheiner, MD
Important considerations
Impact of a model [1,2]
- What is the modeling used for? (e.g., bridging, dose, SmPC1 parameters?)
- Does the conclusion align with the aim?
- What data is available?
- Rich data
- Sparse data
- What is the structural model?
- Exposure-response is generally non-informative if only one dose-level is given, even if weight-adjusted
Reviewing models
- Does my conclusion align with the authors?
- Questions NGN (eNGiNe)
- Need-to-know: Will affect conclusion (Major objection)
- Good-to-know: Could affect conclusion (Other concern)
- Nice-to-know: Won’t affect conclusion (avoid asking this question)
Terminology
Parsimony: The idea is that comparing two models, the model with fewer parameters is preferable, given that all else is equal.
Shrinkage: Shrinkage quantifies how much individual estimates regress towards the population mean under the given sampling schedule [3]. It also measures the amount of information on a parameter in the current data used to estimate it.
Robustness: Robustness describes a model’s ability to perform reliably under varying assumptions or when faced with unusual or extreme data points. Robust models produce stable parameter estimates even if underlying assumptions (normality, independence, linearity, etc.) are not perfectly met. Non-robust models may yield misleading results or parameter instability if assumptions are violated.
Misspecification: Misspecification occurs when a model’s mathematical structure or assumptions differ significantly from the underlying data-generating process.
It is reasonable to assume that there will always be some model misspecification. We recall:
All models are wrong, but some are useful. -George E. P. Box, PhD
Overfitting: The model fits training data so well that it fails on new data (poor generalization).
Identifiability: Whether parameters can be uniquely estimated from the data. Lack of identifiability means multiple parameter combinations yield the same likelihood.
Bias-variance tradeoff: Balancing a model’s simplicity (high bias, low variance) versus complexity (low bias, high variance) to optimize predictive accuracy.
References
