Counteracting the overestimation of effect size in mass-univariate brain-wide association studies using cross-validation

Transl Psychiatry. 2026 Nov 4;16(1):612. doi: 10.1038/s41398-026-04529-x.

ABSTRACT

BACKGROUND: Brain-wide association effects are commonly overestimated in psychiatric neuroimaging research, leading to low replicability and generalizability of findings, and ultimately to poor translation to clinical utility. Cross-validation is currently not standardly used in univariate neuroimaging analyses but could pose a potential counteracting measure for this problem.

METHODS: We systematically evaluated the benefit of a cross-validation procedure to expose inflated effect size estimates in mass-univariate voxel-based analysis, using body mass index (BMI) as an example predictor for gray matter. A total of n = 3401 MRI scans from a psychiatric cohort were used to investigate the impact of applying cross-validation on effect size estimates across a range of sample sizes and across different localizations with a range of underlying pooled effect sizes.

RESULTS: BMI was negatively associated with gray matter density with a maximum underlying effect size of R²p = 0.036 (in the full sample of n = 3401). However, without cross-validation, effects were substantially overestimated in small samples, with effect sizes ranging up to R²p = 0.535. Even with an underlying null-effect, effect size estimates up to R²p = 0.429 were reached in samples with n = 25. Applying cross-validation revealed that these inflated effects in small samples did not generalize to novel data. A minimum sample size of n = 100 was required for largest brain-wide effects to start generalizing, while samples with n = 400 were needed for smaller effects to generalize. This ‚point of initial generalizability‘ was not reached for underlying null effects, even with the maximum available sample of n = 3401.

CONCLUSIONS: Results suggest that cross-validation is an effective measure to expose overestimated effects in small samples and thus could help to counteract low replicability and generalizability in brain-wide mass-univariate association studies. Open-source code is provided for practical implementation.

PMID:42982783 | PMC:PMC13737174 | DOI:10.1038/s41398-026-04529-x

Schreibe einen Kommentar

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert