Every founder building clinical AI wants a clean, round number to put on a slide. “We just need fifty thousand images.” It sounds precise. It looks great in front of investors. It is also close to meaningless.
I never have that number to give, because it depends on things a single figure cannot carry, and the teams that chase a headline count usually end up with a giant archive that still will not support a regulatory submission.
Two pieces of work actually answer the question. One is a map of the acquisition variety your model has to survive. The other is a sample size calculation tied to the exact performance claim you intend to make. Neither of them fits on a slide.
In a 2015 paper, Cho and colleagues asked more or less the question everyone asks: how much data does it take to train a medical image deep learning system to reach high accuracy?
The paper’s contribution was not a number. It was a method for working out the training set size needed to hit a target accuracy on a given task, measured empirically for that task rather than borrowed as a rule of thumb.
The method is a learning curve. Take your data, train on progressively larger subsets, and at each size measure performance on a held-out set. Plot performance against training set size and look at the shape of the line.
If it is still climbing steeply, more of the same data will keep helping you. If it has levelled off, more of the same data will not move your metric, and the money you spend collecting it is wasted. Most teams have never plotted this, which is why they cannot tell whether they are looking at a data problem or a modelling problem.
Two cautions before you trust the curve. Plotting it is not free, because you have to annotate each subset first, so the labels you are trying to size are the same labels you must pay for up front.
A curve that flattens also tells you only that more data from the same distribution is wasted. It says nothing about a scanner or a site the model has never seen, which is where deployed models actually fail. A flat learning curve is not a signal of readiness.
Where the curve levels off is not fixed. It moves with the task and the architecture. Telling apart a large, obvious finding tends to saturate early. Segmenting multiple organs, counting lesions in a rare disease, or tracking change across timepoints saturates much later. Convolutional networks usually get further on fewer studies, while transformer models tend to need more data, or heavier pretraining, before they overtake them.
There is a 2026 version of this question that a serious team cannot skip. Medical imaging foundation models, pretrained on millions of unlabelled studies, can push the curve so far left that a few hundred labelled cases are enough to fine-tune a narrow neuro task that once needed thousands.
Where a good foundation model exists for your modality, the training-data problem shrinks to a fine-tuning problem. What none of this touches is the validation set. Pretraining, foundation models, and heavier augmentation all lower how much labelled training data you need, but they change nothing about how many cases you need to prove the claim. That difference matters more than most teams expect.
Total image count is the wrong thing to fixate on first, because images are not interchangeable units. A brain MRI from a 3T Siemens scanner running one site’s protocol tells you very little about how the model will behave on a 1.5T GE acquisition, in an older population, where the disease shows up at a different rate.
What the model needs is exposure to the range of conditions it will meet once it is deployed. Acquisition is the place to start. Scanner vendor and field strength set the baseline, and protocol, sequence parameters, reconstruction, and dose settings fill in the rest.
Then come the sites and the people in them: operator habits, patient age and sex, body habitus, and comorbidities. Finally, there is the axis that gets forgotten most often: disease severity and how common your positive class actually is in the population you are targeting.
So build a coverage matrix. Put the sources of variation on the axes, including scanner vendor, field strength, protocol, patient age, and disease severity, and mark every cell for which you already hold data.
The empty cells are your real data requirement. Not the ten thousand near-identical 3T brain MRIs already sitting in your bucket. Those are done. Piling more studies into cells you have already filled buys you nothing, and a bigger number on a slide is not the same thing as a better dataset.
An analysis of 203 public medical imaging datasets published in Nature Health makes the point bluntly. Children represent under 1% of all patients in the datasets that report age, and a third of datasets report no children at all, compared with a roughly 22% share of the US population.
Only 6 of 181 datasets, or 3.3%, are pediatric-only. Between 2017 and 2024, just two of 59 medical imaging challenges were pediatric, and at MIDL in 2023 and 2024, only one of 46 studies targeted children.
That is an enormous amount of data with an entire population barely present in it. Aggregate size told you nothing about who was actually covered.
Teams tend to treat annotation as separate from the data question when it is really the same question. If two readers disagree on a third of your borderline cases and nobody settles the disagreement, the model learns that disagreement as noise. The curve flattens early and low, and no amount of extra volume rescues it.
Annotation is usually the largest cost line in an imaging program anyway, since pixel-level segmentation of a single study can occupy an expert reader for an hour. Multi-reader consensus with a documented way of resolving disagreements costs more per case, and it is almost always the better trade. Measure inter-reader agreement before you scale, and adjudicate even if it leaves you with a smaller set.
And be honest about what your labels represent. Reader consensus is usually a proxy, not the truth. Where the real endpoint is histopathology, clinical follow-up, or a second modality, write down how far your annotations sit from the outcome you actually claim to predict. A reviewer will ask that question before accepting the label as ground truth.
The training set is largely an economics decision: you collect until the learning curve stops paying you back. The validation set works differently. Its size is not really a judgement call at all. It falls out of the claim you are making and the precision you are promising around that claim.
Work backward from the label. Say you want to claim 90% sensitivity, with a two-sided 95% confidence interval no wider than plus or minus 5 percentage points. The textbook calculation, assuming performance lands right on 90% and you use a normal-approximation interval, puts that at roughly 140 confirmed positive cases.
Treat that as a floor, not a plan. Two things push a registrational set well above it.
You have to account for the possibility that observed sensitivity comes in below the target. The interval is widest near the middle of the scale, so at 80% observed sensitivity, the same precision needs closer to 250 positives.
Exact intervals, such as Clopper-Pearson rather than the normal approximation, run wider than the textbook figure, and a serious submission uses them.
Plan around both, and the number a realistic test set has to hit is more like 250 to 400 confirmed positives. This is ordinary binomial power analysis, the same mathematics behind any diagnostic accuracy study.
And it counts positive cases, not images. If your positive class shows up in 5% of the population you are targeting, then 300 positives means enrolling roughly 6,000 cases in total.
If you also want to report performance separately across, say, four scanner vendors and three age bands, every one of those subgroups needs its own adequately sized interval. Enrollment multiplies rather than divides.
There is a trap in that arithmetic. Sizing at natural prevalence for a rare positive drives enrollment through the roof, as the 6,000-case example shows. Enriching the positive arm fixes the cost, but you cannot then read a prevalence-dependent metric off an enriched set, because positive and negative predictive value both move with prevalence.
The clean way through is to enrich for the sensitivity and specificity estimates, then draw a separate, naturally sampled set to estimate positive predictive value and negative predictive value.
One caveat sits underneath all of this. The binomial calculation is the mathematics for a binary claim, such as detection or classification, where every case is a positive or a negative and you are counting hits. A lot of imaging AI does not make that kind of claim.
If you are reporting a continuous biomarker, such as a volume, an atrophy rate, or a diffusion measure, you do not size on positives at all. You size on the precision you need around agreement with the reference, the confidence interval on an intraclass correlation or the Bland-Altman limits of agreement, and the smallest change you have to detect reliably.
If you are reporting a segmentation, you size on the precision you need around Dice or volumetric error across the range of anatomy and pathology you will meet.
Different claim, different calculation, same discipline: the validation set is sized by the claim, never by an image count.
This, and not any image count, is what FDA sample size expectations for imaging AI come down to in practice. The guidance does not publish a minimum number of images. What reviewers care about is whether your evidence supports your claim in the intended-use population, with enough precision and the relevant subgroups included.
A model trained on 100,000 images but tested on only 140 positive cases is underpowered, however large the training corpus looks on a slide.
Standalone accuracy is also only half of what a reviewer expects for an assistive device. If the claim is that a clinician reads better with your tool than without it, the evidence is a multi-reader, multi-case study: a panel of readers each interpreting the same cases with and without the AI.
The power of that study is driven by the number of readers multiplied by the number of cases, not by the count of positives alone, and it is usually the study that sets your timeline and budget. Plan for both the standalone assessment and the reader study, because the assistive claim needs both.
Fixed ratios like 70/15/15 come from generic machine learning and do not hold up well against clinical evidence requirements. Each partition has its own job, so size each one according to its own driver.
Training set: Governed by the learning curve. Stop adding to it once held-out performance flattens on data from the same distribution.
Tuning set: Only has to make model selection stable, which usually does not require much data.
Internal test set: Sized according to how precise your development estimate needs to be.
Registrational set: Sized using the sample size calculation behind the regulatory claim, drawn from sites and scanners the model has never touched, and sequestered before the analysis plan is locked.
One rule matters more than the ratios: split by patient and by site, never by slice or individual image.
Splitting at the slice level lets the same patient land in both training and test. This produces numbers that look excellent in development and then collapse during external validation. Holding out whole sites is the only split that tells you what happens when the model meets an acquisition environment it has never seen.
Before you spend a dollar on collection, put your dataset through a short interrogation:
Wherever the answer is no, that is the gap to close. Close it with targeted retrospective data that fills the exact cells you are missing. Collecting more of what you already have just deepens a bias that is already sitting in the dataset.
Retrospective imaging carries identifiers in more places than people expect. Names and medical record numbers sit in DICOM headers. Dates and device identifiers hide in private tags. Facial geometry can be reconstructed from head CT and MRI, which is why defacing belongs inside de-identification rather than as an afterthought beside it.
Under HIPAA, you choose between Safe Harbor and expert determination. GDPR is stricter still, and pseudonymized imaging remains personal data as far as it is concerned.
Whether a given archive is even usable can turn on ethics approval, cross-border transfer rules, and how the original consent was worded. Do that review while selecting the cohort, not after you have paid to transfer it.
The clean number on the slide was never the target. Everything above reduces to two questions a single figure cannot answer.
The first is coverage: what range of scanners, sites, and populations does the model have to survive once it is deployed? You answer that with a matrix and its empty cells, not a total.
The second is precision: what confidence does your claim demand? You answer that by working backward from the exact performance you intend to report.
Get those two right and the rest follows. The training set sizes itself against the learning curve and stops when the curve flattens on held-out data. The validation set sizes itself against the claim, and it counts confirmed cases, not images.
The archive that looks impressive but cannot support a submission stops looking like progress. The teams that get a model cleared are rarely the ones with the most data. They are the ones that can say precisely which data they are still missing, and then go and get exactly that.
Source for the pediatric representation figures: Hua et al., “Underrepresentation of children in public medical imaging datasets,” Nature Health, 2026. https://doi.org/10.1038/s44360-026-00111-3
There is no universal minimum number of medical images needed to train an AI model. The requirement depends on the task, model architecture, label quality, disease prevalence, and diversity of scanners, sites, protocols, and patient populations. The most reliable approach is to build a learning curve and continue adding training data until performance on held-out data stops improving.
Training set size should be estimated using a learning curve. Train the model on progressively larger subsets of data and measure performance on a separate held-out set. If performance is still improving substantially, more comparable data may help. If the curve has flattened, collecting more data from the same distribution is unlikely to improve the model. Missing scanner, site, or population groups must still be addressed separately.
Training data teaches the model, while validation data provides evidence that the model performs as claimed. Training set size is largely determined by the learning curve and available budget. Validation set size must be calculated from the intended performance claim, required confidence interval, disease prevalence, and any subgroup analyses. A large training set cannot compensate for an underpowered validation study.
A medical imaging AI validation set should be sized according to the performance claim, not a fixed percentage of the total dataset. For a binary classification model, this usually means calculating how many confirmed positive and negative cases are needed to estimate sensitivity and specificity with the required precision. Continuous biomarkers and segmentation models require different calculations based on their specific performance measures.
Approximately 140 confirmed positive cases may estimate 90% sensitivity with a two-sided 95% confidence interval of plus or minus 5 percentage points under a normal approximation. That figure should be treated as a minimum. Exact confidence intervals, lower-than-expected performance, exclusions, and subgroup analyses can increase a realistic requirement to approximately 250 to 400 confirmed positive cases.
Dataset diversity determines whether a medical imaging AI model can generalise beyond its development environment. The data should represent the scanner vendors, field strengths, acquisition protocols, clinical sites, disease severities, and patient populations included in the intended use. Thousands of similar images from one scanner or site cannot compensate for missing acquisition environments or underrepresented patient groups.
Medical imaging data should be split by patient and, whenever possible, by clinical site. Images or slices from the same patient must never appear in both development and test sets. Holding out entire sites provides a stronger assessment of how the model will perform with unfamiliar scanners, protocols, and acquisition practices. Fixed ratios such as 70/15/15 should not replace evidence-based sample size calculations.
Foundation models can reduce the amount of labelled training data required for some medical imaging tasks. A pretrained model may be fine-tuned for a narrow task using hundreds rather than thousands of labelled cases. However, pretraining does not reduce the amount of independent validation evidence needed to support a clinical or regulatory claim. The validation study must still represent the intended-use population.
A regulatory validation dataset must independently support the model’s intended-use claim with adequate precision. It should include representative patients, scanners, sites, protocols, disease presentations, and relevant subgroups. The dataset should be separated from model development, supported by reliable reference labels, governed by a predefined analysis plan, and protected from patient-level or site-level leakage.