Artificial Intelligence in Medical Imaging

Artificial intelligence in medical imaging is the use of machine learning algorithms — particularly deep learning — to automatically detect, segment, classify, and quantify features in medical images. AI produces objective, reproducible measurements that support clinical decision-making and serve as quantitative endpoints in clinical trials.

 

What is artificial intelligence in medical imaging?

Artificial intelligence in medical imaging is the application of machine learning algorithms — particularly deep learning — to analyse, interpret, predict, and quantify information in medical images, enabling faster, more consistent, and more reproducible measurement of anatomical structures and pathological findings than manual visual assessment alone.

AI systems are trained on large, labelled imaging datasets. They learn to recognise patterns associated with disease, anatomy, and treatment response, then apply that learning to new images — producing outputs ranging from a flagged region of interest to a precise volumetric measurement to a probability score for a specific diagnosis. A qualified clinician or central review team then reviews those outputs, either to inform patient care or to generate endpoint data for a clinical trial.

AI, machine learning, and deep learning — what is the difference?

In medical imaging literature and clinical conversations, AI, machine learning, and deep learning are often used interchangeably. They describe related but distinct concepts.

Artificial intelligence is the broad category — any software system that performs tasks normally requiring human intelligence, including pattern recognition, classification, and decision support.

Machine learning is a subset of AI in which systems learn patterns from data rather than following explicit programming rules. A lesion detection system is not given a rulebook describing what lesions look like — it learns from thousands of labelled examples.

Deep learning is a subset of machine learning using multi-layered neural networks. It is deep learning — specifically, convolutional neural networks for image data — that powers most modern AI medical image analysis. When a clinician, trial manager, or regulator refers to "AI in medical imaging," they are often describing a deep learning-based system, though some valid imaging AI applications continue to use other machine learning approaches.

The practical implication: deep learning systems require large, high-quality training datasets and rigorous analytical validation. For clinical use, they require regulatory clearance. The quality of the training data and the rigour of the validation are the primary determinants of whether an AI imaging tool is suitable for regulated trial use.

Four core capabilities of AI in medical imaging

All AI medical imaging tools perform one or more of these four functions. Understanding which capability a given tool provides determines how it can be used in a clinical or trial setting.

Detection
Identifies the presence of a finding in an image — a lesion, nodule, or anatomical anomaly. Detection answers: is there something here? It flags regions of interest for clinician review, but does not measure or classify what it has found. Detection is the basis of CADe (computer-aided detection) systems used in radiology screening.

Segmentation
Delineates the precise boundary of a structure or pathological region at the voxel level, enabling volumetric measurement. Segmentation is computationally more demanding than detection and is the foundation of most quantitative imaging biomarker tools used in clinical trials.

Classification
Categorises a finding by type, severity, or biological characteristic — distinguishing enhancing from non-enhancing MS lesions, classifying an amyloid PET scan as positive or negative, or assigning a severity score to white matter hyperintensity burden. Classification outputs can serve directly as trial endpoints when analytically validated.

Quantification
Produces a numerical measurement from an image — a volume in cubic millimetres, a signal intensity ratio, or a percentage change from baseline. Quantification is the capability most relevant to clinical trials: it replaces subjective categorical assessments with objective, reproducible numbers that can be compared across patients, sites, and time points.

Prediction
Uses imaging features — extracted at a single time point or across longitudinal visits — to forecast disease progression, treatment response, or clinical outcomes. Prediction goes beyond characterising what is present in an image today; it models what is likely to happen next. In clinical trials, AI-derived prognostic scores from baseline imaging are used to enrich trial populations for patients most likely to progress (critical in rare disease programmes where patient numbers are limited), to stratify randomisation, and to support the qualification of imaging measurements as surrogate endpoints that predict clinical benefit. Radiomics-based approaches — which extract large numbers of quantitative features from image texture, shape, and intensity — are commonly used for prognostic modelling in oncology and increasingly in neurology.

AI vs. traditional radiological assessment

AI medical imaging systems can outperform human readers in specific, measurable ways — and fall short in others.

Where AI can outperform human readers

Measurement consistency
For specific, well-defined quantitative tasks, AI algorithms can achieve greater consistency than human readers. An AI algorithm applies identical criteria to every image, regardless of reader fatigue, case volume, or experience level. In manual MS lesion counting, inter-reader variability in lesion number and volume has been documented at 20–30% between experienced neuroradiologists reading the same scan. ¹ For these measurement tasks, AI eliminates this variability.

Speed and throughput
Volumetric brain segmentation that requires 30–45 minutes of manual processing takes seconds with a validated AI algorithm. At the scale of a multi-site clinical trial — hundreds of patients, multiple imaging visits — this throughput difference is the determining factor in whether endpoint data is available on trial timelines.

Scalability
AI systems can process thousands of images simultaneously. Human reader capacity is fixed by the number of available specialists and their working hours. For large global trials, AI-assisted processing is the only practical approach for high-throughput biomarker quantification.

Where human expertise remains essential

Distribution shift
A deep learning model trained on 3T brain MRI from a specific scanner model performs less reliably on 1.5T data from a different manufacturer. Experienced radiologists adapt to unfamiliar data in ways current AI systems do not. This makes multi-site deployment challenging without harmonization and validation.

Clinical reasoning
Integrating imaging findings with patient history, symptoms, laboratory results, and prior imaging remains the domain of the expert clinician. AI systems provide inputs to clinical reasoning — they do not replicate it.

In clinical trials, the most relevant comparison is not AI versus radiologist but AI-assisted central review versus unassisted site reads. Centralised AI quantification removes site-level variability while central readers retain clinical oversight.

How AI medical imaging is regulated

AI medical imaging software intended for clinical use is regulated as a Software as a Medical Device (SaMD) by the FDA in the United States and under the Medical Device Regulation (MDR) in the European Union.

Clinical-use AI imaging tools may require FDA clearance through an appropriate premarket pathway before deployment in care settings. The most common pathway is 510(k) — a premarket notification demonstrating substantial equivalence to a legally marketed predicate — though tools with novel indications or higher risk profiles follow the more demanding De Novo or PMA pathways. The applicable pathway depends on the device's risk classification and intended use.

The FDA's Digital Health Center of Excellence provides guidance specific to AI/ML-based SaMD, including the predetermined change control plan (PCCP) framework — a regulatory mechanism that allows cleared AI tools to be updated under pre-specified protocols without a new submission for each software version.²

QMENTA's Care Platform received 510(k) clearance from the FDA in November 2021, clearance number K202718, authorising its use by qualified healthcare providers to manage imaging data and apply AI-powered analysis tools for neurological evaluation.³ This makes QMENTA one of a small number of cloud-based AI imaging platforms with FDA clearance for clinical neuroimaging use in the United States.

For clinical trials, FDA clearance is not required. The operative standard is analytical validation, not clearance status — AI-derived measurements can be used as primary or secondary efficacy endpoints in a regulatory submission without the software holding a 510(k) clearance, provided it is analytically validated for the specific intended use. Clearance is useful supporting evidence that baseline regulatory standards have been met, but it does not substitute for endpoint-specific validation and does not automatically qualify a tool for trial endpoint use. If an AI-derived measurement is being proposed as a formally recognised novel biomarker, sponsors may pursue qualification through the FDA's Drug Development Tool (DDT) programme — a separate track from device clearance. Sponsors should align their validation strategy with the relevant regulatory agency — FDA, EMA, or both — during protocol development and no later than end-of-phase-2 meetings.

AI medical imaging in clinical trials

Using AI in a clinical trial is operationally and regulatorily different from using it in clinical care.

In clinical care, an AI tool supports the radiologist at the point of care — one input among many that informs a decision about an individual patient.

In clinical trials, AI outputs become efficacy endpoints — the quantitative basis for a regulatory submission. A brain atrophy measurement or lesion volume change number is placed before a regulator as evidence that a drug works. This means the generating system must meet a higher standard:

• Analytical validation across scanner types, sites, and time points
• Version control ensuring every patient's images are processed identically
• Complete audit trail documenting every processing event
• Multi-site consistency across the full trial duration

The version control problem is the most practical concern. In a 50-site global trial, site-installed AI tools create the risk of version drift — different sites running different software versions, producing measurements that are not directly comparable. A cloud-based platform eliminates this: every site's images are processed by the same algorithm, the same version, in the same computational environment.

A related challenge is algorithmic provenance. Sponsors frequently want to use cutting-edge algorithms developed by academic research groups — tools that carry scientific credibility and publication history but exist outside any regulatory framework. Running those algorithms directly in a trial is not feasible: they lack version control, audit trails, validation documentation, and the quality management infrastructure a regulatory submission requires. QMENTA solves this by onboarding academic and third-party algorithms onto its GxP-validated, 510(k)-cleared platform, wrapping them in the regulatory infrastructure a trial demands while preserving the scientific integrity of the original tool. The algorithm a sponsor's scientific advisory board trusts can be the same one that generates the endpoint data — deployed under conditions that will withstand regulatory scrutiny.

QMENTA's Imaging Hub deploys over 50 AI imaging biomarker algorithms centrally, with algorithm versions locked at trial initiation. The platform contributed to a consequential example of AI-powered clinical trial imaging: a multi-centre MS study that connected ten leading academic institutions and contributed imaging data to research supporting proposed revisions to MS diagnostic criteria — work that has informed ongoing discussions about how multiple sclerosis is diagnosed in clinical practice.⁴

 

Using AI imaging outputs as trial endpoints?
QMENTA's imaging scientists can advise on algorithm selection, analytical validation requirements, and regulatory documentation for your protocol.
Talk to an expert →

 

AI medical imaging by therapeutic area

Neurology

The most mature area for AI medical imaging. Key applications in current clinical trials include:

• White matter lesion volume measurement and new lesion detection in multiple sclerosis
• Brain atrophy quantification using tools including SIENAX and FreeSurfer in Alzheimer's disease and Parkinson's disease
• Amyloid and tau PET burden classification in Alzheimer's disease trials
• DAT-SPECT binding ratio quantification in Parkinson's disease
• Automated ARIA detection and monitoring in patients receiving anti-amyloid therapies
• Central Vein Sign characterisation — an MRI biomarker evaluated for incorporation into updated MS diagnostic criteria; research supports its role in improving diagnostic specificity, with validation ongoing

Oncology

AI imaging is used extensively for tumour response assessment. Applications include:

• Lesion detection, segmentation, and volumetric measurement under RECIST 1.1 for solid tumour trials
• Brain tumour segmentation and volumetric assessment under RANO criteria
• Automated lesion tracking across imaging visits to standardise progression assessment

Cardiology

A rapidly growing application area, with AI tools for cardiac MRI analysis, including ejection fraction measurement, myocardial tissue characterisation, and late gadolinium enhancement quantification, supporting cardiovascular safety and efficacy monitoring in trials.

 

Explore QMENTA's algorithm catalog by indication
Over 50 AI imaging biomarker tools across neurology, oncology, and cardiology — browse by disease area and imaging modality.
Browse the catalog →

 

Key takeaways

• AI in medical imaging performs four core tasks: detection, segmentation, classification, quantification, and prediction.
• Deep learning drives most modern medical image AI — when clinicians say "AI imaging," they are often describing a deep learning-based system
• For specific quantitative measurement tasks, AI can achieve greater consistency than human readers; it does not replace clinical reasoning
• Clinical-use AI imaging tools are regulated as Software as a Medical Device (SaMD) and may require FDA clearance through an appropriate premarket pathway — most commonly 510(k) — depending on risk classification and intended use
• In clinical trials, AI outputs serve as efficacy endpoints and must meet higher analytical standards than in clinical care
• QMENTA's Imaging Hub deploys over 50 AI imaging biomarker algorithms centrally across multi-site trials