Insights / Articles

Minimizing Variability in Imaging Reads for Clinical trials

Minimizing Variability in Imaging Reads for Clinical trials

In oncology, response to treatment is typically assessed by measuring tumor size and growth over time using medical images obtained from various imaging modalities. Medical images can provide unreliable results if standardization protocols are not implemented and carefully followed. A thorough understanding of quantitative imaging, its potential for variability and the standardization guidelines necessary to produce data that reliably detects drug treatment effects are key to running a successful imaging trial.

The Variability Challenge

Decisions on therapeutic efficacy can only be made based on imaging data that are consistent, reliable, and accurate. Controlling for data variability is one of the greatest challenges clinical researchers will face.

Variability can happen during both image capture and interpretation. Variability introduced in the image construction stage can be due to differences across clinical trial sites in the type of scanner used; the type, volume, and administration of contrast agent; the choice of acquisition parameters (e.g., slice thickness, filters, etc.) or other measurement parameters (e.g., unidimensional versus volumetric measurements), or the way patients are handled or positioned.

However, one of the greatest sources of variability stems from reader interpretation – once images are created and must be analyzed by the reader either at the imaging site or elsewhere for centralized reading – and includes inter- and intra-reader variability, reader interpretation bias, or reader fatigue.

  • Intra-reader variability: measurement differences from a same reader, usually stems from the inherent difficulty of the imaging case
  • Inter-reader variability: measurement differences among multiple readers, usually due to readers’ skills
  • Reader interpretation bias: interpretation/application of tumor assessment criteria
  • Reader fatigue: a reduced capacity to correctly interpret images after long work hours or reading many scans

Overall and not unexpectedly, inter-reader variability has been found to be higher than intra-reader variability. Multiple readers may not always select the same lesions for measurement and choice of lesion or lesion location can impact a reader’s ability to consistently interpret an image.

Inter-reader variation is a serious issue that if left unchecked can lead to misclassification of the RECIST response criteria, for example, either through inaccurate declaration of progressive disease that can lead to prematurely remove a patient from a trial or through the inability to recognize true therapeutic benefit. Technology that offers automated lesion selection measurement and tracking, like Median’s Lesion Management Solution, can overcome part of these issues, significantly reducing variability. One study found that using computer-assisted measurements reduced inter-reader variability by half as compared with manual measurements. [Dinkel 2013] Measurement software  can also decrease the time required for image assessment and creation of a study report.

Minimize Variability through Standardization

The key to reducing variability is standardization. Common standardization strategies include:

  • Defining imaging protocol for image acquisition adapted to the modality. Calibrate the scanner regularly at each clinical site
  • Properly training technologists and radiologists in image acquisition requested in imaging protocol for the trial. Training for interpretation methods and criteria application.
  • Using computer-assisted interpretation software to minimize inter- and intra-reader variability
  • Creating/checking standard tumor measurement methods and compliance with protocol tumor assessment criteria
  • Assessing images in the order in which they were obtained using the same single reader for any patient
  • Optimizing workflows to increase efficiency and prevent data loss

It is critical that all imaging-based clinical trials incorporate standardization practices that are compliant with federal guidelines in order to obtain FDA drug approval.

Post Authored by:  Catherine Klifa, PhD, MBA  Median Technologies, Scientific Liaison

Sources

Dinkel J, Khalilzadeh O, Hintze C et al. (2013) Lung Cancer 82, 76-82;  Korn RL and Crowley JJ. (2013) Clin Cancer Res. 19, 2607-2612 ;  Krajewski KM, Nishino M, Franchetti Y et al. (2014) Cancer 120, 711-721;  Muenzel D, Engels HP, Bruegel M, Kehl V, Rummeny EJ, and Metz S. (2012) Radiol Oncol. 46, 8-18 ;  Oxnard GR, Zhao B, Sima CS et al. (2011) J Clin Oncol. 29, 3114-3119;  Sullivan DC, Schwartz LH, and Zhao B. (2013) Clin Cancer Res. 19, 2621-2628;  Yoon SH, Kim KW, Goo JM, Kim DW, and Hahn S. (2016) Eur J Cancer. 53, 5-15 ;  Zhao B, Tan Y, Bell DJ et al. (2013) Eur J Radiol. 82, 959-968.