Intraorbital foreign body detection and localisation by radiographers: a preliminary JAFROC observer performance study

Introduction - The purpose of this study was to run a preliminary investigation to establish if a short course of learning would increase radiographers’ performance in intraorbital foreign body (IOFB) detection and localisation on pre-magnetic resonance imaging (MRI) orbital computed radiographs (CR).

Method - A multi-reader multi-case (MRMC) human observer study was performed. Fifteen radiographers from 5 hospitals participated. Each radiographer reviewed a pre- and post-training image bank and was instructed to identify the presence or absence of IOFBs, indicating the lesion location on each case whilst scoring the detection using a confidence index on a 5-point scale, for 30 orbital radiographs. The results were analysed using a Jackknife free-response receiver operating characteristic (JAFROC2 equal weighted) methodology.

Results -The performance of the radiographers demonstrated a statistically significant difference after a short period of training in the detection of IOFBs on orbital radiographs (F (1,14)= 12.99, df = 14.0, p = 0.0029). The JAFROC2 analysis averaged figure of merit (FOM) for the radiographers was 0.818 (95% CI 0.769, 0.867) pre-training and 0.920 (95% CI 0.891, 0.950) post-training.

Conclusion - These results suggest that with a short programme of learning in image interpretation for IOFBs in orbital radiographs, radiographers should be able to achieve a high level of accuracy in the identification and localisation of IOFBs prior to MRI examination.