In partial fulfilment of the requirements for the degree of Master in Medical Research.

Abstract:
Background: With the coming age, integration of Artificial Intelligence is seen in almost all aspects of life, even medical field especially the field of radiology. Clinical Decision Support Systems (CDSSs) like ORADII and ORAD DDx are available to help diagnosis oral intra bony lesion. However, their diagnostic validity remains to be fully established and limitations need to be explored, especially when compared to gold standard of Diagnosis i.e. histopathological diagnosis.

Objectives: To evaluate and compare the diagnostic performance of two CDSS tools- ORADIII and ORAD DDx-against histopathological diagnosis in identifying intra-bony jaw lesions using orthopantomograms (OPGs).

Materials and Method: A cross-sectional diagnostic accuracy study was conducted on a samples comprising both lesion and non-lesion cases based on radiographic evaluation. Diagnostic outputs from ORADIII and ORAD DDx were compared with histopathology. Key performance indicators-including sensitivity, specificity, accuracy, F1 score, Positive Predictive Value (PPV), Negative Predictive Value (NPV), and likelihood ratios (LR+ and LR-)-were calculated for both systems. Concordance, partial concordance, and discordance with histopathological diagnosis were also assessed.

Results: Among the 350 samples evaluated, including 175 lesion-positive and 175 non-lesion cases, ORAD DDx demonstrated superior diagnostic performance compared to ORADIII. The sensitivity, specificity, accuracy, and F1 score for ORADIII were 64.57%, 60.00%, 62.29%, and 0.6314, respectively. In contrast, ORAD DDx achieved sensitivity, specificity, accuracy, and F1 score of 70.29%, 65.71%, 68.57%, and 0.6869 respectively. The positive predictive value (PPV) and negative predictive value (NPV) for ORADIII were 61.75% and 62.87%, while for ORAD DDx, these were 67.21% and 68.86%, respectively. The positive likelihood (LR+) and negative likelihood ratio (LR-) were 1.614 and 0.5905 for ORADIII, compared to 2.050 and 0.4513 for ORAD DDx.

Conclusion: in this study, ORADIII and ORAD DDx demonstrated moderate diagnostic performance when compared to the gold standard of histopathological diagnosis. ORAD DDx showed slightly higher sensitivity, specificity, and diagnostic accuracy than ORADIII, with greater concordance, in identifying intra-bony jaw lesions. However, both systems provide radiographic-level diagnoses and do not replace histopathological evaluation. Their utility lies in supporting clinical decision-making, while they show promise, further refinement and validation are needed before clinical integration as reliable diagnostic tools.

Keywords: artificial intelligence; clinical decision support systems; jaw diseases/diagnosis; diagnostic imaging.