Bridging Multi-Modal Radiomics and Deep Learning Features for Precise Lesion Detection using Uncertainty-Aware Cross-Attention Fusion Networks

Authors

  • Ananya Mukherjee Department of Bioengineering, George Mason University

DOI:

https://doi.org/10.66280/cis.v1i1.134

Abstract

The integration of disparate data streams in medical imaging represents a significant frontier in precision oncology and diagnostic radiology. While radiomics provides high-dimensional, engineered features that capture textural and morphological nuances, deep learning offers automated, latent feature extraction capable of identifying complex non-linear patterns. This paper investigates a systemic framework for bridging these two methodologies through the implementation of Uncertainty-Aware Cross-Attention Fusion Networks. We propose a multi-modal architecture that utilizes cross-attention mechanisms to dynamically weigh the contributions of radiomic descriptors and deep-learned representations, supplemented by an uncertainty estimation layer that quantifies the reliability of the fusion process. Beyond the algorithmic architecture, the study provides an exhaustive system-level analysis of the structural trade-offs inherent in multi-modal fusion, the infrastructure required for large-scale clinical deployment, and the governance frameworks necessary to ensure algorithmic fairness and robustness. We further discuss the socio-technical implications of deploying uncertainty-aware systems, emphasizing how transparency in model confidence can influence clinical decision-making and policy development. By examining the intersection of engineering complexity and clinical utility, this research outlines a sustainable path for the integration of hybrid diagnostic systems into the modern healthcare ecosystem, ensuring that precision lesion detection is both technically rigorous and ethically grounded.

References

1.Aerts, H. J., et al. (2014). Decoding Tumour Phenotype by Noninvasive Imaging using a Quantitative Radiomics Approach. Nature Communications, 5(1), 1-9.

2.Arbabshirani, M. R., et al. (2018). Advanced Machine Learning in Action: Identifying Patients with Abnormal Findings on Computed Tomography of the Head. NPJ Digital Medicine, 1(1), 1-10.

3.Carion, N., et al. (2020). End-to-End Object Detection with Transformers. In European Conference on Computer Vision (pp. 213-229). Springer, Cham.

4.Chang, C., Fu, M., Chen, X., Feng, S., Zhang, M., Zhou, X., ... & Liu, Z. (2025, November). Research on PDU-Net Lung Nodule Segmentation Algorithm Based on Path Aggregation and Dual Attention. In 2025 4th International Conference on Image Processing, Computer Vision and Machine Learning (ICICML) (pp. 1897-1900). IEEE.

5.Chen, J., et al. (2021). TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv preprint arXiv:2102.04306.

6.Dercle, B., et al. (2022). Artificial Intelligence in Oncology: From Research to Clinical Practice. CA: A Cancer Journal for Clinicians, 72(5), 452-482.

7.Gal, Y., & Ghahramani, Z. (2016). Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning. International Conference on Machine Learning (ICML).

8.Gillies, R. J., Kinahan, P. E., & Hricak, H. (2016). Radiomics: Images Are More than Pictures, They Are Data. Radiology, 278(2), 563-577.

9.Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.

10.Hatamizadeh, A., et al. (2022). UNETR: Transformers for 3D Medical Image Segmentation. WACV.

11.Hosny, A., et al. (2018). Artificial Intelligence in Radiology. Nature Reviews Cancer, 18(8), 500-510.

12.Isensee, F., et al. (2021). nnU-Net: a Self-configuring Method for Deep Learning-based Biomedical Image Segmentation. Nature Methods, 18(2), 203-211.

13.Kendall, A., & Gal, Y. (2017). What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? Advances in Neural Information Processing Systems (NeurIPS).

14.Lambin, P., et al. (2017). Radiomics: the Bridge between Medical Imaging and Personalized Medicine. Nature Reviews Clinical Oncology, 14(12), 749-762.

15.LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep Learning. Nature, 521(7553), 436-444.

16.Liu, Z., et al. (2021). Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. ICCV.

17.McKinney, S. M., et al. (2020). International Evaluation of an AI System for Breast Cancer Screening. Nature, 577(7788), 89-94.

18.Müller, H., et al. (2022). Ethics and Governance of AI in Medical Imaging. Journal of the American College of Radiology.

19.Parmar, C., et al. (2015). Radiomics: Machine Learning Management of Intratumor Heterogeneity in Cancer Research. Scientific Reports, 5(1), 1-12.

20.Rajpurkar, P., et al. (2022). AI in Health and Medicine. Nature Medicine, 28(1), 31-38.

21.Ronneberger, O., et al. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI.

22.Shamshad, F., et al. (2023). Transformers in Medical Imaging: A Survey. Medical Image Analysis, 88, 102802.

23.Topol, E. J. (2019). High-performance Medicine: the Convergence of Human and Artificial Intelligence. Nature Medicine, 25(1), 44-56.

24.Vaswani, A., et al. (2017). Attention is All You Need. NeurIPS.

25.Varoquaux, G., & Cheplygina, V. (2022). Machine Learning for Medical Imaging: Methodological Failures and Recommendations for the Future. NPJ Digital Medicine.

26.Wang, G., et al. (2019). Aleatoric Uncertainty Estimation with Trainable Class-dependent Parameters for Medical Image Segmentation. MIDL.

27.Xie, Y., et al. (2021). CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation. MICCAI.

28.Yala, A., et al. (2019). A Deep Learning Mammography-based Model for Breast Cancer Risk Prediction. Radiology, 292(1), 60-66.

29.Yu, Q., et al. (2022). TransNorm: Transformer Provides a Strong Baseline for Medical Image Segmentation. arXiv preprint arXiv:2203.04780.

30.Zhang, Y., et al. (2021). Medical Image Segmentation using Leverage of Swin Transformer and U-Net. Pattern Recognition.

31.Zhou, H. Y., et al. (2021). NNFormer: Interleaved Transformer for Volumetric Medical Image Segmentation. arXiv preprint arXiv:2109.03201.

Downloads

Published

2026-05-03

How to Cite

Ananya Mukherjee. (2026). Bridging Multi-Modal Radiomics and Deep Learning Features for Precise Lesion Detection using Uncertainty-Aware Cross-Attention Fusion Networks. Computational Intelligence Systems, 4(1). https://doi.org/10.66280/cis.v1i1.134