Bob de Vos
Amsterdam UMC

Postdoc
E-mail: b [dot] d [dot] devos [at] amsterdamumc [dot] nl
Phone: +31 20 566 89 78
LinkedIn; Google Scholar

In 2012 Bob finished his Master Biomedical Engineering at the University of Groningen (RUG). During his study Bob focused on clinical physics and became interested in medical image processing. After a couple of temporary assignments in the industry, Bob got the opportunity to be a PhD-candidate at the ISI. His main focus during his PhD was on finding biomarkers in CT lung cancer screening scans to predict cardiovascular risk. Currently Bob holds a position as a postdoctoral researcher. His project focuses on dimensionality reduction for disease analysis in high-dimensional data.

Bob is co-organizer of the MICCAI Challenge on Automatic Coronary Calcium Scoring.

Journal Articles
1.	J. Sander, B.D. de Vos, I. Išgum Automatic segmentation with detection of local segmentation failures in cardiac MRI Journal Article Scientific Reports, 10 (21769 ), 2020. Abstract | Links | BibTeX @article{Sander2020b, title = {Automatic segmentation with detection of local segmentation failures in cardiac MRI}, author = {J. Sander, B.D. de Vos, I. Išgum}, url = {https://www.nature.com/articles/s41598-020-77733-4}, year = {2020}, date = {2020-12-10}, journal = {Scientific Reports}, volume = {10}, number = {21769 }, abstract = {Segmentation of cardiac anatomical structures in cardiac magnetic resonance images (CMRI) is a prerequisite for automatic diagnosis and prognosis of cardiovascular diseases. To increase robustness and performance of segmentation methods this study combines automatic segmentation and assessment of segmentation uncertainty in CMRI to detect image regions containing local segmentation failures. Three existing state-of-the-art convolutional neural networks (CNN) were trained to automatically segment cardiac anatomical structures and obtain two measures of predictive uncertainty: entropy and a measure derived by MC-dropout. Thereafter, using the uncertainties another CNN was trained to detect local segmentation failures that potentially need correction by an expert. Finally, manual correction of the detected regions was simulated in the complete set of scans of 100 patients and manually performed in a random subset of scans of 50 patients. Using publicly available CMR scans from the MICCAI 2017 ACDC challenge, the impact of CNN architecture and loss function for segmentation, and the uncertainty measure was investigated. Performance was evaluated using the Dice coefficient, 3D Hausdorff distance and clinical metrics between manual and (corrected) automatic segmentation. The experiments reveal that combining automatic segmentation with manual correction of detected segmentation failures results in improved segmentation and to 10-fold reduction of expert time compared to manual expert segmentation.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Segmentation of cardiac anatomical structures in cardiac magnetic resonance images (CMRI) is a prerequisite for automatic diagnosis and prognosis of cardiovascular diseases. To increase robustness and performance of segmentation methods this study combines automatic segmentation and assessment of segmentation uncertainty in CMRI to detect image regions containing local segmentation failures. Three existing state-of-the-art convolutional neural networks (CNN) were trained to automatically segment cardiac anatomical structures and obtain two measures of predictive uncertainty: entropy and a measure derived by MC-dropout. Thereafter, using the uncertainties another CNN was trained to detect local segmentation failures that potentially need correction by an expert. Finally, manual correction of the detected regions was simulated in the complete set of scans of 100 patients and manually performed in a random subset of scans of 50 patients. Using publicly available CMR scans from the MICCAI 2017 ACDC challenge, the impact of CNN architecture and loss function for segmentation, and the uncertainty measure was investigated. Performance was evaluated using the Dice coefficient, 3D Hausdorff distance and clinical metrics between manual and (corrected) automatic segmentation. The experiments reveal that combining automatic segmentation with manual correction of detected segmentation failures results in improved segmentation and to 10-fold reduction of expert time compared to manual expert segmentation. Close https://www.nature.com/articles/s41598-020-77733-4 Close
2.	J.M.H. Noothout, B.D. de Vos, J.M. Wolterink, E.M. Postma, P.A.M. Smeets, R.A.P. Takx, T. Leiner, M.A. Viergever, I. Išgum Deep learning-based regression and classification for automatic landmark localization in medical images Journal Article IEEE Transactions on Medical Imaging, 39 (12), pp. 4011-4022, 2020, ISSN: 1558-254X. Abstract | Links | BibTeX @article{Noothout2020, title = {Deep learning-based regression and classification for automatic landmark localization in medical images}, author = {J.M.H. Noothout, B.D. de Vos, J.M. Wolterink, E.M. Postma, P.A.M. Smeets, R.A.P. Takx, T. Leiner, M.A. Viergever, I. Išgum}, url = {https://arxiv.org/pdf/2007.05295.pdf}, doi = {10.1109/TMI.2020.3009002}, issn = {1558-254X}, year = {2020}, date = {2020-07-09}, journal = {IEEE Transactions on Medical Imaging}, volume = {39}, number = {12}, pages = {4011-4022}, abstract = {In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vectors pointing from the center of image patches towards landmark locations are determined. In classification, presence of landmarks of interest in the patch is established. Global landmark locations are obtained by averaging the predicted displacement vectors, where the contribution of each displacement vector is weighted by the posterior classification probability of the patch that it is pointing from. Subsequently, for each landmark localized with global localization, local analysis is performed. Specialized FCNNs refine the global landmark locations by analyzing local sub-images in a similar manner, i.e. by performing regression and classification simultaneously and combining the results. Evaluation was performed through localization of 8 anatomical landmarks in CCTA scans, 2 landmarks in olfactory MR scans, and 19 landmarks in cephalometric X-rays. We demonstrate that the method performs similarly to a second observer and is able to localize landmarks in a diverse set of medical images, differing in image modality, image dimensionality, and anatomical coverage. }, keywords = {}, pubstate = {published}, tppubtype = {article} } Close In this study, we propose a fast and accurate method to automatically localize anatomical landmarks in medical images. We employ a global-to-local localization approach using fully convolutional neural networks (FCNNs). First, a global FCNN localizes multiple landmarks through the analysis of image patches, performing regression and classification simultaneously. In regression, displacement vectors pointing from the center of image patches towards landmark locations are determined. In classification, presence of landmarks of interest in the patch is established. Global landmark locations are obtained by averaging the predicted displacement vectors, where the contribution of each displacement vector is weighted by the posterior classification probability of the patch that it is pointing from. Subsequently, for each landmark localized with global localization, local analysis is performed. Specialized FCNNs refine the global landmark locations by analyzing local sub-images in a similar manner, i.e. by performing regression and classification simultaneously and combining the results. Evaluation was performed through localization of 8 anatomical landmarks in CCTA scans, 2 landmarks in olfactory MR scans, and 19 landmarks in cephalometric X-rays. We demonstrate that the method performs similarly to a second observer and is able to localize landmarks in a diverse set of medical images, differing in image modality, image dimensionality, and anatomical coverage. Close https://arxiv.org/pdf/2007.05295.pdf doi:10.1109/TMI.2020.3009002 Close
3.	G. Litjens, F. Ciompi, J.M. Wolterink, B.D. de Vos, T. Leiner, J. Teuwen, I. Išgum State-of-the-art deep learning in cardiovascular image analysis Journal Article JACC: Cardiovascular Imaging, 12 (8 Part 1), pp. 1549-1565, 2019. Abstract | Links | BibTeX @article{Litjens2019, title = {State-of-the-art deep learning in cardiovascular image analysis}, author = {G. Litjens, F. Ciompi, J.M. Wolterink, B.D. de Vos, T. Leiner, J. Teuwen, I. Išgum}, url = {https://doi.org/10.1016/j.jcmg.2019.06.009}, year = {2019}, date = {2019-08-05}, journal = {JACC: Cardiovascular Imaging}, volume = {12}, number = {8 Part 1}, pages = {1549-1565}, abstract = {Cardiovascular imaging is going to change substantially in the next decade, fueled by the deep learning revolution. For medical professionals, it is important to keep track of these developments to ensure that deep learning can have meaningful impact on clinical practice. This review aims to be a stepping stone in this process. The general concepts underlying most successful deep learning algorithms are explained, and an overview of the state-of-the-art deep learning in cardiovascular imaging is provided. This review discusses >80 papers, covering modalities ranging from cardiac magnetic resonance, computed tomography, and single-photon emission computed tomography, to intravascular optical coherence tomography and echocardiography. Many different machines learning algorithms were used throughout these papers, with the most common being convolutional neural networks. Recent algorithms such as generative adversarial models were also used. The potential implications of deep learning algorithms on clinical practice, now and in the near future, are discussed.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Cardiovascular imaging is going to change substantially in the next decade, fueled by the deep learning revolution. For medical professionals, it is important to keep track of these developments to ensure that deep learning can have meaningful impact on clinical practice. This review aims to be a stepping stone in this process. The general concepts underlying most successful deep learning algorithms are explained, and an overview of the state-of-the-art deep learning in cardiovascular imaging is provided. This review discusses >80 papers, covering modalities ranging from cardiac magnetic resonance, computed tomography, and single-photon emission computed tomography, to intravascular optical coherence tomography and echocardiography. Many different machines learning algorithms were used throughout these papers, with the most common being convolutional neural networks. Recent algorithms such as generative adversarial models were also used. The potential implications of deep learning algorithms on clinical practice, now and in the near future, are discussed. Close https://doi.org/10.1016/j.jcmg.2019.06.009 Close
4.	B.D. de Vos; F.F. Berendsen; M.A. Viergever; H. Sokooti; M. Staring; I. Išgum A deep learning framework for unsupervised affine and deformable image registration Journal Article Medical Image Analysis, 52 , pp. 128 - 143, 2019. Abstract | Links | BibTeX @article{deVos2019b, title = {A deep learning framework for unsupervised affine and deformable image registration}, author = {B.D. de Vos and F.F. Berendsen and M.A. Viergever and H. Sokooti and M. Staring and I. Išgum}, url = {http://www.sciencedirect.com/science/article/pii/S1361841518300495}, year = {2019}, date = {2019-02-21}, journal = {Medical Image Analysis}, volume = {52}, pages = {128 - 143}, abstract = {Deep learning, Unsupervised learning, Affine image registration, Deformable image registration, Cardiac cine MRI, Chest CT\", abstract = \"Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for unsupervised affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Deep learning, Unsupervised learning, Affine image registration, Deformable image registration, Cardiac cine MRI, Chest CT", abstract = "Image registration, the process of aligning two or more images, is the core technique of many (semi-)automatic medical image analysis tasks. Recent studies have shown that deep learning methods, notably convolutional neural networks (ConvNets), can be used for image registration. Thus far training of ConvNets for registration was supervised using predefined example registrations. However, obtaining example registrations is not trivial. To circumvent the need for predefined examples, and thereby to increase convenience of training ConvNets for image registration, we propose the Deep Learning Image Registration (DLIR) framework for unsupervised affine and deformable image registration. In the DLIR framework ConvNets are trained for image registration by exploiting image similarity analogous to conventional intensity-based image registration. After a ConvNet has been trained with the DLIR framework, it can be used to register pairs of unseen images in one shot. We propose flexible ConvNets designs for affine image registration and for deformable image registration. By stacking multiple of these ConvNets into a larger architecture, we are able to perform coarse-to-fine image registration. We show for registration of cardiac cine MRI and registration of chest CT that performance of the DLIR framework is comparable to conventional image registration while being several orders of magnitude faster. Close http://www.sciencedirect.com/science/article/pii/S1361841518300495 Close
5.	B.D. de Vos, J.M. Wolterink, T. Leiner, P.A. de Jong, N. Lessmann, I. Išgum Direct automatic coronary calcium scoring in cardiac and chest CT Journal Article IEEE Transactions on Medical Imaging, 34 , pp. 123-136, 2019. Abstract | Links | BibTeX @article{deVos2019, title = {Direct automatic coronary calcium scoring in cardiac and chest CT}, author = {B.D. de Vos, J.M. Wolterink, T. Leiner, P.A. de Jong, N. Lessmann, I. Išgum}, url = {https://ieeexplore.ieee.org/abstract/document/8643342 https://arxiv.org/abs/1902.05408}, year = {2019}, date = {2019-02-21}, journal = {IEEE Transactions on Medical Imaging}, volume = {34}, pages = {123-136}, abstract = {Cardiovascular disease (CVD) is the global leading cause of death. A strong risk factor for CVD events is the amount of coronary artery calcium (CAC). To meet demands of the increasing interest in quantification of CAC, i.e. coronary calcium scoring, especially as an unrequested finding for screening and research, automatic methods have been proposed. Current automatic calcium scoring methods are relatively computationally expensive and only provide scores for one type of CT. To address this, we propose a computationally efficient method that employs two ConvNets: the first performs registration to align the fields of view of input CTs and the second performs direct regression of the calcium score, thereby circumventing time-consuming intermediate CAC segmentation. Optional decision feedback provides insight in the regions that contributed to the calcium score. Experiments were performed using 903 cardiac CT and 1,687 chest CT scans. The method predicted calcium scores in less than 0.3 s. Intra-class correlation coefficient between predicted and manual calcium scores was 0.98 for both cardiac and chest CT. The method showed almost perfect agreement between automatic and manual CVD risk categorization in both datasets, with a linearly weighted Cohen's kappa of 0.95 in cardiac CT and 0.93 in chest CT. Performance is similar to that of state-of-the-art methods, but the proposed method is hundreds of times faster. By providing visual feedback, insight is given in the decision process, making it readily implementable in clinical and research settings.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Cardiovascular disease (CVD) is the global leading cause of death. A strong risk factor for CVD events is the amount of coronary artery calcium (CAC). To meet demands of the increasing interest in quantification of CAC, i.e. coronary calcium scoring, especially as an unrequested finding for screening and research, automatic methods have been proposed. Current automatic calcium scoring methods are relatively computationally expensive and only provide scores for one type of CT. To address this, we propose a computationally efficient method that employs two ConvNets: the first performs registration to align the fields of view of input CTs and the second performs direct regression of the calcium score, thereby circumventing time-consuming intermediate CAC segmentation. Optional decision feedback provides insight in the regions that contributed to the calcium score. Experiments were performed using 903 cardiac CT and 1,687 chest CT scans. The method predicted calcium scores in less than 0.3 s. Intra-class correlation coefficient between predicted and manual calcium scores was 0.98 for both cardiac and chest CT. The method showed almost perfect agreement between automatic and manual CVD risk categorization in both datasets, with a linearly weighted Cohen's kappa of 0.95 in cardiac CT and 0.93 in chest CT. Performance is similar to that of state-of-the-art methods, but the proposed method is hundreds of times faster. By providing visual feedback, insight is given in the decision process, making it readily implementable in clinical and research settings. Close https://ieeexplore.ieee.org/abstract/document/8643342 https://arxiv.org/abs/1902.05408 Close
6.	J. Šprem; B.D. de Vos; N. Lessmann; P.A. de Jong; M.A. Viergever; I. Išgum Impact of automatically detected motion artifacts on coronary calcium scoring in chest CT Journal Article Journal of Medical Imaging, 5 (4), pp. 044007 , 2018. Abstract | Links | BibTeX @article{Šprem2018, title = {Impact of automatically detected motion artifacts on coronary calcium scoring in chest CT}, author = {J. Šprem and B.D. de Vos and N. Lessmann and P.A. de Jong and M.A. Viergever and I. Išgum}, url = {https://doi.org/10.1117/1.JMI.5.4.044007}, year = {2018}, date = {2018-12-11}, journal = {Journal of Medical Imaging}, volume = {5}, number = {4}, pages = {044007 }, abstract = {The amount of coronary artery calcification (CAC) quantified in CT scans enables prediction of cardiovascular disease (CVD) risk. However, interscan variability of CAC quantification is high, especially in scans made without ECG synchronization. We propose a method for automatic detection of CACs that are severely affected by cardiac motion. Subsequently, we evaluate the impact of such CACs on CAC quantification and CVD risk determination. This study includes 1000 baseline and 585 one year follow-up low-dose chest CTs from the National Lung Screening Trial. 415 baseline scans are used to train and evaluate a convolutional neural network that identifies observer determined CACs affected by severe motion artifacts. Thereafter, 585 paired scans acquired at baseline and follow-up were used to evaluate the impact of severe motion artifacts on CAC quantification and risk categorization. Based on the CAC amount the scans were categorized into four standard CVD risk categories. The method identified CACs affected by severe motion artifacts with 85.2% accuracy. Moreover, reproducibility of CAC scores in scan pairs is higher in scans containing mostly CACs not affected by severe cardiac motion. Hence, the proposed method enables identification of scans affected by severe cardiac motion where CAC quantification may not be reproducible.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close The amount of coronary artery calcification (CAC) quantified in CT scans enables prediction of cardiovascular disease (CVD) risk. However, interscan variability of CAC quantification is high, especially in scans made without ECG synchronization. We propose a method for automatic detection of CACs that are severely affected by cardiac motion. Subsequently, we evaluate the impact of such CACs on CAC quantification and CVD risk determination. This study includes 1000 baseline and 585 one year follow-up low-dose chest CTs from the National Lung Screening Trial. 415 baseline scans are used to train and evaluate a convolutional neural network that identifies observer determined CACs affected by severe motion artifacts. Thereafter, 585 paired scans acquired at baseline and follow-up were used to evaluate the impact of severe motion artifacts on CAC quantification and risk categorization. Based on the CAC amount the scans were categorized into four standard CVD risk categories. The method identified CACs affected by severe motion artifacts with 85.2% accuracy. Moreover, reproducibility of CAC scores in scan pairs is higher in scans containing mostly CACs not affected by severe cardiac motion. Hence, the proposed method enables identification of scans affected by severe cardiac motion where CAC quantification may not be reproducible. Close https://doi.org/10.1117/1.JMI.5.4.044007 Close
7.	N. Lessmann, B. van Ginneken, M. Zreik, P.A. de Jong, B.D. de Vos, M.A. Viergever, I. Išgum Automatic calcium scoring in low-dose chest CT using deep neural networks with dilated convolutions Journal Article IEEE Transactions on Medical Imaging, 37 (2), pp. 615-625, 2018. Abstract | Links | BibTeX @article{Lessmann2017, title = {Automatic calcium scoring in low-dose chest CT using deep neural networks with dilated convolutions}, author = {N. Lessmann, B. van Ginneken, M. Zreik, P.A. de Jong, B.D. de Vos, M.A. Viergever, I. Išgum}, url = {https://arxiv.org/pdf/1711.00349.pdf}, year = {2018}, date = {2018-02-01}, journal = {IEEE Transactions on Medical Imaging}, volume = {37}, number = {2}, pages = {615-625}, abstract = {Heavy smokers undergoing screening with low-dose chest CT are affected by cardiovascular disease as much as by lung cancer. Low-dose chest CT scans acquired in screening enable quantification of atherosclerotic calcifications and thus enable identification of subjects at increased cardiovascular risk. This paper presents a method for automatic detection of coronary artery, thoracic aorta and cardiac valve calcifications in low-dose chest CT using two consecutive convolutional neural networks. The first network identifies and labels potential calcifications according to their anatomical location and the second network identifies true calcifications among the detected candidates. This method was trained and evaluated on a set of 1744 CT scans from the National Lung Screening Trial. To determine whether any reconstruction or only images reconstructed with soft tissue filters can be used for calcification detection, we evaluated the method on soft and medium/sharp filter reconstructions separately. On soft filter reconstructions, the method achieved F1 scores of 0.89, 0.89, 0.67, and 0.55 for coronary artery, thoracic aorta, aortic valve and mitral valve calcifications, respectively. On sharp filter reconstructions, the F1 scores were 0.84, 0.81, 0.64, and 0.66, respectively. Linearly weighted kappa coefficients for risk category assignment based on per subject coronary artery calcium were 0.91 and 0.90 for soft and sharp filter reconstructions, respectively. These results demonstrate that the presented method enables reliable automatic cardiovascular risk assessment in all low-dose chest CT scans acquired for lung cancer screening.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Heavy smokers undergoing screening with low-dose chest CT are affected by cardiovascular disease as much as by lung cancer. Low-dose chest CT scans acquired in screening enable quantification of atherosclerotic calcifications and thus enable identification of subjects at increased cardiovascular risk. This paper presents a method for automatic detection of coronary artery, thoracic aorta and cardiac valve calcifications in low-dose chest CT using two consecutive convolutional neural networks. The first network identifies and labels potential calcifications according to their anatomical location and the second network identifies true calcifications among the detected candidates. This method was trained and evaluated on a set of 1744 CT scans from the National Lung Screening Trial. To determine whether any reconstruction or only images reconstructed with soft tissue filters can be used for calcification detection, we evaluated the method on soft and medium/sharp filter reconstructions separately. On soft filter reconstructions, the method achieved F1 scores of 0.89, 0.89, 0.67, and 0.55 for coronary artery, thoracic aorta, aortic valve and mitral valve calcifications, respectively. On sharp filter reconstructions, the F1 scores were 0.84, 0.81, 0.64, and 0.66, respectively. Linearly weighted kappa coefficients for risk category assignment based on per subject coronary artery calcium were 0.91 and 0.90 for soft and sharp filter reconstructions, respectively. These results demonstrate that the presented method enables reliable automatic cardiovascular risk assessment in all low-dose chest CT scans acquired for lung cancer screening. Close https://arxiv.org/pdf/1711.00349.pdf Close
8.	I. Isgum, B.D. de Vos, J.M. Wolterink, D. Dey, D.S. Berman, M. Rubeaux, T. Leiner, P. J. Slomka Erratum to: Automatic determination of cardiovascular risk by CT attenuation correction maps in Rb-82 PET/CT Journal Article Journal of Nuclear Cardiology, 25 (6), pp. 2143, 2017. Abstract | Links | BibTeX @article{Isgum2017b, title = {Erratum to: Automatic determination of cardiovascular risk by CT attenuation correction maps in Rb-82 PET/CT}, author = {I. Isgum, B.D. de Vos, J.M. Wolterink, D. Dey, D.S. Berman, M. Rubeaux, T. Leiner, P. J. Slomka}, url = {http://dx.doi.org/10.1007/s12350-017-0946-4}, year = {2017}, date = {2017-06-06}, journal = {Journal of Nuclear Cardiology}, volume = {25}, number = {6}, pages = {2143}, abstract = {Regrettably an error was introduced in Table 3 during the article’s production. The very first cell (row: Very low 0; column: Very low) should read ‘12’ and not ‘21’ as originally published.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Regrettably an error was introduced in Table 3 during the article’s production. The very first cell (row: Very low 0; column: Very low) should read ‘12’ and not ‘21’ as originally published. Close http://dx.doi.org/10.1007/s12350-017-0946-4 Close
9.	I. Isgum, B.D. de Vos, J.M. Wolterink, D. Dey, D.S. Berman, M. Rubeaux, T. Leiner, P.J. Slomka Automatic determination of cardiovascular risk by CT attenuation correction maps in Rb-82 PET/CT Journal Article Journal of Nuclear Cardiology, 25 (6), pp. 2133-2142, 2017. Abstract | Links | BibTeX @article{Isgum2017, title = {Automatic determination of cardiovascular risk by CT attenuation correction maps in Rb-82 PET/CT}, author = {I. Isgum, B.D. de Vos, J.M. Wolterink, D. Dey, D.S. Berman, M. Rubeaux, T. Leiner, P.J. Slomka}, url = {http://dx.doi.org/10.1007/s12350-017-0866-3}, year = {2017}, date = {2017-03-15}, journal = {Journal of Nuclear Cardiology}, volume = {25}, number = {6}, pages = {2133-2142}, abstract = {BACKGROUND: We investigated fully automatic coronary artery calcium (CAC) scoring and cardiovascular disease (CVD) risk categorization from CT attenuation correction (CTAC) acquired at rest and stress during cardiac PET/CT and compared it with manual annotations in CTAC and with dedicated calcium scoring CT (CSCT). METHODS AND RESULTS: We included 133 consecutive patients undergoing myocardial perfusion 82Rb PET/CT with the acquisition of low-dose CTAC at rest and stress. Additionally, a dedicated CSCT was performed for all patients. Manual CAC annotations in CTAC and CSCT provided the reference standard. In CTAC, CAC was scored automatically using a previously developed machine learning algorithm. Patients were assigned to a CVD risk category based on their Agatston score (0, 1-10, 11-100, 101-400, >400). Agreement in CVD risk categorization between manual and automatic scoring in CTAC at rest and stress resulted in Cohen\'s linearly weighted κ of 0.85 and 0.89, respectively. The agreement between CSCT and CTAC at rest resulted in κ of 0.82 and 0.74, using manual and automatic scoring, respectively. For CTAC at stress, these were 0.79 and 0.70, respectively. CONCLUSION: Automatic CAC scoring from CTAC PET/CT may allow routine CVD risk assessment from the CTAC component of PET/CT without any additional radiation dose or scan time.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close BACKGROUND: We investigated fully automatic coronary artery calcium (CAC) scoring and cardiovascular disease (CVD) risk categorization from CT attenuation correction (CTAC) acquired at rest and stress during cardiac PET/CT and compared it with manual annotations in CTAC and with dedicated calcium scoring CT (CSCT). METHODS AND RESULTS: We included 133 consecutive patients undergoing myocardial perfusion 82Rb PET/CT with the acquisition of low-dose CTAC at rest and stress. Additionally, a dedicated CSCT was performed for all patients. Manual CAC annotations in CTAC and CSCT provided the reference standard. In CTAC, CAC was scored automatically using a previously developed machine learning algorithm. Patients were assigned to a CVD risk category based on their Agatston score (0, 1-10, 11-100, 101-400, >400). Agreement in CVD risk categorization between manual and automatic scoring in CTAC at rest and stress resulted in Cohen's linearly weighted κ of 0.85 and 0.89, respectively. The agreement between CSCT and CTAC at rest resulted in κ of 0.82 and 0.74, using manual and automatic scoring, respectively. For CTAC at stress, these were 0.79 and 0.70, respectively. CONCLUSION: Automatic CAC scoring from CTAC PET/CT may allow routine CVD risk assessment from the CTAC component of PET/CT without any additional radiation dose or scan time. Close http://dx.doi.org/10.1007/s12350-017-0866-3 Close
10.	B.D. de Vos, J.M. Wolterink, P.A. de Jong, T. Leiner, M.A. Viergever, I. Isgum ConvNet-based localization of anatomical structures in 3D medical images Journal Article IEEE Transactions on Medical Imaging, 36 (7), pp. 1470-1481, 2017. Abstract | BibTeX @article{devos2017, title = {ConvNet-based localization of anatomical structures in 3D medical images}, author = {B.D. de Vos, J.M. Wolterink, P.A. de Jong, T. Leiner, M.A. Viergever, I. Isgum}, year = {2017}, date = {2017-02-17}, journal = {IEEE Transactions on Medical Imaging}, volume = {36}, number = {7}, pages = {1470-1481}, abstract = {Localization of anatomical structures is a prerequisite for many tasks in medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3D medical images through detection of their presence in 2D image slices using a convolutional neural network (ConvNet). A single ConvNet is trained to detect presence of the anatomical structure of interest in axial, coronal, and sagittal slices extracted from a 3D image. To allow the ConvNet to analyze slices of different sizes, spatial pyramid pooling is applied. After detection, 3D bounding boxes are created by combining the output of the ConvNet in all slices. In the experiments 200 chest CT, 100 cardiac CT angiography (CTA), and 100 abdomen CT scans were used. The heart, ascending aorta, aortic arch, and descending aorta were localized in chest CT scans, the left cardiac ventricle in cardiac CTA scans, and the liver in abdomen CT scans. Localization was evaluated using the distances between automatically and manually defined reference bounding box centroids and walls. The best results were achieved in localization of structures with clearly defined boundaries (e.g. aortic arch) and the worst when the structure boundary was not clearly visible (e.g. liver). The method was more robust and accurate in localization multiple structures.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Localization of anatomical structures is a prerequisite for many tasks in medical image analysis. We propose a method for automatic localization of one or more anatomical structures in 3D medical images through detection of their presence in 2D image slices using a convolutional neural network (ConvNet). A single ConvNet is trained to detect presence of the anatomical structure of interest in axial, coronal, and sagittal slices extracted from a 3D image. To allow the ConvNet to analyze slices of different sizes, spatial pyramid pooling is applied. After detection, 3D bounding boxes are created by combining the output of the ConvNet in all slices. In the experiments 200 chest CT, 100 cardiac CT angiography (CTA), and 100 abdomen CT scans were used. The heart, ascending aorta, aortic arch, and descending aorta were localized in chest CT scans, the left cardiac ventricle in cardiac CTA scans, and the liver in abdomen CT scans. Localization was evaluated using the distances between automatically and manually defined reference bounding box centroids and walls. The best results were achieved in localization of structures with clearly defined boundaries (e.g. aortic arch) and the worst when the structure boundary was not clearly visible (e.g. liver). The method was more robust and accurate in localization multiple structures. Close
11.	S.A.M. Gernaat, I. Isgum, B.D. de Vos, R.A.P. Takx, D.A. Young Afat, N. Rijnberg, D.E. Grobbee, Y. van der Graaf, P.A. de Jong, T. Leiner, H.J.G.D. van den Bongard, J.P. Pignol, H.M. Verkooijen Automatic coronary artery calcium scoring on radiotherapy planning CT scans of breast cancer patients: reproducibility and association with traditional cardiovascular risk factors Journal Article Plos One, 11 (12), pp. e0167925, 2016. Abstract | BibTeX @article{Gern16, title = {Automatic coronary artery calcium scoring on radiotherapy planning CT scans of breast cancer patients: reproducibility and association with traditional cardiovascular risk factors}, author = {S.A.M. Gernaat, I. Isgum, B.D. de Vos, R.A.P. Takx, D.A. Young Afat, N. Rijnberg, D.E. Grobbee, Y. van der Graaf, P.A. de Jong, T. Leiner, H.J.G.D. van den Bongard, J.P. Pignol, H.M. Verkooijen}, year = {2016}, date = {2016-12-09}, journal = {Plos One}, volume = {11}, number = {12}, pages = {e0167925}, abstract = {OBJECTIVES: Coronary artery calcium (CAC) is a strong and independent predictor of cardiovascular disease (CVD) risk. This study assesses reproducibility of automatic CAC scoring on radiotherapy planning computed tomography (CT) scans of breast cancer patients, and examines its association with traditional cardiovascular risk factors. METHODS: This study included 561 breast cancer patients undergoing radiotherapy between 2013 and 2015. CAC was automatically scored with an algorithm using supervised pattern recognition, expressed as Agatston scores and categorized into five categories (0, 1-10, 11-100, 101-400, >400). Reproducibility between automatic and manual expert scoring was assessed in 79 patients with automatically determined CAC above zero and 84 randomly selected patients without automatically determined CAC. Interscan reproducibility of automatic scoring was assessed in 294 patients having received two scans (82% on the same day). Association between CAC and CVD risk factors was assessed in 36 patients with CAC scores >100, 72 randomly selected patients with scores 1-100, and 72 randomly selected patients without CAC. Reliability was assessed with linearly weighted kappa and agreement with proportional agreement. RESULTS: 134 out of 561 (24%) patients had a CAC score above zero. Reliability of CVD risk categorization between automatic and manual scoring was 0.80 (95% Confidence Interval (CI): 0.74-0.87), and slightly higher for scans with breath-hold. Agreement was 0.79 (95% CI: 0.72-0.85). Interscan reliability was 0.61 (95% CI: 0.50-0.72) with an agreement of 0.84 (95% CI: 0.80-0.89). Ten out of 36 (27.8%) patients with CAC scores above 100 did not have other cardiovascular risk factors. CONCLUSIONS: Automatic CAC scoring on radiotherapy planning CT scans is a reliable method to assess CVD risk based on Agatston scores. One in four breast cancer patients planned for radiotherapy have elevated CAC score. One in three patients with high CAC scores don't have other CVD risk factors and wouldn't have been identified as high risk.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close OBJECTIVES: Coronary artery calcium (CAC) is a strong and independent predictor of cardiovascular disease (CVD) risk. This study assesses reproducibility of automatic CAC scoring on radiotherapy planning computed tomography (CT) scans of breast cancer patients, and examines its association with traditional cardiovascular risk factors. METHODS: This study included 561 breast cancer patients undergoing radiotherapy between 2013 and 2015. CAC was automatically scored with an algorithm using supervised pattern recognition, expressed as Agatston scores and categorized into five categories (0, 1-10, 11-100, 101-400, >400). Reproducibility between automatic and manual expert scoring was assessed in 79 patients with automatically determined CAC above zero and 84 randomly selected patients without automatically determined CAC. Interscan reproducibility of automatic scoring was assessed in 294 patients having received two scans (82% on the same day). Association between CAC and CVD risk factors was assessed in 36 patients with CAC scores >100, 72 randomly selected patients with scores 1-100, and 72 randomly selected patients without CAC. Reliability was assessed with linearly weighted kappa and agreement with proportional agreement. RESULTS: 134 out of 561 (24%) patients had a CAC score above zero. Reliability of CVD risk categorization between automatic and manual scoring was 0.80 (95% Confidence Interval (CI): 0.74-0.87), and slightly higher for scans with breath-hold. Agreement was 0.79 (95% CI: 0.72-0.85). Interscan reliability was 0.61 (95% CI: 0.50-0.72) with an agreement of 0.84 (95% CI: 0.80-0.89). Ten out of 36 (27.8%) patients with CAC scores above 100 did not have other cardiovascular risk factors. CONCLUSIONS: Automatic CAC scoring on radiotherapy planning CT scans is a reliable method to assess CVD risk based on Agatston scores. One in four breast cancer patients planned for radiotherapy have elevated CAC score. One in three patients with high CAC scores don't have other CVD risk factors and wouldn't have been identified as high risk. Close
12.	J.M. Wolterink, T. Leiner, B.D. de Vos, R.W. van Hamersvelt, M.A. Viergever, I. Isgum Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks Journal Article Medical Image Analysis, 34 , pp. 123-136, 2016. Abstract | BibTeX @article{Wolt16b, title = {Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks}, author = {J.M. Wolterink, T. Leiner, B.D. de Vos, R.W. van Hamersvelt, M.A. Viergever, I. Isgum}, year = {2016}, date = {2016-05-11}, journal = {Medical Image Analysis}, volume = {34}, pages = {123-136}, abstract = {The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular events. CAC is clinically quantified in cardiac calcium scoring CT (CSCT), but it has been shown that cardiac CT angiography (CCTA) may also be used for this purpose. We present a method for automatic CAC quantification in CCTA. This method uses supervised learning to directly identify and quantify CAC without a need for coronary artery extraction commonly used in existing methods. The study included cardiac CT exams of 250 patients for whom both a CCTA and a CSCT scan were available. To restrict the volume-of-interest for analysis, a bounding box around the heart is automatically determined. The bounding box detection algorithm employs a combination of three ConvNets, where each detects the heart in a different orthogonal plane (axial, sagittal, coronal). These ConvNets were trained using 50 cardiac CT exams. In the remaining 200 exams, a reference standard for CAC was defined in CSCT and CCTA. Out of these, 100 CCTA scans were used for training, and the remaining 100 for evaluation of a voxel classification method for CAC identification. The method uses ConvPairs, pairs of convolutional neural networks (ConvNets). The first ConvNet in a pair identifies voxels likely to be CAC, thereby discarding the majority of non-CAC-like voxels such as lung and fatty tissue. The identified CAC-like voxels are further classified by the second ConvNet in the pair, which distinguishes between CAC and CAC-like negatives. Given the different task of each ConvNet, they share their architecture, but not their weights. Input patches are either 2.5D or 3D. The ConvNets are purely convolutional, i.e. no pooling layers are present and fully connected layers are implemented as convolutions, thereby allowing efficient voxel classification. The performance of individual 2.5D and 3D ConvPairs with input sizes of 15 and 25 voxels, as well as the performance of ensembles of these ConvPairs, were evaluated by a comparison with reference annotations in CCTA and CSCT. In all cases, ensembles of ConvPairs outperformed their individual members. The best performing individual ConvPair detected 72% of lesions in the test set, with on average 0.85 false positive (FP) errors per scan. The best performing ensemble combined all ConvPairs and obtained a sensitivity of 71% at 0.48 FP errors per scan. For this ensemble, agreement with the reference mass score in CSCT was excellent (ICC 0.944 [0.918–0.962]). Aditionally, based on the Agatston score in CCTA, this ensemble assigned 83% of patients to the same cardiovascular risk category as reference CSCT. In conclusion, CAC can be accurately automatically identified and quantified in CCTA using the proposed pattern recognition method. This might obviate the need to acquire a dedicated CSCT scan for CAC scoring, which is regularly acquired prior to a CCTA, and thus reduce the CT radiation dose received by patients.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular events. CAC is clinically quantified in cardiac calcium scoring CT (CSCT), but it has been shown that cardiac CT angiography (CCTA) may also be used for this purpose. We present a method for automatic CAC quantification in CCTA. This method uses supervised learning to directly identify and quantify CAC without a need for coronary artery extraction commonly used in existing methods. The study included cardiac CT exams of 250 patients for whom both a CCTA and a CSCT scan were available. To restrict the volume-of-interest for analysis, a bounding box around the heart is automatically determined. The bounding box detection algorithm employs a combination of three ConvNets, where each detects the heart in a different orthogonal plane (axial, sagittal, coronal). These ConvNets were trained using 50 cardiac CT exams. In the remaining 200 exams, a reference standard for CAC was defined in CSCT and CCTA. Out of these, 100 CCTA scans were used for training, and the remaining 100 for evaluation of a voxel classification method for CAC identification. The method uses ConvPairs, pairs of convolutional neural networks (ConvNets). The first ConvNet in a pair identifies voxels likely to be CAC, thereby discarding the majority of non-CAC-like voxels such as lung and fatty tissue. The identified CAC-like voxels are further classified by the second ConvNet in the pair, which distinguishes between CAC and CAC-like negatives. Given the different task of each ConvNet, they share their architecture, but not their weights. Input patches are either 2.5D or 3D. The ConvNets are purely convolutional, i.e. no pooling layers are present and fully connected layers are implemented as convolutions, thereby allowing efficient voxel classification. The performance of individual 2.5D and 3D ConvPairs with input sizes of 15 and 25 voxels, as well as the performance of ensembles of these ConvPairs, were evaluated by a comparison with reference annotations in CCTA and CSCT. In all cases, ensembles of ConvPairs outperformed their individual members. The best performing individual ConvPair detected 72% of lesions in the test set, with on average 0.85 false positive (FP) errors per scan. The best performing ensemble combined all ConvPairs and obtained a sensitivity of 71% at 0.48 FP errors per scan. For this ensemble, agreement with the reference mass score in CSCT was excellent (ICC 0.944 [0.918–0.962]). Aditionally, based on the Agatston score in CCTA, this ensemble assigned 83% of patients to the same cardiovascular risk category as reference CSCT. In conclusion, CAC can be accurately automatically identified and quantified in CCTA using the proposed pattern recognition method. This might obviate the need to acquire a dedicated CSCT scan for CAC scoring, which is regularly acquired prior to a CCTA, and thus reduce the CT radiation dose received by patients. Close
13.	J.M. Wolterink, T. Leiner, B.D. de Vos, J-L. Coatrieux, B.M. Kelm, S. Kondo, R.A. Salgado, R. Shahzad, H. Shu, M. Snoeren, R.A.P. Takx, L.J. van Vliet, T. van Walsum, T.P. Willems, G. Yang, Y. Zheng, M.A. Viergever, I. Isgum An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework Journal Article Medical Physics, 43 (5), pp. 2361, 2016. Abstract | BibTeX @article{Wolt16, title = {An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework}, author = {J.M. Wolterink, T. Leiner, B.D. de Vos, J-L. Coatrieux, B.M. Kelm, S. Kondo, R.A. Salgado, R. Shahzad, H. Shu, M. Snoeren, R.A.P. Takx, L.J. van Vliet, T. van Walsum, T.P. Willems, G. Yang, Y. Zheng, M.A. Viergever, I. Isgum}, year = {2016}, date = {2016-03-29}, journal = {Medical Physics}, volume = {43}, number = {5}, pages = {2361}, abstract = {Purpose: The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular disease (CVD) events. In clinical practice, CAC is manually identified and automatically quantified in cardiacCT using commercially available software. This is a tedious and time-consuming process in large-scale studies. Therefore, a number of automatic methods that require no interaction and semiautomatic methods that require very limited interaction for the identification of CAC in cardiacCT have been proposed. Thus far, a comparison of their performance has been lacking. The objective of this study was to perform an independent evaluation of (semi)automatic methods for CAC scoring in cardiacCT using a publicly available standardized framework. Methods: CardiacCT exams of 72 patients distributed over four CVD risk categories were provided for (semi)automatic CAC scoring. Each exam consisted of a noncontrast-enhanced calcium scoring CT (CSCT) and a corresponding coronary CT angiography (CCTA) scan. The exams were acquired in four different hospitals using state-of-the-art equipment from four major CTscanner vendors. The data were divided into 32 training exams and 40 test exams. A reference standard for CAC in CSCT was defined by consensus of two experts following a clinical protocol. The framework organizers evaluated the performance of (semi)automatic methods on test CSCT scans, per lesion, artery, and patient. Results: Five (semi)automatic methods were evaluated. Four methods used both CSCT and CCTA to identify CAC, and one method used only CSCT. The evaluated methods correctly detected between 52% and 94% of CAC lesions with positive predictive values between 65% and 96%. Lesions in distal coronary arteries were most commonly missed and aortic calcifications close to the coronary ostia were the most common false positive errors. The majority (between 88% and 98%) of correctly identified CAC lesions were assigned to the correct artery. Linearly weighted Cohen’s kappa for patient CVD risk categorization by the evaluated methods ranged from 0.80 to 1.00. Conclusions: A publicly available standardized framework for the evaluation of (semi)automatic methods for CAC identification in cardiacCT is described. An evaluation of five (semi)automatic methods within this framework shows that automatic per patient CVD risk categorization is feasible. CAC lesions at ambiguous locations such as the coronary ostia remain challenging, but their detection had limited impact on CVD risk determination.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Close Purpose: The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular disease (CVD) events. In clinical practice, CAC is manually identified and automatically quantified in cardiacCT using commercially available software. This is a tedious and time-consuming process in large-scale studies. Therefore, a number of automatic methods that require no interaction and semiautomatic methods that require very limited interaction for the identification of CAC in cardiacCT have been proposed. Thus far, a comparison of their performance has been lacking. The objective of this study was to perform an independent evaluation of (semi)automatic methods for CAC scoring in cardiacCT using a publicly available standardized framework. Methods: CardiacCT exams of 72 patients distributed over four CVD risk categories were provided for (semi)automatic CAC scoring. Each exam consisted of a noncontrast-enhanced calcium scoring CT (CSCT) and a corresponding coronary CT angiography (CCTA) scan. The exams were acquired in four different hospitals using state-of-the-art equipment from four major CTscanner vendors. The data were divided into 32 training exams and 40 test exams. A reference standard for CAC in CSCT was defined by consensus of two experts following a clinical protocol. The framework organizers evaluated the performance of (semi)automatic methods on test CSCT scans, per lesion, artery, and patient. Results: Five (semi)automatic methods were evaluated. Four methods used both CSCT and CCTA to identify CAC, and one method used only CSCT. The evaluated methods correctly detected between 52% and 94% of CAC lesions with positive predictive values between 65% and 96%. Lesions in distal coronary arteries were most commonly missed and aortic calcifications close to the coronary ostia were the most common false positive errors. The majority (between 88% and 98%) of correctly identified CAC lesions were assigned to the correct artery. Linearly weighted Cohen’s kappa for patient CVD risk categorization by the evaluated methods ranged from 0.80 to 1.00. Conclusions: A publicly available standardized framework for the evaluation of (semi)automatic methods for CAC identification in cardiacCT is described. An evaluation of five (semi)automatic methods within this framework shows that automatic per patient CVD risk categorization is feasible. CAC lesions at ambiguous locations such as the coronary ostia remain challenging, but their detection had limited impact on CVD risk determination. Close