The amount of coronary artery calcification as quantified with CT images is a strong and independent predictor of cardiovascular events.
The goal of this project is to develop a system for automatic identification, characterization and quantification of calcified plaque in coronary arteries using cardiac CT scans to enable determination of cardiovascular risk in individual patients.
Researchers
Ivana Išgum
Max Viergever
Tim Leiner
Jelmer Wolterink
Journal Articles |
|
| 1. | J.M. Wolterink, T. Leiner, B.D. de Vos, J-L. Coatrieux, B.M. Kelm, S. Kondo, R.A. Salgado, R. Shahzad, H. Shu, M. Snoeren, R.A.P. Takx, L.J. van Vliet, T. van Walsum, T.P. Willems, G. Yang, Y. Zheng, M.A. Viergever, I. Isgum An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework Journal Article Medical Physics, 43 (5), pp. 2361, 2016. @article{Wolt16, title = {An evaluation of automatic coronary artery calcium scoring methods with cardiac CT using the orCaScore framework}, author = {J.M. Wolterink, T. Leiner, B.D. de Vos, J-L. Coatrieux, B.M. Kelm, S. Kondo, R.A. Salgado, R. Shahzad, H. Shu, M. Snoeren, R.A.P. Takx, L.J. van Vliet, T. van Walsum, T.P. Willems, G. Yang, Y. Zheng, M.A. Viergever, I. Isgum}, year = {2016}, date = {2016-03-29}, journal = {Medical Physics}, volume = {43}, number = {5}, pages = {2361}, abstract = {Purpose: The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular disease (CVD) events. In clinical practice, CAC is manually identified and automatically quantified in cardiacCT using commercially available software. This is a tedious and time-consuming process in large-scale studies. Therefore, a number of automatic methods that require no interaction and semiautomatic methods that require very limited interaction for the identification of CAC in cardiacCT have been proposed. Thus far, a comparison of their performance has been lacking. The objective of this study was to perform an independent evaluation of (semi)automatic methods for CAC scoring in cardiacCT using a publicly available standardized framework. Methods: CardiacCT exams of 72 patients distributed over four CVD risk categories were provided for (semi)automatic CAC scoring. Each exam consisted of a noncontrast-enhanced calcium scoring CT (CSCT) and a corresponding coronary CT angiography (CCTA) scan. The exams were acquired in four different hospitals using state-of-the-art equipment from four major CTscanner vendors. The data were divided into 32 training exams and 40 test exams. A reference standard for CAC in CSCT was defined by consensus of two experts following a clinical protocol. The framework organizers evaluated the performance of (semi)automatic methods on test CSCT scans, per lesion, artery, and patient. Results: Five (semi)automatic methods were evaluated. Four methods used both CSCT and CCTA to identify CAC, and one method used only CSCT. The evaluated methods correctly detected between 52% and 94% of CAC lesions with positive predictive values between 65% and 96%. Lesions in distal coronary arteries were most commonly missed and aortic calcifications close to the coronary ostia were the most common false positive errors. The majority (between 88% and 98%) of correctly identified CAC lesions were assigned to the correct artery. Linearly weighted Cohen’s kappa for patient CVD risk categorization by the evaluated methods ranged from 0.80 to 1.00. Conclusions: A publicly available standardized framework for the evaluation of (semi)automatic methods for CAC identification in cardiacCT is described. An evaluation of five (semi)automatic methods within this framework shows that automatic per patient CVD risk categorization is feasible. CAC lesions at ambiguous locations such as the coronary ostia remain challenging, but their detection had limited impact on CVD risk determination.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Purpose: The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular disease (CVD) events. In clinical practice, CAC is manually identified and automatically quantified in cardiacCT using commercially available software. This is a tedious and time-consuming process in large-scale studies. Therefore, a number of automatic methods that require no interaction and semiautomatic methods that require very limited interaction for the identification of CAC in cardiacCT have been proposed. Thus far, a comparison of their performance has been lacking. The objective of this study was to perform an independent evaluation of (semi)automatic methods for CAC scoring in cardiacCT using a publicly available standardized framework. Methods: CardiacCT exams of 72 patients distributed over four CVD risk categories were provided for (semi)automatic CAC scoring. Each exam consisted of a noncontrast-enhanced calcium scoring CT (CSCT) and a corresponding coronary CT angiography (CCTA) scan. The exams were acquired in four different hospitals using state-of-the-art equipment from four major CTscanner vendors. The data were divided into 32 training exams and 40 test exams. A reference standard for CAC in CSCT was defined by consensus of two experts following a clinical protocol. The framework organizers evaluated the performance of (semi)automatic methods on test CSCT scans, per lesion, artery, and patient. Results: Five (semi)automatic methods were evaluated. Four methods used both CSCT and CCTA to identify CAC, and one method used only CSCT. The evaluated methods correctly detected between 52% and 94% of CAC lesions with positive predictive values between 65% and 96%. Lesions in distal coronary arteries were most commonly missed and aortic calcifications close to the coronary ostia were the most common false positive errors. The majority (between 88% and 98%) of correctly identified CAC lesions were assigned to the correct artery. Linearly weighted Cohen’s kappa for patient CVD risk categorization by the evaluated methods ranged from 0.80 to 1.00. Conclusions: A publicly available standardized framework for the evaluation of (semi)automatic methods for CAC identification in cardiacCT is described. An evaluation of five (semi)automatic methods within this framework shows that automatic per patient CVD risk categorization is feasible. CAC lesions at ambiguous locations such as the coronary ostia remain challenging, but their detection had limited impact on CVD risk determination. |
| 2. | J.M. Wolterink, T. Leiner, R.A.P. Takx, M.A. Viergever; I. Isgum Automatic coronary calcium scoring in non-contrast-enhanced ECG-triggered cardiac CT with ambiguity detection Journal Article IEEE Transactions on Medical Imaging, 34 (9), pp. 1867-1878, 2015. @article{Wolt15, title = {Automatic coronary calcium scoring in non-contrast-enhanced ECG-triggered cardiac CT with ambiguity detection}, author = {J.M. Wolterink, T. Leiner, R.A.P. Takx, M.A. Viergever and I. Isgum}, year = {2015}, date = {2015-03-09}, journal = {IEEE Transactions on Medical Imaging}, volume = {34}, number = {9}, pages = {1867-1878}, abstract = {The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular events. We present a system that automatically quantifies total patient and per coronary artery CAC in non-contrast-enhanced, ECG-triggered cardiac CT. The system identifies candidate calcifications that cannot be automatically labeled with high certainty and optionally presents these to an expert for review. Candidates were extracted by intensity-based thresholding and described by location features derived from estimated coronary artery positions, as well as size, shape and intensity features. Next, a two-class classifier distinguished between coronary calcifications and negatives or a multiclass classifier labeled CAC per coronary artery. Candidates that could not be labeled with high certainty were identified by entropy-based ambiguity detection and presented to an expert for review and possible relabeling. The system was evaluated with 530 test images. Using the two-class classifier, the intra-class correlation coefficient (ICC) between reference and automatically determined total patient CAC volume was 0.95. Using the multiclass classifier, the ICC between reference and automatically determined per artery CAC volume was 0.98 (LAD), 0.69 (LCX), and 0.95 (RCA). In 49% of CTs, no ambiguous candidates were identified, while review of the remaining CTs increased the ICC for total patient CAC volume to 1.00, and per artery CAC volume to 1.00 (LAD), 0.95 (LCX), and 0.99 (RCA). In conclusion, CAC can be automatically identified in noncontrast- enhanced ECG-triggered cardiac CT. Ambiguity detection with expert review may enable the application of automatic CAC scoring in the clinic with a performance comparable to that of a human expert.}, keywords = {}, pubstate = {published}, tppubtype = {article} } The amount of coronary artery calcification (CAC) is a strong and independent predictor of cardiovascular events. We present a system that automatically quantifies total patient and per coronary artery CAC in non-contrast-enhanced, ECG-triggered cardiac CT. The system identifies candidate calcifications that cannot be automatically labeled with high certainty and optionally presents these to an expert for review. Candidates were extracted by intensity-based thresholding and described by location features derived from estimated coronary artery positions, as well as size, shape and intensity features. Next, a two-class classifier distinguished between coronary calcifications and negatives or a multiclass classifier labeled CAC per coronary artery. Candidates that could not be labeled with high certainty were identified by entropy-based ambiguity detection and presented to an expert for review and possible relabeling. The system was evaluated with 530 test images. Using the two-class classifier, the intra-class correlation coefficient (ICC) between reference and automatically determined total patient CAC volume was 0.95. Using the multiclass classifier, the ICC between reference and automatically determined per artery CAC volume was 0.98 (LAD), 0.69 (LCX), and 0.95 (RCA). In 49% of CTs, no ambiguous candidates were identified, while review of the remaining CTs increased the ICC for total patient CAC volume to 1.00, and per artery CAC volume to 1.00 (LAD), 0.95 (LCX), and 0.99 (RCA). In conclusion, CAC can be automatically identified in noncontrast- enhanced ECG-triggered cardiac CT. Ambiguity detection with expert review may enable the application of automatic CAC scoring in the clinic with a performance comparable to that of a human expert. |
Inproceedings |
|
| 1. | J.M. Wolterink; T. Leiner; R.A.P. Takx; M.A. Viergever; I. Isgum An automatic machine learning system for coronary calcium scoring in clinical non-contrast enhanced, ECG-triggered cardiac CT Inproceedings In: SPIE Medical Imaging, 2014. @inproceedings{wolterink:2014-2525, title = {An automatic machine learning system for coronary calcium scoring in clinical non-contrast enhanced, ECG-triggered cardiac CT}, author = {J.M. Wolterink and T. Leiner and R.A.P. Takx and M.A. Viergever and I. Isgum}, year = {2014}, date = {2014-01-01}, booktitle = {SPIE Medical Imaging}, volume = {9035}, abstract = {Presence of coronary artery calcium (CAC) is a strong and independent predictor of cardiovascular events. We present a system using a forest of extremely randomized trees to automatically identify and quantify CAC in routinely acquired cardiac non-contrast enhanced CT. Candidate lesions the system could not label with high certainty were automatically identified and presented to an expert who could relabel them to achieve high scoring accuracy with minimal effort. The study included 200 consecutive non-contrast enhanced ECG-triggered cardiac CTs (120 kV, 55 mAs, 3 mm section thickness). Expert CAC annotations made as part of the clinical routine served as the reference standard. CAC candidates were extracted by thresholding (130 HU) and 3-D connected component analysis. They were described by shape, intensity and spatial features calculated using multi-atlas segmentation of coronary artery centerlines from ten CTA scans. CAC was identified using a randomized decision tree ensemble classifier in a ten-fold stratified cross-validation experiment and quantified in Agatston and volume scores for each patient. After classification, candidates with posterior probability indicating uncertain labeling were selected for further assessment by an expert. Images with metal implants were excluded. In the remaining 164 images, Spearman's rho between automatic and reference scores was 0.94 for both Agatston and volume scores. On average 1.8 candidate lesions per scan were subsequently presented to an expert. After correction, Spearman's rho was 0.98. We have described a system for automatic CAC scoring in cardiac CT images which is able to effectively select difficult examinations for further refinement by an expert.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Presence of coronary artery calcium (CAC) is a strong and independent predictor of cardiovascular events. We present a system using a forest of extremely randomized trees to automatically identify and quantify CAC in routinely acquired cardiac non-contrast enhanced CT. Candidate lesions the system could not label with high certainty were automatically identified and presented to an expert who could relabel them to achieve high scoring accuracy with minimal effort. The study included 200 consecutive non-contrast enhanced ECG-triggered cardiac CTs (120 kV, 55 mAs, 3 mm section thickness). Expert CAC annotations made as part of the clinical routine served as the reference standard. CAC candidates were extracted by thresholding (130 HU) and 3-D connected component analysis. They were described by shape, intensity and spatial features calculated using multi-atlas segmentation of coronary artery centerlines from ten CTA scans. CAC was identified using a randomized decision tree ensemble classifier in a ten-fold stratified cross-validation experiment and quantified in Agatston and volume scores for each patient. After classification, candidates with posterior probability indicating uncertain labeling were selected for further assessment by an expert. Images with metal implants were excluded. In the remaining 164 images, Spearman's rho between automatic and reference scores was 0.94 for both Agatston and volume scores. On average 1.8 candidate lesions per scan were subsequently presented to an expert. After correction, Spearman's rho was 0.98. We have described a system for automatic CAC scoring in cardiac CT images which is able to effectively select difficult examinations for further refinement by an expert. |
Abstracts |
|
| 1. | J.M. Wolterink; M.J. Willemink; R.A.P. Takx; M. Prokop; J. de Mey; M. Das; P.A. de Jong; R.P.J. Budde; A.M.R. Schilham; R.L.A.W. Bleys; N. Buls; J.E. Wildberger; M.A. Viergever; I. Isgum; T. Leiner Differences in coronary artery calcification scores obtained with different CT scanners are not software related Abstract In: 2014. @booklet{wolterink:2014-2618b, title = {Differences in coronary artery calcification scores obtained with different CT scanners are not software related}, author = {J.M. Wolterink and M.J. Willemink and R.A.P. Takx and M. Prokop and J. de Mey and M. Das and P.A. de Jong and R.P.J. Budde and A.M.R. Schilham and R.L.A.W. Bleys and N. Buls and J.E. Wildberger and M.A. Viergever and I. Isgum and T. Leiner}, year = {2014}, date = {2014-01-01}, booktitle = {Society of Cardiovascular Computed Tomography, 9th Annual Scientific Meeting}, organization = {Society for Cardiovascular Computed Tomography}, abstract = {Introduction: The Agatston score quantifying the amount of coronary artery calcifications provides risk discrimination and reclassification of patients at intermediate cardiovascular risk. However, state-of-the-art CT scanners from four different vendors have been shown to yield significantly different Agatston scores. The aim of this study was to investigate whether these differences are due to image acquisition and reconstruction or to differences in the vendor-specific calcium scoring software. Methods: CT images of fifteen ex-vivo human hearts were previously acquired on state-of-the-art CT scanners of four different vendors. All images were made with unenhanced ECG-triggered step-and-shoot protocols at equal radiation dose settings and reconstructed to 3 mm slice thickness and increment. In addition to previous work, where an Agatston score was obtained for each image using semi-automatic calcium scoring software from the scanner vendor, custom semi-automatic calcium scoring software was used to score calcium in all images. Differences in scores of all images of the same heart were analyzed using the Friedman test (significance level P < 0.05). Differences in scores obtained on the same image using the vendor-specific and the custom software were analyzed with the Wilcoxon signed-rank test (significance level P < 0.05). Results: Significant inter-scan differences (P < 0.05) in Agatston scores obtained using vendor software were previously reported for fourteen hearts with coronary calcifications. Corresponding median (interquartile range) Agatston scores were 332 (114-1135), 353 (172-1246), 410 (177-1454), and 469 (183-1381), respectively. Significant inter-scan differences persisted (P < 0.05) when images were scored in the custom software. Median (interquartile range) Agatston scores were 327 (115-1135), 353 (172-1246), 410 (180-1457), and 469 (183-1369), respectively. No significant intra-scan differences were found between scores obtained on the vendor software and on the custom software. Conclusions: Cardiac CT images obtained with scanners from four different vendors result in significantly different Agatston scores, also when all evaluated using the same software. There are no significant differences between scores obtained on vendor and custom software. Therefore, differences in calcium scores obtained using CT scanners from four vendors are not software related.}, month = {01}, keywords = {}, pubstate = {published}, tppubtype = {booklet} } Introduction: The Agatston score quantifying the amount of coronary artery calcifications provides risk discrimination and reclassification of patients at intermediate cardiovascular risk. However, state-of-the-art CT scanners from four different vendors have been shown to yield significantly different Agatston scores. The aim of this study was to investigate whether these differences are due to image acquisition and reconstruction or to differences in the vendor-specific calcium scoring software. Methods: CT images of fifteen ex-vivo human hearts were previously acquired on state-of-the-art CT scanners of four different vendors. All images were made with unenhanced ECG-triggered step-and-shoot protocols at equal radiation dose settings and reconstructed to 3 mm slice thickness and increment. In addition to previous work, where an Agatston score was obtained for each image using semi-automatic calcium scoring software from the scanner vendor, custom semi-automatic calcium scoring software was used to score calcium in all images. Differences in scores of all images of the same heart were analyzed using the Friedman test (significance level P < 0.05). Differences in scores obtained on the same image using the vendor-specific and the custom software were analyzed with the Wilcoxon signed-rank test (significance level P < 0.05). Results: Significant inter-scan differences (P < 0.05) in Agatston scores obtained using vendor software were previously reported for fourteen hearts with coronary calcifications. Corresponding median (interquartile range) Agatston scores were 332 (114-1135), 353 (172-1246), 410 (177-1454), and 469 (183-1381), respectively. Significant inter-scan differences persisted (P < 0.05) when images were scored in the custom software. Median (interquartile range) Agatston scores were 327 (115-1135), 353 (172-1246), 410 (180-1457), and 469 (183-1369), respectively. No significant intra-scan differences were found between scores obtained on the vendor software and on the custom software. Conclusions: Cardiac CT images obtained with scanners from four different vendors result in significantly different Agatston scores, also when all evaluated using the same software. There are no significant differences between scores obtained on vendor and custom software. Therefore, differences in calcium scores obtained using CT scanners from four vendors are not software related. |
| 2. | J.M. Wolterink; T. Leiner; P.A. de Jong; M.A. Viergever; I. Isgum Automatic coronary calcium scoring in ECG-triggered cardiac CT Abstract In: 2013. @booklet{wolterink:2013-2524, title = {Automatic coronary calcium scoring in ECG-triggered cardiac CT}, author = {J.M. Wolterink and T. Leiner and P.A. de Jong and M.A. Viergever and I. Isgum}, year = {2013}, date = {2013-01-01}, booktitle = {Radiological Society of North America, 99th Annual Meeting}, abstract = {PURPOSE Presence of coronary artery calcium (CAC) is a strong and independent predictor of cardiovascular events. Given the rapid increase in cardiac CT imaging, fully automated quantification of CAC is desired. We have developed an algorithm for quantification of CAC in routinely acquired cardiac CT employing a coronary calcium atlas and features previously proposed for automatic CAC scoring in non-ECG synchronized chest scans. METHOD AND MATERIALS The study included 161 consecutive cardiac patients who underwent CT for determination of CAC (256-detector row CT, 120 kV, 55 mAs, 3 mm section thickness). In all scans CAC was manually scored by experts. These annotations served as the reference standard and were used for training the algorithm. To automatically detect calcifications, scans were thresholded at 130 HU and all connected components were considered CAC candidates. These were described by size, intensity and spatial features. Spatial features were computed using a CAC atlas previously created with non ECG-gated chest CT scans. Subsequently, true positives were identified by classification using a forest of extremely randomized trees. Ten-fold cross validation was performed. Using the detected calcifications, Agatston and volume CAC scores were computed for each patient. Agreement between reference and automatic scores was established with Spearman's rho. Patients were assigned to a cardiovascular risk category based on their Agatston score (0, 1-10, 11-100, 101-400, >400) and linearly weighted kappa was calculated to evaluate the agreement in category assignment. RESULTS 24 cases with stents and beam hardening artifacts due to metal implants were excluded. In the remaining 137 patients median reference Agatston score was 8.8 (range: 0-4067) and automatic median score was 11.2 (range: 0-2793). Spearman's rho between automatic and reference scores was 0.93 for Agatston and 0.92 for volume. The correct risk category was assigned to 81% of patients, and 15% and 4% were one and two categories off, respectively (kappa=0.85). CONCLUSION The presented algorithm performs fully automatic CAC scoring. Based on the derived CAC scores, the vast majority of patients were assigned to the correct cardiovascular risk category. CLINICAL RELEVANCE/APPLICATION Automatic CAC scoring in cardiac CT can be performed using the features and CAC atlas described for scoring in chest CT. This eliminates the necessity for constructing a population specific CAC atlas.}, month = {01}, keywords = {}, pubstate = {published}, tppubtype = {booklet} } PURPOSE Presence of coronary artery calcium (CAC) is a strong and independent predictor of cardiovascular events. Given the rapid increase in cardiac CT imaging, fully automated quantification of CAC is desired. We have developed an algorithm for quantification of CAC in routinely acquired cardiac CT employing a coronary calcium atlas and features previously proposed for automatic CAC scoring in non-ECG synchronized chest scans. METHOD AND MATERIALS The study included 161 consecutive cardiac patients who underwent CT for determination of CAC (256-detector row CT, 120 kV, 55 mAs, 3 mm section thickness). In all scans CAC was manually scored by experts. These annotations served as the reference standard and were used for training the algorithm. To automatically detect calcifications, scans were thresholded at 130 HU and all connected components were considered CAC candidates. These were described by size, intensity and spatial features. Spatial features were computed using a CAC atlas previously created with non ECG-gated chest CT scans. Subsequently, true positives were identified by classification using a forest of extremely randomized trees. Ten-fold cross validation was performed. Using the detected calcifications, Agatston and volume CAC scores were computed for each patient. Agreement between reference and automatic scores was established with Spearman's rho. Patients were assigned to a cardiovascular risk category based on their Agatston score (0, 1-10, 11-100, 101-400, >400) and linearly weighted kappa was calculated to evaluate the agreement in category assignment. RESULTS 24 cases with stents and beam hardening artifacts due to metal implants were excluded. In the remaining 137 patients median reference Agatston score was 8.8 (range: 0-4067) and automatic median score was 11.2 (range: 0-2793). Spearman's rho between automatic and reference scores was 0.93 for Agatston and 0.92 for volume. The correct risk category was assigned to 81% of patients, and 15% and 4% were one and two categories off, respectively (kappa=0.85). CONCLUSION The presented algorithm performs fully automatic CAC scoring. Based on the derived CAC scores, the vast majority of patients were assigned to the correct cardiovascular risk category. CLINICAL RELEVANCE/APPLICATION Automatic CAC scoring in cardiac CT can be performed using the features and CAC atlas described for scoring in chest CT. This eliminates the necessity for constructing a population specific CAC atlas. |