The 3DPathology project addresses the increasing digital pathology demands with the reducing number of qualified pathologists by developing AI methods for automated analyzing, grading digital slices. Moreover, this project addresses the generation of 3D pathology samples and spectroscopic analysis.
Researchers
Partners
Philips, Barco, Prodrive, Xavis, PS-Tech.
Publications
Journal Articles |
|
1. | I. Jansen, M. Lucas, J. Bosschieter, O.J. de Boer, S.L. Meijer, T.G. van Leeuwen, H.A. Marquering, J.A. Nieuwenhuijzen, D.M. de Bruin,; C.D. Savci-Heijink Automated detection and grading of non-muscle-invasive urothelial cell carcinoma of the bladder Journal Article The American journal of pathology, 190 (7), pp. 1483-1490, 2020. @article{jansen2020, title = {Automated detection and grading of non-muscle-invasive urothelial cell carcinoma of the bladder}, author = {I. Jansen, M. Lucas, J. Bosschieter, O.J. de Boer, S.L. Meijer, T.G. van Leeuwen, H.A. Marquering, J.A. Nieuwenhuijzen, D.M. de Bruin, and C.D. Savci-Heijink}, doi = {10.1016/j.ajpath.2020.03.013 }, year = {2020}, date = {2020-04-10}, journal = {The American journal of pathology}, volume = {190}, number = {7}, pages = {1483-1490}, abstract = {Accurate grading of non-muscle-invasive urothelial cell carcinoma is of major importance; however, high interobserver variability exists. A fully automated detection and grading network based on deep learning is proposed to enhance reproducibility. A total of 328 transurethral resection specimens from 232 patients were included, and a consensus reading by three specialized pathologists was used. The slides were digitized, and the urothelium was annotated by expert observers. The U-Net-based segmentation network was trained to automatically detect urothelium. This detection was used as input for the classification network. The classification network aimed to grade the tumors according to the World Health Organization grading system adopted in 2004. The automated grading was compared with the consensus and individual grading. The segmentation network resulted in an accurate detection of urothelium. The automated grading shows moderate agreement (κ = 0.48 ± 0.14 SEM) with the consensus reading. The agreement among pathologists ranges between fair (κ = 0.35 ± 0.13 SEM and κ = 0.38 ± 0.11 SEM) and moderate (κ = 0.52 ± 0.13 SEM). The automated classification correctly graded 76% of the low-grade cancers and 71% of the high-grade cancers according to the consensus reading. These results indicate that deep learning can be used for the fully automated detection and grading of urothelial cell carcinoma. }, keywords = {}, pubstate = {published}, tppubtype = {article} } Accurate grading of non-muscle-invasive urothelial cell carcinoma is of major importance; however, high interobserver variability exists. A fully automated detection and grading network based on deep learning is proposed to enhance reproducibility. A total of 328 transurethral resection specimens from 232 patients were included, and a consensus reading by three specialized pathologists was used. The slides were digitized, and the urothelium was annotated by expert observers. The U-Net-based segmentation network was trained to automatically detect urothelium. This detection was used as input for the classification network. The classification network aimed to grade the tumors according to the World Health Organization grading system adopted in 2004. The automated grading was compared with the consensus and individual grading. The segmentation network resulted in an accurate detection of urothelium. The automated grading shows moderate agreement (κ = 0.48 ± 0.14 SEM) with the consensus reading. The agreement among pathologists ranges between fair (κ = 0.35 ± 0.13 SEM and κ = 0.38 ± 0.11 SEM) and moderate (κ = 0.52 ± 0.13 SEM). The automated classification correctly graded 76% of the low-grade cancers and 71% of the high-grade cancers according to the consensus reading. These results indicate that deep learning can be used for the fully automated detection and grading of urothelial cell carcinoma. |
2. | M. Lucas, E.I. Liem, C.D. Savci-Heijink, J.E. Freund, H.A. Marquering, T.G. van Leeuwen; D.M. de Bruin Toward Automated In Vivo Bladder Tumor Stratification Using Confocal Laser Endomicroscopy Journal Article Journal of Endourology, 33 (11), pp. 930-937, 2019. @article{lucas2020toward, title = {Toward Automated In Vivo Bladder Tumor Stratification Using Confocal Laser Endomicroscopy}, author = {M. Lucas, E.I. Liem, C.D. Savci-Heijink, J.E. Freund, H.A. Marquering, T.G. van Leeuwen and D.M. de Bruin}, doi = {10.1089/end.2019.0354}, year = {2019}, date = {2019-10-29}, journal = {Journal of Endourology}, volume = {33}, number = {11}, pages = {930-937}, abstract = {Purpose: Urothelial carcinoma of the bladder (UCB) is the most common urinary cancer. White-light cystoscopy (WLC) forms the corner stone for the diagnosis of UCB. However, histopathological assessment is required for adjuvant treatment selection. Probe-based confocal laser endomicroscopy (pCLE) enables visualization of the microarchitecture of bladder lesions during WLC, which allows for real-time tissue differentiation and grading of UCB. To improve the diagnostic process of UCB, computer-aided classification of pCLE videos of in vivo bladder lesions were evaluated in this study. Materials and Methods: We implemented preprocessing methods to optimize contrast and to reduce striping artifacts in each individual pCLE frame. Subsequently, a semiautomatic frame selection was performed. The selected frames were used to train a feature extractor based on pretrained ImageNet networks. A recurrent neural network, in specific long short-term memory (LSTM), was used to predict the grade of bladder lesions. Differentiation of lesions was performed at two levels, namely (i) healthy and benign vs malignant tissue and (ii) low-grade vs high-grade papillary UCB. A total of 53 patients with 72 lesions were included in this study, resulting in ∼140,000 pCLE frames. Results: The semiautomated frame selection reduced the number of frames to ∼66,500 informative frames. The accuracy for differentiation of (i) healthy and benign vs malignant urothelium was 79% and (ii) high-grade and low-grade papillary UCB was 82%. Conclusions: A feature extractor in combination with LSTM results in proper stratification of pCLE videos of in vivo bladder lesions.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Purpose: Urothelial carcinoma of the bladder (UCB) is the most common urinary cancer. White-light cystoscopy (WLC) forms the corner stone for the diagnosis of UCB. However, histopathological assessment is required for adjuvant treatment selection. Probe-based confocal laser endomicroscopy (pCLE) enables visualization of the microarchitecture of bladder lesions during WLC, which allows for real-time tissue differentiation and grading of UCB. To improve the diagnostic process of UCB, computer-aided classification of pCLE videos of in vivo bladder lesions were evaluated in this study. Materials and Methods: We implemented preprocessing methods to optimize contrast and to reduce striping artifacts in each individual pCLE frame. Subsequently, a semiautomatic frame selection was performed. The selected frames were used to train a feature extractor based on pretrained ImageNet networks. A recurrent neural network, in specific long short-term memory (LSTM), was used to predict the grade of bladder lesions. Differentiation of lesions was performed at two levels, namely (i) healthy and benign vs malignant tissue and (ii) low-grade vs high-grade papillary UCB. A total of 53 patients with 72 lesions were included in this study, resulting in ∼140,000 pCLE frames. Results: The semiautomated frame selection reduced the number of frames to ∼66,500 informative frames. The accuracy for differentiation of (i) healthy and benign vs malignant urothelium was 79% and (ii) high-grade and low-grade papillary UCB was 82%. Conclusions: A feature extractor in combination with LSTM results in proper stratification of pCLE videos of in vivo bladder lesions. |
3. | M. Lucas, I. Jansen, C.D. Savci-Heijink, S.L. Meijer, O.J. de Boer, T.G. van Leeuwen, D.M. de Bruin; H.A. Marquering Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies Journal Article Virchows Archiv, 475 (1), pp. 77-83, 2019. @article{lucas2019deep, title = {Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies}, author = {M. Lucas, I. Jansen, C.D. Savci-Heijink, S.L. Meijer, O.J. de Boer, T.G. van Leeuwen, D.M. de Bruin and H.A. Marquering}, doi = {10.1007/s00428-019-02577-x}, year = {2019}, date = {2019-07-01}, journal = {Virchows Archiv}, volume = {475}, number = {1}, pages = {77-83}, abstract = {Histopathologic grading of prostate cancer using Gleason patterns (GPs) is subject to a large inter-observer variability, which may result in suboptimal treatment of patients. With the introduction of digitization and whole-slide images of prostate biopsies, computer-aided grading becomes feasible. Computer-aided grading has the potential to improve histopathological grading and treatment selection for prostate cancer. Automated detection of GPs and determination of the grade groups (GG) using a convolutional neural network. In total, 96 prostate biopsies from 38 patients are annotated on pixel-level. Automated detection of GP 3 and GP ≥ 4 in digitized prostate biopsies is performed by re-training the Inception-v3 convolutional neural network (CNN). The outcome of the CNN is subsequently converted into probability maps of GP ≥ 3 and GP ≥ 4, and the GG of the whole biopsy is obtained according to these probability maps. Differentiation between non-atypical and malignant (GP ≥ 3) areas resulted in an accuracy of 92% with a sensitivity and specificity of 90 and 93%, respectively. The differentiation between GP ≥ 4 and GP ≤ 3 was accurate for 90%, with a sensitivity and specificity of 77 and 94%, respectively. Concordance of our automated GG determination method with a genitourinary pathologist was obtained in 65% (κ = 0.70), indicating substantial agreement. A CNN allows for accurate differentiation between non-atypical and malignant areas as defined by GPs, leading to a substantial agreement with the pathologist in defining the GG.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Histopathologic grading of prostate cancer using Gleason patterns (GPs) is subject to a large inter-observer variability, which may result in suboptimal treatment of patients. With the introduction of digitization and whole-slide images of prostate biopsies, computer-aided grading becomes feasible. Computer-aided grading has the potential to improve histopathological grading and treatment selection for prostate cancer. Automated detection of GPs and determination of the grade groups (GG) using a convolutional neural network. In total, 96 prostate biopsies from 38 patients are annotated on pixel-level. Automated detection of GP 3 and GP ≥ 4 in digitized prostate biopsies is performed by re-training the Inception-v3 convolutional neural network (CNN). The outcome of the CNN is subsequently converted into probability maps of GP ≥ 3 and GP ≥ 4, and the GG of the whole biopsy is obtained according to these probability maps. Differentiation between non-atypical and malignant (GP ≥ 3) areas resulted in an accuracy of 92% with a sensitivity and specificity of 90 and 93%, respectively. The differentiation between GP ≥ 4 and GP ≤ 3 was accurate for 90%, with a sensitivity and specificity of 77 and 94%, respectively. Concordance of our automated GG determination method with a genitourinary pathologist was obtained in 65% (κ = 0.70), indicating substantial agreement. A CNN allows for accurate differentiation between non-atypical and malignant areas as defined by GPs, leading to a substantial agreement with the pathologist in defining the GG. |
4. | I. Jansen, M. Lucas, C.D. Savci-Heijink, S.L. Meijer, E.I. Liem, O.J. de Boer, T.G. van Leeuwen, H.A. Marquering; D.M. de Bruin Three-dimensional histopathological reconstruction of bladder tumours Journal Article Diagnostic pathology, 14 (1), pp. 1-7, 2019. @article{jansen2019three, title = {Three-dimensional histopathological reconstruction of bladder tumours}, author = {I. Jansen, M. Lucas, C.D. Savci-Heijink, S.L. Meijer, E.I. Liem, O.J. de Boer, T.G. van Leeuwen, H.A. Marquering and D.M. de Bruin}, doi = {10.1186/s13000-019-0803-7}, year = {2019}, date = {2019-03-28}, journal = {Diagnostic pathology}, volume = {14}, number = {1}, pages = {1-7}, abstract = {Background Histopathological analysis is the cornerstone in bladder cancer (BCa) diagnosis. These analysis suffer from a moderate observer agreement in the staging of bladder cancer. Three-dimensional reconstructions have the potential to support the pathologists in visualizing spatial arrangements of structures, which may improve the interpretation of specimen. The aim of this study is to present three-dimensional (3D) reconstructions of histology images. Methods En-bloc specimens of transurethral bladder tumour resections were formalin fixed and paraffin embedded. Specimens were cut into sections of 4 μm and stained with Hematoxylin and Eosin (H&E). With a Phillips IntelliSite UltraFast scanner, glass slides were digitized at 20x magnification. The digital images were aligned by performing rigid and affine image alignment. The tumour and the muscularis propria (MP) were manually delineated to create 3D segmentations. In conjunction with a 3D display, the results were visualized with the Vesalius3D interactive visualization application for a 3D workstation. Results En-bloc resection was performed in 21 BCa patients. Per case, 26–30 sections were included for the reconstruction into a 3D volume. Five cases were excluded due to export problems, size of the dataset or condition of the tissue block. Qualitative evaluation suggested an accurate registration for 13 out of 16 cases. The segmentations allowed full 3D visualization and evaluation of the spatial relationship of the BCa tumour and the MP. Conclusion Digital scanning of en-bloc resected specimens allows a full-fledged 3D reconstruction and analysis and has a potential role to support pathologists in the staging of BCa.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Background Histopathological analysis is the cornerstone in bladder cancer (BCa) diagnosis. These analysis suffer from a moderate observer agreement in the staging of bladder cancer. Three-dimensional reconstructions have the potential to support the pathologists in visualizing spatial arrangements of structures, which may improve the interpretation of specimen. The aim of this study is to present three-dimensional (3D) reconstructions of histology images. Methods En-bloc specimens of transurethral bladder tumour resections were formalin fixed and paraffin embedded. Specimens were cut into sections of 4 μm and stained with Hematoxylin and Eosin (H&E). With a Phillips IntelliSite UltraFast scanner, glass slides were digitized at 20x magnification. The digital images were aligned by performing rigid and affine image alignment. The tumour and the muscularis propria (MP) were manually delineated to create 3D segmentations. In conjunction with a 3D display, the results were visualized with the Vesalius3D interactive visualization application for a 3D workstation. Results En-bloc resection was performed in 21 BCa patients. Per case, 26–30 sections were included for the reconstruction into a 3D volume. Five cases were excluded due to export problems, size of the dataset or condition of the tissue block. Qualitative evaluation suggested an accurate registration for 13 out of 16 cases. The segmentations allowed full 3D visualization and evaluation of the spatial relationship of the BCa tumour and the MP. Conclusion Digital scanning of en-bloc resected specimens allows a full-fledged 3D reconstruction and analysis and has a potential role to support pathologists in the staging of BCa. |
5. | D.R.N. Vos, I. Jansen, M. Lucas, M.R.L. Paine, O.J. de Boer, S.L. Meijer, C.D. Savci-Heijink, H.A. Marquering, D.M. de Bruin, R.M.A. Heeren; S.R. Ellis Strategies for managing multi-patient 3D mass spectrometry imaging data Journal Article Journal of Proteomics, 193 , pp. 184-191, 2019. @article{vos2019stategies, title = {Strategies for managing multi-patient 3D mass spectrometry imaging data}, author = {D.R.N. Vos, I. Jansen, M. Lucas, M.R.L. Paine, O.J. de Boer, S.L. Meijer, C.D. Savci-Heijink, H.A. Marquering, D.M. de Bruin, R.M.A. Heeren and S.R. Ellis}, doi = {10.1016/j.jprot.2018.10.008}, year = {2019}, date = {2019-02-20}, journal = {Journal of Proteomics}, volume = {193}, pages = {184-191}, abstract = {Mass spectrometry imaging (MSI) has emerged as a powerful tool in biomedical research to reveal the localization of a broad scale of compounds ranging from metabolites to proteins in diseased tissues, such as malignant tumors. MSI is most commonly used for the two-dimensional imaging of tissues from multiple patients or for the three-dimensional (3D) imaging of tissue from a single patient. These applications are potentially introducing a sampling bias on a sample or patient level, respectively. The aim of this study is therefore to investigate the consequences of sampling bias on sample representativeness and on the precision of biomarker discovery for histological grading of human bladder cancers by MSI. We therefore submitted formalin-fixed paraffin-embedded tissues from 14 bladder cancer patients with varying histological grades to 3D analysis by matrix-assisted laser desorption/ionization (MALDI) MSI. We found that, after removing 20% of the data based on novel outlier detection routines for 3D-MSI data based on the evaluation of digestion efficacy and z-directed regression, on average 33% of a sample has to be measured in order to obtain sufficient coverage of the existing biological variance within a tissue sample. Significance: In this study, 3D MALDI-MSI is applied for the first time on a cohort of bladder cancer patients using formalin-fixed paraffin-embedded (FFPE) tissue of bladder cancer resections. This work portrays the reproducibility that can be achieved when employing an optimized sample preparation and subsequent data evaluation approach. Our data shows the influence of sampling bias on the variability of the results, especially for a small patient cohort. Furthermore, the presented data analysis workflow can be used by others as a 3D FFPE data-analysis pipeline working on multi-patient 3D-MSI studies.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Mass spectrometry imaging (MSI) has emerged as a powerful tool in biomedical research to reveal the localization of a broad scale of compounds ranging from metabolites to proteins in diseased tissues, such as malignant tumors. MSI is most commonly used for the two-dimensional imaging of tissues from multiple patients or for the three-dimensional (3D) imaging of tissue from a single patient. These applications are potentially introducing a sampling bias on a sample or patient level, respectively. The aim of this study is therefore to investigate the consequences of sampling bias on sample representativeness and on the precision of biomarker discovery for histological grading of human bladder cancers by MSI. We therefore submitted formalin-fixed paraffin-embedded tissues from 14 bladder cancer patients with varying histological grades to 3D analysis by matrix-assisted laser desorption/ionization (MALDI) MSI. We found that, after removing 20% of the data based on novel outlier detection routines for 3D-MSI data based on the evaluation of digestion efficacy and z-directed regression, on average 33% of a sample has to be measured in order to obtain sufficient coverage of the existing biological variance within a tissue sample. Significance: In this study, 3D MALDI-MSI is applied for the first time on a cohort of bladder cancer patients using formalin-fixed paraffin-embedded (FFPE) tissue of bladder cancer resections. This work portrays the reproducibility that can be achieved when employing an optimized sample preparation and subsequent data evaluation approach. Our data shows the influence of sampling bias on the variability of the results, especially for a small patient cohort. Furthermore, the presented data analysis workflow can be used by others as a 3D FFPE data-analysis pipeline working on multi-patient 3D-MSI studies. |
6. | I. Jansen, M. Lucas, C.D. Savci-Heijink, S.L. Meijer, H.A. Marquering, D.M. de Bruin; P.J. Zondervan Histopathology: ditch the slides, because digital and 3D are on show Journal Article World journal of urology, 36 (4), pp. 549-555, 2018. @article{jansen2018histopathology, title = {Histopathology: ditch the slides, because digital and 3D are on show}, author = {I. Jansen, M. Lucas, C.D. Savci-Heijink, S.L. Meijer, H.A. Marquering, D.M. de Bruin and P.J. Zondervan}, doi = {10.1007/s00345-018-2202-1}, year = {2018}, date = {2018-04-01}, journal = {World journal of urology}, volume = {36}, number = {4}, pages = {549-555}, abstract = {Due to the growing field of digital pathology, more and more digital histology slides are becoming available. This improves the accessibility, allows teleconsultations from specialized pathologists, improves education, and might give urologist the possibility to review the slides in patient management systems. Moreover, by stacking multiple two-dimensional (2D) digital slides, three-dimensional volumes can be created, allowing improved insight in the growth pattern of a tumor. With the addition of computer-aided diagnosis systems, pathologist can be guided to regions of interest, potentially reducing the workload and interobserver variation. Digital (3D) pathology has the potential to improve dialog between the pathologist and urologist, and, therefore, results in a better treatment selection for urologic patients.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Due to the growing field of digital pathology, more and more digital histology slides are becoming available. This improves the accessibility, allows teleconsultations from specialized pathologists, improves education, and might give urologist the possibility to review the slides in patient management systems. Moreover, by stacking multiple two-dimensional (2D) digital slides, three-dimensional volumes can be created, allowing improved insight in the growth pattern of a tumor. With the addition of computer-aided diagnosis systems, pathologist can be guided to regions of interest, potentially reducing the workload and interobserver variation. Digital (3D) pathology has the potential to improve dialog between the pathologist and urologist, and, therefore, results in a better treatment selection for urologic patients. |