We all know pathology, thinking of the healthcare professional examining a
tissue section under a microscope, looking for evidence of cancerous cells. Many years ago, this process was
digitalised by capturing the glass slides with a scanning device to provide a high-resolution digital image
that can be viewed on a computer screen or mobile device, facilitating the acquisition, management, sharing
and interpretation of pathology information. This way pathologists can engage, evaluate and collaborate
rapidly and remotely, and thereby improve efficiency and productivity, as views, analysis and workflows are
improved, while errors and turnaround times are reduced.
In 2015, the digital pathology market was set to multiply by more than five times within five years. This
growth, coupled with declining numbers of qualified pathologists, represented a tremendous challenge for
pathology departments of clinical and pharmaceutical organisations, and increased the urgency for higher
quality diagnostic information to enable more effective and efficient treatments.
Digital images were becoming increasingly more accurate to render 3D shapes of objects, as seen in some
medical imaging techniques. Organs structures and contents were already revealed in 3D distribution, but
this was not yet the case for tissues, which require microscopic spatial resolution to develop 3D analysis.
The main bottleneck to achieving efficient 3D imaging of tissues was to provide a quantitative and global
analysis at microscopic resolution.
The ITEA project 3DPathology, headed by Barco and
Philips along with knowledge partners and university
hospitals from six countries, was set out to create a 3D digital pathology solution, based on a combination
of multiple existing pathology modalities, for same-day diagnosis and much more personalised treatment of
Unique 3D multiple imaging
In order to overcome the anticipated challenges and to achieve this multi-modal 3D quantitative
pathology analysis platform, the project had to come to terms with solving five major technological
- The first was the 3D acquisition of data using multiple imaging modalities in a fast and automated
- Secondly, the acquired data of some modalities had to be reconstructed and aligned from actual 2D
images of pathologic slices into 3D images, combined with different modalities (size, resolution,
storage format, spectral bandwidth), including the incorporation of techniques like alternating
co-registration, light- or molecule channel-based slide alignment and reconstruction.
- Then the aligned 3D data from different modalities had to be analysed to improve the quality of
diagnosis. This was achieved by extracting and combining the relevant data, using techniques such as
quantification, segmentation, machine learning and data mining.
- Furthermore, the project came up with the development of new 3D visualisation and interaction
technologies (equipment and algorithms) optimised for multi-modal 3D pathology.
- And finally, an IT backbone was created to deal with data of tremendous size, produced by the
individual imaging modalities (data sets in the range of Tera- to Petabytes).
A 3D multi-modal pathology demonstrator, the first of its kind in the world, enables unique features such as
access to the microscopic organisation of tissue sub-structures in 3D, providing complete chemical
information and access to unexplored dimensions of histology. By scaling up 3D microscopic images of bio
samples, a better understanding is gained of the relationship
between the morphological structures and the molecular, biochemical and metabolic information
of tissue while the extraction of quantitative data enhances the molecular/chemical parameters
ratio with respect to the structured components of a tissue. Finally, the 3D visualisation of, and
interaction with, the relevant data from multiple imaging modalities optimises the presentation
of the relevant views and parameters and allows the huge amounts of data (e.g. storage, transfer,
processing, rendering) to be handled.
Impressive high-impact results
The 3DPathology project has had a significant impact on JPEG XS standardisation, which focuses on near
loss-free, low-latency coding of high-resolution data. Intensive collaboration between imec, ETRO and VUB
resulted in the launch of a new extension of JPEG 2000, namely part 15 High-throughput JPEG 2000 that
particularly reduces the computational complexity and memory footprint of the EBCOT component of the JPEG
2000 standard, which makes it very interesting for the pathology use case.
In the field of dissemination, the project partners also achieved incredible results. Virchows Archiv
European Journal of Pathology has awarded the unique ‘Virchows Archiv prize for the best paper of the year
2019’ to the paper ‘Deep learning for automatic Gleason pattern classification for grade group determination
of prostate biopsies’ co-written by one of the project associates Dr. Marit Lucas from Academic Medical
Center of the University of Amsterdam.
"By exposing pathologists to more efficient workflows and faster learning, their knowledge and experience increase, enabling them to be more efficient"
In addition, by exposing pathologists to more efficient workflows and faster learning, their knowledge and
experience increase, enabling them to be more efficient. Increasing the accuracy in pathological examination
practice and interpretation has a significant impact on improving quality of life due to personalised
treatment, limiting re-occurrence as a result of better treatment outcomes and a reduction in the cost of
healthcare from fewer readmissions. Thanks to the insights and results, Academic Medical Center of the
University of Amsterdam reported a 10% reduction of re-occurrences/readmissions, considering the cost for a
typical re-occurrence/readmission for bladder cancer diagnostics is 2 to 3k euros for every 6 months. Added
benefits of workflow efficiency are a lower burden on pathology labs and mitigation of the lack of qualified
pathologists in the future.
In respect of exploitation, Philips has been given FDA clearance in the US to market its IntelliSite
Pathology Solution for primary diagnostic use there. Philips expects the results of the project to help
bring new pathology scanners to market and an innovative multi-layer bright field imaging solution to
increase its market share of the bright field pathology. Increased usability range and robustness will
address the needs for both small labs and large medical centres.
Prodrive Technologies will support Philips to increase market share and secure Philips’ position in the
Pathology market. As a result of the 3DPathology project, Prodrive Technologies finalised the scan engine
design and the production tools, which are now available. This emerging technology project will provide
better patient care. Furthermore, it will result in significantly higher revenue and contribute to a healthy
and stable business relationship. In parallel, the 3D functionality will be improved and released. After
finalising the 3D functionality, Prodrive Technologies will resell the 3D camera and parts of the scan
engine technology in other application areas.
Slimmer AI, formerly Target Holding, gained a lot of experience in image handling and analysis from 3D
molecular image alignment. This has been applied in different customer cases and proofs of concept (PoC),
such as cooperations in optical (infrared) product inspection, change detection in sewer levels for water
authorities (multi-modal: satellite, aerial photograph, on-spot), in aerial photographs for the land and
land-use registration authority, and in technical drawings. It also was a starting point for the successful
participation in the ITEA Medolution project.
Currently, the AI-based image analysis line is combined with Slimmer AI’s Natural Language Processing
developments to form the PoC-version of an innovative data-room tool, in co-creation with a launching
customer. This tool might become Slimmer AI’s next product, leading to a 20 FTE-spin-out within 5 years.
Slimmer AI developed the prototype of the 3D Digital Pathology IT backbone infrastructure and provided it to
the consortium for the secure exchange of (here: anonymised) medical data. The blueprint is still in use for
high-security data-exchange with clients.
- A 3D multi-modal pathology demonstrator, the first of its kind in the world, enables unique
features such as access to the microscopic organisation of tissue sub-structures in 3D, providing
complete chemical information and access to unexplored dimensions of histology.
- Academic Medical Center of the University of Amsterdam reported a 10% reduction of
re-occurrences/readmissions, considering the cost for a typical re-occurrence/readmission for
bladder cancer diagnostics is 2 to 3k euros for every 6 months.
- Barco has already sold several hundred optimised display systems that address a variety of
pathology lab needs, worldwide, and in the next few years Barco is expecting a large increase in
sales of display systems for Digital Pathology.
- Slimmer AI combines the AI-based image analysis line with its Natural Language Processing
developments to form the PoC-version of an innovative data-room tool, in co-creation with a
launching customer. This tool might become Slimmer AI’s next product, leading to a 20 FTE-spin-out
within 5 years.
- Philips has been given FDA clearance in the US to market its IntelliSite Pathology Solution for
primary diagnostic use there.
Barco has developed optimised display systems that address a variety of pathology lab needs for review,
positioning of samples but also for diagnostic purposes. Several hundred of those displays have been sold
already worldwide, and in the next few years Barco is expecting a large increase in sales of display systems
for Digital Pathology. In addition, Barco prepared a White Paper for the Medical Imaging Working Group of
ICC describing the importance of colour calibration for medical imaging and perceptual colour calibration.
The White Paper is a first step towards the standardisation of medical colour imaging.
PS-Tech has developed masking technology for extractions within volumetric datasets. This technology, which
has now been commercialised in Vesalius3D, is used for preoperative planning in various cardiovascular
procedures. Applications for lung and tumour extractions are under development. The technology developed for
zooming within large datasets is still under further development and will be included in a Vesalius3d
Imec, formerly iMinds, contributed to the standardisation of formal evaluation methodologies to assess the
performance of visually loss-free compression systems, for still images and video sequences, and for high
dynamic range content, as part of JPEG’s AIC activity (ISO/IEC 29170-1 and ISO/IEC 29170-2). These
methodologies can be used as a basis to design a clinical validation test for the compressed 3D pathology
data, in cooperation with Barco.
For the Korean partner Xavis, the ITEA 3DPathology project will result in bringing to market new 3D X-ray
Microscopy Instrumentation capable of high-resolution, non-destructive imaging and analysis for the
quantification of internal structural parameters at submicron to nanometre scale. Although the global and
domestic investments have decreased because of the COVID-19 pandemic, XAVIS exerts all possible efforts at
continual marketing to a number of medical facilities and national research institutes by sample test
reporting and online meetings.
In addition to these strong results, 3DPathology was the first project in ITEA to involve Taiwan with the
active participation of Academia Sinica and Bio Ma-tek, who strengthened this project as a solution provider
for nano-biomaterial characterisation and analysis.
The novel disease diagnosis mechanism of cell units is expected to be investigated by the nano-biomaterial
X-ray microscope system, which is not available in the conventional X-ray system. On the basis of the novel
X-ray microscope system, nano-scale medical devices will be adjustable not only for the new disease
diagnosis methods but also for the new cancer cell detection methods. As a result, the nano-biomaterial
X-ray microscope system is expected to be an innovative development for medical and diagnostic services that
contribute to improving the healthcare system on a global scale.