Geometric alignment appears in a variety of applications, ranging from domain adaptation, optimal transport, and normalizing flows in machine learning; optical flow and learned augmentation in computer vision and deformable registration within biomedical imaging. A recurring challenge is the alignment of domains whose topology is not the same; a problem that is routinely ignored, potentially introducing bias in downstream analysis. As a first step towards solving such alignment problems, we propose an unsupervised algorithm for the detection of changes in image topology. The model is based on a conditional variational auto-encoder and detects topological changes between two images during the registration step. We account for both topological changes in the image under spatial variation and unexpected transformations. Our approach is validated on two tasks and datasets: detection of topological changes in microscopy images of cells, and unsupervised anomaly detection brain imaging.
@inproceedings{czolbe2021topology,author={Czolbe, Steffen and Feragen, Aasa and Krause, Oswin},file={:home/steffen/.local/share/data/Mendeley Ltd./Mendeley Desktop/Downloaded/Anonymous - 2021 - Spot the Difference Detection of Topological Changes via Geometric Alignment.pdf:pdf},booktitle={Advances in Neural Information Processing Systems},title={{Spot the Difference: Detection of Topological Changes via Geometric Alignment}},volume={34},year={2021},month=dec,abbr={NeurIPS},github={https://github.com/SteffenCzolbe/TopologicalChangeDetection},talk={https://www.youtube.com/watch?v=MQ4GvM0up5c&ab_channel=Steffen},project={_projects/2021-11-neurips2021_topology.md},pdf={spot_the_difference_detection_.pdf},selected={true},bibtex_show={true}}
MIDL
Award
Semantic similarity metrics for learned image registration
We propose a semantic similarity metric for image registration. Existing metrics like Euclidean Distance or Normalized Cross-Correlation focus on aligning intensity values, giving difficulties with low intensity contrast or noise. Our approach learns dataset-specific features that drive the optimization of a learning-based registration model. We train both an unsupervised approach using an auto-encoder, and a semi-supervised approach using supplemental segmentation data to extract semantic features for image registration. Comparing to existing methods across multiple image modalities and applications, we achieve consistently high registration accuracy. A learned invariance to noise gives smoother transformations on low-quality images.
@inproceedings{czolbe2021semantic,archiveprefix={arXiv},arxivid={2104.10051},author={Czolbe, Steffen and Krause, Oswin and Feragen, Aasa},eprint={2104.10051},booktitle={Proceedings of Machine Learning Research},month=jul,keywords={Deep Learning,Image Registration,Representation Learning},title={{Semantic similarity metrics for learned image registration}},url={https://arxiv.org/abs/2104.10051v1},year={2021},abbr={MIDL},tag={Award},github={https://github.com/SteffenCzolbe/DeepSimRegistration},talk={https://www.youtube.com/watch?v=Hs9X3wSO774&ab_channel=Steffen},pdf={semantic_similarity_metrics_fo.pdf},additional={<b>Best Paper Award Runner-Up</b>},selected={true},project={_projects/2021-07-midl2021_deepsim.md},bibtex_show={true}}
Probabilistic image segmentation encodes varying prediction confidence and
inherent ambiguity in the segmentation problem. While different probabilistic
segmentation models are designed to capture different aspects of segmentation
uncertainty and ambiguity, these modelling differences are rarely discussed in
the context of applications of uncertainty. We consider two common use cases of
segmentation uncertainty, namely assessment of segmentation quality and active
learning. We consider four established strategies for probabilistic
segmentation, discuss their modelling capabilities, and investigate their
performance in these two tasks. We find that for all models and both tasks,
returned uncertainty correlates positively with segmentation error, but does
not prove to be useful for active learning.
@inproceedings{czolbe2021segmentation,abbr={IPMI},author={Czolbe, Steffen* and Arnavaz, Kasra* and Krause, Oswin and Feragen, Aasa},booktitle={Lecture Notes in Computer Science},keywords={Active learning,Image segmentation,Uncertainty quantification},month=mar,pages={715--726},publisher={Springer Science and Business Media Deutschland GmbH},title={{Is segmentation uncertainty useful?}},url={https://arxiv.org/abs/2103.16265v1},volume={12729 LNCS},year={2021},pdf={is_segmentation_uncertainty_useful.pdf},github={https://github.com/SteffenCzolbe/probabilistic_segmentation},project={_projects/2021-06-ipmi2021_prob_segmentation.md},bibtex_show={true}}
2020
NeurIPS
A Loss Function for Generative Neural Networks Based on Watson’s Perceptual Model
@inproceedings{czolbe2020loss,title={A Loss Function for Generative Neural Networks Based on Watson’s Perceptual Model},author={Czolbe, Steffen and Krause, Oswin and Cox, Ingemar and Igel, Christian},booktitle={Advances in Neural Information Processing Systems},volume={33},month=dec,year={2020},abbr={NeurIPS},pdf={neurips2020.pdf},github={https://github.com/SteffenCzolbe/PerceptualSimilarity},talk={https://www.youtube.com/watch?v=qPmHQbR4DeI&ab_channel=Steffen},project={_projects/2020-12-neurips2020_watsons_loss.md},selected={true},bibtex_show={true}}
MedNeurIPS
Workshop
DeepSim: Semantic similarity metrics for learned image registration