Unsupervised Microscopy Video Denoising
| Mary Damilola Aiyetigbo1 | Alexander Korte1 | Ethan Anderson1 | Reda Chalhoub2 | Peter Kalivas2 | Feng Luo1 | Nianyi Li1 |
| 1Clemson University | 2The Medical University of South Carolina |
Two Photon Calcium Imaging
Abstract
In this paper, we introduce a novel unsupervised network to denoise microscopy videos featured by image sequences captured by a fixed location microscopy camera. Specifically, we propose a DeepTemporal Interpolation method, leveraging a temporal signal filter integrated into the bottom CNN layers, to restore microscopy videos corrupted by unknown noise types. Our unsupervised denoising architecture is distinguished by its ability to adapt to multiple noise conditions without the need for pre-existing noise distribution knowledge, addressing a significant challenge in real-world medical applications. Furthermore, we evaluate our denoising framework using both real microscopy recordings and simulated data, validating our outperforming video denoising performance across a broad spectrum of noise scenarios. Extensive experiments demonstrate that our unsupervised model consistently outperforms state-of-the-art supervised and unsupervised video denoising techniques, proving especially effective for microscopy videos.
Architecture
Our method enhances video denoising by integrating two main components: a feature generator $\mathcal{G}\phi$ and a Denoiser $\mathcal{D}\theta$. This approach diverges from traditional methods by applying a temporal filter to feature maps generated by CNN layers, before denoising, rather than directly interpolating frames. This process involves selecting a subset of frames from a noisy video, processing these through the feature generator to produce feature maps, and then weighting these feature maps based on their temporal proximity to the central frame using a temporal filter. The temporal filter ${ \gamma_t}{t=1}^N$ weights these feature maps ${\mathbf{F}t}{t=1}^N$ by assigning diminished values to features nearer the central frame compared to those more distant. The weighted feature maps are then concatenated and processed by the Denoiser to produce the final denoised frame. This technique is particularly effective for videos with high frame rates and slow-moving objects, showcasing a significant improvement in denoising capabilities.
Results
One Photon Calcium Imaging
| Two Photon Calcium Imaging | Fluorescence Microscopy |
|---|---|
 |
 |
Results on Natural Videos
| Bobblehead |
|---|
 |
| Runner |
 |
Performance in Denoising Synthetic Noise. This table presents a comparison of average PSNR/SSIM values of denoised performance on LIVE-YT-HFR datasets on different noise types and intensities. Our method demonstrates superior performance in most cases and remains highly competitive with the supervised methods.