Marine Snow Removal Benchmarking Dataset

\authorblockN Reina Kaneko\authorrefmark1, Yuya Sato\authorrefmark2, Takumi Ueda\authorrefmark2, Hiroshi Higashi\authorrefmark1, and Yuichi Tanaka\authorrefmark1
\authorblockA \authorrefmark1 Osaka University, Japan
\authorrefmark2 Tokyo University of Agriculture and Technology, Japan
\authorblockA E-mail: [email protected], {higashi, ytanaka}@comm.eng.osaka-u.ac.jp This work is supported in part by Japan Science and Technology Agency (JST) Advanced International Collaborative Research Program (AdCORP) under Grant JPMJKB2307 and JSPS KAKENHI under Grant 23K17461.

Abstract

This paper introduces a new benchmarking dataset for marine snow removal of underwater images. Marine snow is one of the main degradation sources of underwater images that are caused by small particles, e.g., organic matter and sand, between the underwater scene and photosensors. We mathematically model two typical types of marine snow from the observations of real underwater images. The modeled artifacts are synthesized with underwater images to construct large-scale pairs of ground truth and degraded images to calculate objective qualities for marine snow removal and to train a deep neural network. We propose two marine snow removal tasks using the dataset and show the first benchmarking results of marine snow removal. The Marine Snow Removal Benchmarking Dataset is publicly available online.

1 Introduction

Image restoration has been one of the main topics of computer vision for decades [38, 10, 28, 35, 11, 36, 37]. In the current era of deep learning, the quality of image restoration for various tasks has been significantly improved under a sufficient number of pairs of ground truth and degraded images. Accordingly, the current focus on image restoration will be for images taken under extreme situations such as underwater, satellite, and medical images [1, 6, 24, 32, 33, 23, 14, 20, 12, 39, 41, 27, 31, 26, 34, 3]. This problem is called extreme image restoration herein. Extreme images often have an untypical degradation beyond that of a Gaussian model, and the numbers of available images for training and modifying restoration algorithms are also limited. As a result, extreme image restoration is a challenging problem.

In this paper, we focus on underwater image enhancement. Its main challenge is mostly due to the fact that we generally have no ground truth images. This makes it difficult to evaluate the objective qualities of the restored images and measure the restoration performance. Moreover, the lack of pairs of ground truth and degraded image causes a difficulty in training a neural network (if we consider using a deep neural network for restoration) because the loss functions of the image restoration typically evaluate some objective image quality metrics such as mean squared error (MSE) and structured similarity index (SSIM) [40]. Hence, unsupervised restoration methods, including linear and nonlinear filtering, are still popular in underwater image enhancement [6, 24, 2].

Several studies for underwater image enhancement with deep neural networks have been proposed [32, 33, 23, 14, 20, 18, 22, 42]. Most focus on removing the color shift from underwater images. That is, the blueish underwater images are restored by enhancing the red and green channels. Conventional underwater image enhancement methods using deep neural networks tackle the problem on the lack of large-scale underwater image datasets in various ways. For example, the studies in [33, 4, 32, 14] synthetically generate degraded images from clean images taken on the ground by simulating the degradation processes of absorption and scattering. Moreover, [20, 25, 15] use generative adversarial networks. And a number of challenges to improving the restoration performance still exist, some promising results have been presented. However, such studies ignore another main source of degradation for underwater images: Marine snow.

In this paper, we consider a marine snow removal (MSR) problem for underwater images by constructing a large-scale dataset containing underwater image pairs. Marine snow is a typical underwater image degradation, examples of which are shown in Fig. 1. Marine snow artifacts are visible in digital images when we take an underwater image using a flashlight. Small particles in the scene, e.g., organic matter and sand, cause marine snow. The particles reflect the light from the flashlight, and the reflected light will be captured by the photosensors. Marine snow is often annoying in underwater photography because it affects the overall image quality. However, the particles are unevenly distributed in the scene, and the degradation process is difficult to model completely. Therefore, standard methods for marine snow removal are still limited to simple median-filter-based approaches [5, 13, 21, 18]. Moreover, as with standard underwater image enhancement, we do not have a pair of ground truth and degraded images to calculate the objective image qualities and tune the parameters of the restoration algorithms.

Refer to caption — Figure 1: Marine snow artifacts in real underwater images.

We propose marine snow synthesis methods for solving the above-mentioned problems of MSR. In other words, we intentionally append the marine snow artifacts to real underwater images without marine snow and generate pairs of ground truth images and degraded images (i.e., clean underwater images and their corresponding images with marine snow artifacts). The key for synthesizing artifacts is to mathematically model marine snow in digital images as accurately as possible. From actual marine snow images, we discover that many marine snow artifacts can be modeled through two typical pixel value distributions. By utilizing this finding, we can synthetically generate many pairs of underwater images with/without marine snow artifacts.

We also propose two marine snow removal tasks and develop datasets corresponding to these tasks. This Marine Snow Removal Benchmarking Dataset (MSRB Dataset) is the first attempt in this field and is publicly available online¹¹1https://github.com/ychtanaka/marine-snow. We also present the first MSR benchmarking results using the MSRB Dataset: We compare the objective performance of marine snow removal among median filters and a deep neural network. Real MSR results as well as the limitations are also described.

The remainder of this paper is organized as follows: Section 2 reviews several related studies for underwater image enhancement. Two representative marine snow models that we discovered are presented in Section 3 along with their synthesizing method. The MSRB Dataset specifications are introduced in Section 4 and include the description of the two tasks. The first MSR benchmarking results with objective image quality metrics are shown in Section 5. Real MSR results and limitations are described in Section 6. Finally, we provide some concluding remarks in Section 7.

2 Related Studies

There have been few studies on modeling marine snow in digital images. Its seminal study is described in [7, 8]. A marine snow artifact is modeled using a Gaussian function in which the artifact is less transparent in its center than in its surrounding area. This method is better than the traditional salt and pepper noise; however, it does not simulate the actual shapes of marine snow artifacts. As a result, the center part of marine snow becomes excessively thick and the generated images are far from the real ones.

MSR is also an underrepresented problem in underwater image enhancement. Here, we briefly review some existing approaches.

A widely used method for MSR is a median filter (MF) [9, 19]. If marine snow artifacts can be assumed sufficiently small (typically $1$ – $3$ pixels in diameter), MF is expected to work because small marine snow artifacts can be considered to have a salt-and-pepper effect. However, it significantly blurs the entire image, particularly when we use a large filter kernel size.

Specific to MSR, a modified version of a MF is proposed in [5, 17]. This method applies the MF selectively if the target pixel has a higher intensity than the surrounding pixels. However, it is still difficult to remove large marine snow artifacts.

A few MSR methods for video sequences have also been proposed. They utilize the fact that marine snow artifacts continuously move in consecutive video frames. In [16], background modeling is used to remove marine snow artifacts from a static scene. Marine snow artifacts are tracked and a customized MF is applied to the detected artifacts in [13]. However, they are not applicable to a single underwater image. Importantly, all methods above are based on a model-based approach. This is mainly due to a lack of high-quality datasets, as mentioned in Section 1.

A deep learning-based MSR method is proposed in [18]. Its training dataset consists of synthesized marine snow images, corresponding ground truth images, marine snow masks. This dataset also contains real underwater images with marine snow (without ground truth). Marine snow particles are manually made with Adobe Photoshop: That results in a lack of randomness and scalability.

In the following, we address the contributions of this study; marine snow models and the dataset specifications.

3 Marine Snow Models

There are various sources of marine snow and it is impractical to estimate the sources of all particles from a single underwater image. Instead of the estimation of the sources, we model the pixel value distributions of marine snow artifacts from observations of underwater images.

3.1 Real Marine Snow Examples

First, we start by showing marine snow examples in real underwater images. Figs. 2(a) and (c) show enlarged portions of representative marine snow artifacts cropped from real underwater images. Although they look similar, their pixel value distributions are slightly different.

Taking a closer look, the 3D plots of Figs. 2(a) and (c) are shown in Figs. 2(b) and (d), respectively. As clearly observed, they do not have a shape like a Gaussian function in contrast to the conventional assumption in [8, 7]. Rather than a Gaussian function, these 3D plots are similar to elliptic conical frusta, i.e., sliced elliptic cones. Furthermore, the top surfaces of the frusta have different characteristics between Figs. 2(b) and (d). In our preliminary observation, most marine snow artifacts can be classified into these two representative shapes. In the following, we present the mathematical models of marine snow artifacts that reflect the above-mentioned observations.

3.2 Marine Snow Model 1: Highland Type

The first marine snow model is the Highland Type (type H). The type H marine snow corresponds to Fig. 2(a) and can be modeled as an elliptic conical frustum with a rough surface.

For modeling type H marine snow, we first suppose to have two ellipses $f_{1}$ and $f_{2}$ , both centered at the coordinates $(k,l)$ . As shown in Fig. 3, we assume the focal points of $f_{1}$ and $f_{2}$ are located on the horizontal axis, i.e., the ellipses are wider, for simplicity. Let $a_{i}$ and $b_{i}$ be the semi-major and semi-minor axes of $f_{i}$ where $i\in\{1,2\}$ , respectively. Suppose that $f_{1}$ is larger than $f_{2}$ , i.e., $a_{1}>a_{2}$ and $b_{1}>b_{2}$ and the eccentricities of $f_{1}$ and $f_{2}$ are identical, i.e., $\frac{a_{1}}{b_{1}}=\frac{a_{2}}{b_{2}}$ .

For notational simplicity, $(k,l)$ is set to $(0,0)$ hereafter. The $(m,n)$ th pixel value of the type H marine snow $h(m,n)$ is formulated as follows:

h(m,n)=\begin{cases}c+\epsilon&\frac{m^{2}}{a_{2}^{2}}+\frac{n^{2}}{b_{2}^{2}}\leq 1\\ c\frac{d(\mathbf{p},\mathbf{q}_{1})}{d(\mathbf{q}_{1},\mathbf{q}_{2})}+\epsilon&1<\frac{m^{2}}{a_{2}^{2}}+\frac{n^{2}}{b_{2}^{2}}\land\frac{m^{2}}{a_{1}^{2}}+\frac{n^{2}}{b_{1}^{2}}\leq 1\\ 0&\text{otherwise},\end{cases}

(1)

where $c$ is a constant that determines the transparency of the marine snow, $d(\cdot,\cdot)$ calculates the Euclidean distance between two points, and $\epsilon$ is a small perturbation that mimics a rough surface. Furthermore, $\mathbf{p}:=[m,n]^{\top}$ and $\mathbf{q}_{i}$ is the intersection of the ellipse $f_{i}$ and the straight line crossing $(m,n)$ , as illustrated in Fig. 3.

An example of the synthesized type H marine snow artifacts is shown in Fig. 4(a) along with its 3D plot in Fig. 4(b). See the similarity in Figs. 2(a) and (b). For creating a dataset, the major axis of $h(m,n)$ is randomly rotated and the transparency $c$ is also randomly chosen. These specifications are shown later in Section 4.

Table 1: Parameter settings for marine snow synthesis where

\mathcal{U}(v_{\min},v_{\max})

is a continuous uniform distribution between

v_{\min}

and

v_{\max}

$\dagger$ $b_{1}$ is obtained from the eccentricity of $f_{2}(m,n)$
Parameter	Setting
$c$ in (1)	$\mathcal{U}\{10,80\}^{\ast}$
$c_{r}$ in (2)	$\mathcal{U}\{1,40\}^{\ast}$
Gaussian filter size in (3)	$(2r+1)\times(2r+1)$ where $r\sim\mathcal{U}(1,6)$
$a_{2}$	$\begin{cases}\mathcal{U}(1,3)&\text{for small marine snow artifacts}\\ \mathcal{U}(3,16)&\text{for large marine snow artifacts}\end{cases}$
$b_{2}$	$\begin{cases}\mathcal{U}(0.3a_{2},a_{2})&\text{for small marine snow artifacts}\\ \mathcal{U}(0.8a_{2},a_{2})&\text{for large marine snow artifacts}\end{cases}$
$a_{1}$	$\mathcal{U}(a_{2},2a_{2})^{\dagger}$
$a_{3}$	$\mathcal{U}(0.5a_{2},a_{2})^{\ddagger}$
$\ddagger$ $b_{3}$ is obtained from the eccentricity of $f_{2}(m,n)$

3.3 Marine Snow Model 2: Volcanic Crater Type

The second marine snow model is the volcanic crater type (type V). An example of type V marine snow is shown in Figs. 2(c) and (d). This can be modeled by a modified version of the type H with an overshoot top edge of the frustum.

For the type V marine snow, we consider another ellipse $f_{3}$ , where $a_{3}$ and $b_{3}$ are its semi-major and semi-minor axes, respectively. It is located inside $f_{2}$ , as illustrated in Fig. 3. As for the type H, focal points of all ellipses are assumed to be located along the horizontal axis and their eccentricities are identical, i.e., $\frac{a_{1}}{b_{1}}=\frac{a_{2}}{b_{2}}=\frac{a_{3}}{b_{3}}$ .

Based on the type H function $h(m,n)$ , the $(m,n)$ th pixel value of the type V marine snow $v(m,n)$ is formulated as $v(m,n)=h(m,n)+g(m,n)$ , where $g(m,n)$ is the rim term defined as

\begin{split}&g(m,n)\\ &=\begin{cases}c_{r}(1\!-\!\frac{d(\mathbf{p},\frac{1}{2}(\mathbf{q}_{2}+\mathbf{q}_{3}))}{\frac{1}{2}d(\mathbf{q}_{2},\mathbf{q}_{3})})&\!\!1<\frac{m^{2}}{a_{3}^{2}}+\frac{n^{2}}{b_{3}^{2}}\land\frac{m^{2}}{a_{2}^{2}}+\frac{n^{2}}{b_{2}^{2}}\leq 1\\ 0&\text{otherwise,}\\ \end{cases}\end{split}

(2)

where $c_{r}$ is the constant for the maximum rim height and $\mathbf{q}_{3}$ indicates the coordinates of the intersection of the ellipse $f_{3}$ and the straight line crossing $(m,n)$ .

The synthesized type V marine snow artifact is shown in Figs. 4(c) and (d), which are similar to the real versions in Figs. 2(c) and (d).

3.4 Synthesizing Images with Marine Snow

By using the marine snow models introduced in the previous subsections, we synthesize the marine snow with underwater images.

Let $U\in\mathbb{R}^{H\times W\times 3}$ be an original underwater image. First, we randomly select a target pixel coordinate $(k,l)$ to append the marine snow. Second, we randomly select the major and minor axes of the ellipses $a_{i}$ and $b_{i}$ . In addition, the direction of the major axis is randomly set. Third, for each channel, type H or V marine snow is independently generated, where the parameters $c$ and $c_{r}$ are independently set. We denote an image with one marine snow artifact as $M$ , where pixel values in $M$ are all-zero except the marine snow artifact. Finally, $U$ and $M$ are combined as follows to yield the synthesized image $I$ :

I_{k,l,t}=\begin{cases}M_{k,l,t}+G_{k,l,t}&M_{k,l,t}\geq 0\\ U_{k,l,t}&\text{otherwise}\end{cases}

(3)

where $G\in\mathbb{R}^{H\times W\times 3}$ is the Gaussian filtered version of $U$ that reflects blur owing to marine snow.

We iterate the process until a predetermined number of marine snow artifacts are appended.

4 Dataset Specifications

In this section, we present the specifications of synthesized marine snow artifacts in the MSRB Dataset. It is designed for two tasks:

Task 1: : Removal of small-sized marine snow artifacts.
Task 2: : Removal of various-sized marine snow artifacts.

Clearly, Task 2 is more difficult than Task 1.

First, detailed parameters and setups shared in both tasks are presented. Second, we introduce MSR Tasks with corresponding synthesized images.

Original underwater images.

Synthesized images for MSR Task 1.

Synthesized images for MSR Task 2.

Figure 5: Example images in MSRB dataset.

Restoration results by MF.

Restoration results by adaptive MF.

Restoration results by U-Net.

Figure 6: MSR Task 1 results.

Restoration results by MF.

Restoration results by adaptive MF.

Restoration results by U-Net.

Figure 7: MSR Task 2 results.

4.1 General Setup

Each sub-dataset corresponding to an MSR Task contains $2,300$ training image pairs and $400$ test image pairs, all having a pixel resolution of $384\times 384$ . An image pair contains one original underwater image and one image containing synthesized marine snow artifacts. All original images are collected from flickr²²2https://www.flickr.com/ under a Creative Commons Attribution-NonCommercial-ShareAlike 2.0 Generic (CC BY-NC-SA 2.0) License.

Each synthesized image contains $N$ marine snow particles where $N$ is chosen from $\mathcal{U}\{100,600\}$ , in which $\mathcal{U}\{v_{\min},v_{\max}\}$ is a discrete uniform distribution between $v_{\min}$ and $v_{\max}$ . In each synthesized image, type H and V marine snow particles are randomly generated with a probability of $0.7$ for type H and $0.3$ for type V, according to our preliminary observations. The other representative parameters introduced in the previous section are listed in Table 1. Most of the parameters are determined according to our preliminary observations of real underwater images with marine snow artifacts.

4.2 MSR Task 1: Removing Small-Sized Marine Snow Artifacts

We introduce the first MSR Task: Removal of small-sized marine snow artifacts. In Task 1, the maximum width/height of the artifacts is restricted to $6$ pixels, which corresponds to roughly $1.6$ % of the image width/height. Task 1 is designed for underwater scenes where particles are relatively far from the photosensors. Because the task is relatively simple, the conventional MF approach is expected to be successfully applied (at least partially).

Examples of test image pairs in MSR Task 1 are shown in Fig. 5. As clearly visualized, the synthesized images have various sized (but small) marine snow artifacts.

4.3 MSR Task 2: Removing Various-Sized Marine Snow Artifacts

The second MSR task is designed for underwater scenes containing particles in various distances. As a result, the synthesized images for Task 2 have small- and large-sized marine snow artifacts. For large-sized artifacts, we set the largest width/height of marine snow to $32$ pixels, which corresponds to $8.3$ % compared to the image width. Furthermore, the probabilities of small- and large-sized artifacts are set to $0.7$ and $0.3$ , respectively.

Training image examples for Task 2 are also shown in Fig. 5. The images have various-sized artifacts and the degraded areas are larger than the images in Task 1.

In the following, we show the benchmarking results of MSR as well as the real MSR.

5 MSR Benchmarking Results

In this section, we present the first benchmarking performance for MSR Tasks 1 and 2. The methods used for the benchmarking are 1) MF [19], 2) adaptive MF [5], and 3) U-Net [29]. The kernel size of MFs is set to $3\times 3$ or $5\times 5$ pixels. Thanks to the MSRB Dataset, we can train a deep neural network for removing (synthesized) marine snow. For a benchmarking purpose, we train U-Net [29] using the dataset. The parameter settings are slightly different from the original.

The MSR results for Tasks 1 and 2 are shown in Figs. 6 and 7, respectively. As clearly observed, MF oversmoothes the images, which results in blur even for regions without marine snow artifacts. The adaptive MF suppresses blur; however, many artifacts are not removed mainly owing to the threshold-based algorithm. Furthermore, both MFs do not operate successfully for large marine snow artifacts in Task 2. By contrast, U-Net successfully removes marine snow for both tasks. Even for Task 2, it is able to suppress the large marine snow artifacts because of its multiresolution structure and the large number of trainable parameters.

For an objective comparison, we compute the average PSNRs and SSIMs over the test datasets. We summarize the results in Table 2. The objective measure indicates that U-Net is superior to the MFs for both MSR tasks. Note that all MFs present smaller PSNRs/SSIMs than those of the synthesized images because they filtered areas without marine snow. By contrast, U-Net achieves a better objective quality than the synthesized images, which implies the effect of a deep neural network for MSR.

Table 2: Average PSNRs and SSIMs over the test dataset.

	Task 1		Task 2
Method	PSNR	SSIM	PSNR	SSIM
MF (3 $\times$ 3)	28.55	0.846	22.81	0.770
MF (5 $\times$ 5)	25.98	0.711	21.93	0.645
Adaptive MF (3 $\times$ 3)	29.88	0.910	23.35	0.842
Adaptive MF (5 $\times$ 5)	28.08	0.861	22.83	0.794
U-Net	36.82	0.978	30.95	0.932
Synthesized image	32.20	0.945	23.83	0.876

6 Real Marine Snow Removal Examples

We construct the MSRB Dataset to mimic real marine snow artifacts. In this section, we show the real MSR results and present limitations of the dataset.

6.1 Real Marine Snow Removal

Fig. 8 shows MSR results for some real images. We use MF, adaptive MF, and U-Net as in the previous section. In addition, Deep WaveNet [30] is applied to validate the regular underwater enhancement model works for marine snow removal. U-Net is trained using MSRB Task 2 Dataset. The trained model of Deep WaveNet is taken from the original authors’ GitHub repository.

As clearly observed, MFs cannot suppress marine snow artifacts because of their various sizes and different transparencies. Deep WaveNet also fails to remove marine snow artifacts. That means marine snow cannot be suppressed by underwater image enhancement techniques alone. U-Net presents a better restoration image quality than MFs: It (partly) suppresses artifacts having both small and large sizes. This implies the effectiveness of deep-learning-based methods trained using the MSRB Dataset for real MSR.

Note that thick marine snow artifacts remain in the restored images. The reason for this could be two-fold: First, U-Net is not specifically designed for MSR although we slightly customized its structure. A neural network designed specifically for MSR would improve the performance. Second, our proposed model classifies marine snow artifacts into two representative types. Marine snow artifacts not fitted to these types are not well removed. This limitation is described in the next section.

6.2 Limitations

Because our MSR benchmarking dataset is based on synthesizing marine snow artificially into real underwater images, some limitations exist. First, the dataset is designed for removing marine snow artifacts and does not aim at achieving other underwater image enhancements such as color correction. The design of a simultaneous restoration of MSR and color correction is an interesting area of future study.

Although U-Net trained using the MSRB Dataset presents promising results, the dataset itself is imperfect. Specifically, real MSR performances are occasionally limited if MSR artifacts are not fitted to our marine snow models. Examples are shown in Fig. 9, which compares real underwater images and their restoration results using U-Net. Typically, U-Net fails to suppress artifacts when marine snow artifacts are larger and denser than our MSR benchmarking dataset. Moreover, if the lighting conditions differ from the dataset, the current version of U-Net will fail. Through a future study, this can be improved by carefully updating the MSRB Dataset to reflect various underwater scenes.

7 Conclusions

In this paper, the Marine Snow Removal Benchmarking Dataset was proposed. We mathematically modeled two representative marine snow artifacts and synthesized them in real underwater images. Two MSR tasks, the removal of small-sized and various-sized marine snow artifacts were also proposed. The first benchmarking results were also shown, which revealed the effectiveness of a deep neural network for MSR compared to median filter-based methods. MSR for real underwater images indicates that our MSRB Dataset contributes to real MSR problems. Our future studies will include improving the marine snow models as well as designing deep neural networks specifically designed for MSR.

References

[1] Derya Akkaynak and Tali Treibitz. A revised underwater image formation model. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6723–6732, 2018.
[2] Derya Akkaynak and Tali Treibitz. Sea-thru: A method for removing water from underwater images. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1682–1691, 2019.
[3] Saeed Anwar and Chongyi Li. Diving deeper into underwater image enhancement: A survey. Signal Processing: Image Communication, 89:115978, Nov. 2020.
[4] Saeed Anwar, Chongyi Li, and Fatih Porikli. Deep Underwater Image Enhancement. arXiv:1807.03528, July 2018.
[5] Soma Banerjee, Gautam Sanyal, Shatadal Ghosh, Ranjit Ray, and Nath Shome, Shome. Elimination of Marine Snow effect from underwater image - An adaptive probabilistic approach. In 2014 IEEE Students’ Conference on Electrical, Electronics and Computer Science, pages 1–4, Mar. 2014.
[6] Dana Berman, Tali Treibitz, and Shai Avidan. Diving into haze-lines: Color restoration of underwater images. In Proc. British Machine Vision Conference (BMVC), volume 1, 2017.
[7] Matthieu Boffety and Frédéric Galland. Phenomenological marine snow model for optical underwater image simulation: applications to color restoration. In 2012 Oceans-Yeosu, pages 1–6, 2012.
[8] Matthieu Boffety, Frédéric Galland, and Anne-Gaëlle Allais. Color image simulation for underwater optics. Applied optics, 51(23):5633–5642, 2012.
[9] David RK Brownrigg. The weighted median filter. Communications of the ACM, 27(8):807–818, 1984.
[10] Stanley H. Chan, Xiran Wang, and Omar A. Elgendy. Plug-and-play admm for image restoration: Fixed-point convergence and applications. IEEE Transactions on Computational Imaging, 3(1):84–98, 2016.
[11] Chang Chen, Zhiwei Xiong, Xinmei Tian, and Feng Wu. Deep boosting for image denoising. In Proceedings of the European Conference on Computer Vision (ECCV), pages 3–18, 2018.
[12] Xueyun Chen, Shiming Xiang, Cheng-Lin Liu, and Chun-Hong Pan. Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geoscience and remote sensing letters, 11(10):1797–1801, 2014.
[13] Boguslaw Cyganek and Karol Gongola. Real-time marine snow noise removal from underwater video sequences. Journal of Electronic Imaging, 27(4):043002, July 2018.
[14] Akshay Dudhane, Praful Hambarde, Prashant Patil, and Subrahmanyam Murala. Deep underwater image restoration and beyond. IEEE Signal Processing Letters, 27:675–679, 2020.
[15] Cameron Fabbri, Md Jahidul Islam, and Junaed Sattar. Enhancing Underwater Imagery Using Generative Adversarial Networks. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 7159–7165, May 2018.
[16] Fahimeh Farhadifard, Martin Radolko, and Uwe Freiherr von Lukas. Marine Snow Detection and Removal: Underwater Image Restoration Using Background Modeling. Václav Skala - UNION Agency, 2017.
[17] Fahimeh Farhadifard, Martin Radolko, and Uwe von Lukas. Single image marine snow removal based on a supervised median filtering scheme. In 12th International Conference on Computer Vision Theory and Applications, VISIGRAPP 2017. Proceedings. Vol.4: VISAPP, pages 280–287, 2017.
[18] Dongsheng Guo, Yiqing Huang, Tianshun Han, Haiyong Zheng, Zhaorui Gu, and Bing Zheng. Marine snow removal. In OCEANS 2022 - Chennai, pages 1–7, 2022.
[19] Thomas Huang, GJTGY Yang, and Greory Tang. A fast two-dimensional median filtering algorithm. IEEE Transactions on Acoustics, Speech, and Signal Processing, 27(1):13–18, 1979.
[20] Qin Jiang, Yang Chen, Guoyu Wang, and Tingting Ji. A novel deep neural network for noise removal from underwater image. Signal Processing: Image Communication, 87:115921, 2020.
[21] Michał Koziarski and Bogusław Cyganek. Marine Snow Removal Using a Fully Convolutional 3D Neural Network Combined with an Adaptive Median Filter. In Pattern Recognition and Information Forensics, pages 16–25, 2019.
[22] Chongyi Li, Saeed Anwar, Junhui Hou, Runmin Cong, Chunle Guo, and Wenqi Ren. Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Transactions on Image Processing, 30:4985–5000, 2021.
[23] Chongyi Li, Saeed Anwar, and Fatih Porikli. Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recognition, 98:107038, 2020.
[24] Chongyi Li, Chunle Guo, Wenqi Ren, Runmin Cong, Junhui Hou, Sam Kwong, and Dacheng Tao. An underwater image enhancement benchmark dataset and beyond. IEEE Transactions on Image Processing, 29:4376–4389, 2020.
[25] Jie Li, Katherine A. Skinner, Ryan M. Eustice, and Matthew Johnson-Roberson. WaterGAN: Unsupervised Generative Network to Enable Real-Time Color Correction of Monocular Underwater Images. IEEE Robotics and Automation Letters, 3(1):387–394, Jan. 2018.
[26] Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I Sánchez. A survey on deep learning in medical image analysis. Medical image analysis, 42:60–88, 2017.
[27] Emmanuel Maggiori, Yuliya Tarabalka, Guillaume Charpiat, and Pierre Alliez. Convolutional neural networks for large-scale remote-sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 55(2):645–657, 2016.
[28] Yaniv Romano, Michael Elad, and Peyman Milanfar. The little engine that could: Regularization by denoising (red). SIAM Journal on Imaging Sciences, 10(4):1804–1844, 2017.
[29] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015.
[30] Prasen Sharma, Ira Bisht, and Arijit Sur. Wavelength-based attributed deep neural network for underwater image restoration. ACM Transactions on Multimedia Computing, Communications and Applications, 19(1):1–23, 2023.
[31] Dinggang Shen, Guorong Wu, and Heung-Il Suk. Deep learning in medical image analysis. Annual review of biomedical engineering, 19:221–248, 2017.
[32] Takumi Ueda, Koki Yamada, and Yuichi Tanaka. Underwater image synthesis from rgb-d images and its application to deep underwater image restoration. In Proc. 2019 IEEE International Conference on Image Processing (ICIP), pages 2115–2119. IEEE, 2019.
[33] Yang Wang, Jing Zhang, Yang Cao, and Zengfu Wang. A deep CNN method for underwater image enhancement. In 2017 IEEE International Conference on Image Processing (ICIP), pages 1382–1386, Sept. 2017.
[34] Martin Weigert, Uwe Schmidt, Tobias Boothe, Andreas Müller, Alexandr Dibrov, Akanksha Jain, Benjamin Wilhelm, Deborah Schmidt, Coleman Broaddus, Siân Culley, et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nature methods, 15(12):1090–1097, 2018.
[35] Junyuan Xie, Linli Xu, and Enhong Chen. Image denoising and inpainting with deep neural networks. Advances in neural information processing systems, 25:341–349, 2012.
[36] Li Xu, Jimmy SJ Ren, Ce Liu, and Jiaya Jia. Deep convolutional neural network for image deconvolution. In Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 1, pages 1790–1798, 2014.
[37] Raymond A. Yeh, Chen Chen, Teck Yian Lim, Alexander G Schwing, Mark Hasegawa-Johnson, and Minh N. Do. Semantic image inpainting with deep generative models. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5485–5493, 2017.
[38] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, July 2017.
[39] Liangpei Zhang, Lefei Zhang, and Bo Du. Deep learning for remote sensing data: A technical tutorial on the state of the art. IEEE Geoscience and Remote Sensing Magazine, 4(2):22–40, 2016.
[40] Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, Apr. 2004.
[41] Xiao Xiang Zhu, Devis Tuia, Lichao Mou, Gui-Song Xia, Liangpei Zhang, Feng Xu, and Friedrich Fraundorfer. Deep learning in remote sensing: A comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine, 5(4):8–36, 2017.
[42] Peixian Zhuang, Jiamin Wu, Fatih Porikli, and Chongyi Li. Underwater image enhancement with hyper-laplacian reflectance priors. IEEE Transactions on Image Processing, 31:5442–5455, 2022.