Hyperspectral Image Dataset for Individual Penguin Identification

Remote individual animal identification is an important task that allows researchers to understand animal behavior and study population parameters such as population size, movement patterns, etc. There are invasive and non-invasive methods for individual animal identification. Invasive methods require capturing an animal and physical tagging, which is costly and likely to cause stress. Non-invasive methods use biometric traits such as DNA collected from hair or feces or visual assessment based on images captured in inhabited regions. DNA collection and analysis are costly and sometimes infeasible because they require going to dangerous areas to acquire the samples. Therefore, visual assessment and animal biometrics using images are highly demanded.

In this paper, we focus on individual penguin identification based on hyperspectral (HS) images. Existing image-based animal identification methods [1] [2] [3] rely on the spatial information of specific parts of the animal, such as the belly pattern of a penguin or the face of a panda. Those existing image-based methods are infeasible to use for the small size of a pixel of the target. Then, we use HS data of the single pixel. We assume an application with HS images as shown in Fig. 1. First, a single pixel in the target penguin is selected from the given HS image. Then, the machine learning model classifies penguin individuals. We analyze HS data of a single pixel because the pixel size of the target penguin in remote sensing is often too small to analyze spatial information.

To the best of our knowledge, it is the first work to analyze spectral differences between penguin individuals using an HS camera. There are existing studies identifying deep-sea megafauna using HS image [4], however, this work focused on the seafloor and did not capture the individuals. HS data has also been used in a limited number of studies to differentiate amphibian and reptile species [5] [6] [7] and insects [8], these studies only analyzed differences between species, not individuals. Kolmann et.al. distinguishing HS data from $47$ serrasalmid species also documented interspecific variation in pacus that corresponds to cryptic lineages [9].

In this paper, we tackle individual penguin identification using HS data of the single pixel. For this task, we constructed a novel dataset. This dataset comprises $990$ HS images, annotated with $27$ different penguins. Machine learning is used for the identification to evaluate the effectiveness of using HS data. In the experiment, as shown in Fig. 1, pixels of the penguins are selected from the HS images, and the selected spectral data are classified into individuals using a machine learning model.

Our contributions are twofold: First, we examined individual penguin identification using HS data and demonstrated its effectiveness. Second, we constructed a novel dataset for individual penguin identification composed of HS images ¹¹1Our dataset is available here: https://huggingface.co/datasets/dekkaiinu/hyper_penguin.

We collected $990$ HS images of $27$ African penguins at Ueno Zoological Gardens [10]. For collecting HS images, we used the HS camera, which can capture the wave range of $350-1100$[nm], with $151$ bands and a spatial resolution of $5$[nm] [11]. The image size is $2048\times 1080$ pixels. The distance between the HS camera and the penguins was approximately 3 to 6 meters, with the camera positioned at a height of 1.2 meters, as illustrated in Fig. 2. We collected HS penguin images outside, as shown in Fig. 3. Additionally, the camera angle was adjusted for each image to ensure that the target penguins were captured in the frame. We captured the HS images in such a manner that a group of penguins is depicted in a single frame. Each image includes 1 to 6 penguins. Figure 4 shows an example of images where an RGB image was converted from an HS image.

This dataset is annotated in two ways. The first annotation includes individual IDs of penguins at the pixel level for identifying individuals from (spectral data in) all HS images. The second consists of bounding box annotations marked with individual IDs of penguins, which are used for detecting penguins within the images. Those two ways of annotation allow us to analyze a variety of tasks.

We built a learning-based network model for pixel-wise individual penguin identification from the HS data of the single pixel. The network model is a simple $5$-layer multi-layer perceptron (MLP) model with batch normalization, rectified linear unit (ReLU), dropout, and softmax. For the denoising purpose, we applied a $5\times 5$ sized spatial box filter as preprocessing. Then, we feed the complete information of the single pixel’s HS data directly, while the dimension reduction by a principal component analysis (PCA) is recommended in some remote sensing research with HS data [12, 13].

We conducted the pixel-wise individual penguin identification based on the HS data. We compare three different input data: RGB data, compressed HS data by the PCA, and the proposed complete HS data. The RGB data was synthesized from the HS data assuming standard RGB color space. For the compressed HS data, we apply the dimension reduction by the PCA. The number of components was set to five, following existing papers [12, 13].

We visualize the differences in spectral among penguin individuals to discuss our experimental results. Figure 7 represents the average spectral of penguins captured in HS images taken over specified one-hour intervals, with each spectral corresponding to a different individual. Figure 7 covers images from $10$ am on June $23$rd for one hour, Figure 7 from $2$ pm on the same day, and Figure 7 from $12$ pm on June $27$th for one hour. The plots depict the average values of the HS data obtained from the white parts of each penguin’s body, representing the individual’s spectral.

To minimize the influence of sunlight conditions, we visualized the HS data by dividing them into hourly intervals in Figure 7. We captured our dataset over one day to obtain the data with various variations since HS data is highly sensitive to sunlight conditions. The strongest spectral intensity, evident in Figure 7, corresponds to the time of day when sunlight is most intense.

The spectral shape of each individual is different for all intervals in Figure 7, so the average accuracy was high (shown in Table 1). Further, the distinctive spectral of the penguin ID$5$ in Figure 7 and Figure 7 seem to contribute to the highest classification accuracy of $94.6$ [%]. Thus, the results of the spectral plots demonstrate the effectiveness of HS data in individual identification.

This paper has presented the individual penguin identification based on HS data of the single pixel. For that purpose, first, we built the penguin HS image dataset with annotation. Then, we experimentally demonstrated that we can identify individual penguins with HS data of the single pixel by the simple MLP network. In future work, we will propose a novel data augmentation method for penguin identification, and we will try to recognize individual animals other than penguins.

We thank and honor the Ueno Zoological Gardens [10] for allowing us to capture HS images of African penguins for the future of image processing research.