Classification of cosmic structures for galaxies with deep learning: connecting cosmological simulations with observations

To feed our model in Section 3.3, we create data of three-dimensional cubic regions of galaxy distribution tagged with the labels of the cosmic structures. We set the side length of all cubic data to be $10~{}{\rm Mpc}$, and a cube is binned with $16^{3}$ voxels whose size is $625~{}{\rm kpc}$. Each cube is centred on its ‘classification point’ assigned with the labels obtained in Section 2: knot, sheet, filament or void. A position of the classification point is selected at random from the grid points in the case of the grid-based classification, whereas it is sampled from the galaxies in the galaxy-based classification. Next, we rotate all galaxies three-dimensionally around the normal, transverse and longitudinal axes at random while fixing the classification point. Then, galaxies in the cubic region are placed in the voxels according to their positions. In the galaxy-based classification, the galaxy selected as the classification point is also included in the data. Here we do not apply the could-in-cell algorithm or any other smoothing schemes since we find that applying such a smoothing does not improve the performance of our models. Finally, we tag the cubic data with the class label at the classification point. Fig. 4 illustrates the above procedures schematically.

The side length of the cubic region, $10~{}{\rm Mpc}$, is set arbitrarily. We find, however, that our results hardly change in the range from $\sim 10$ to $20~{}{\rm Mpc}$ and become less accurate outside this range. The number of voxels in the cube, $16^{3}$, is also an arbitrary choice; however we confirm that this little affects the accuracy of our models between $8^{3}$ and $32^{3}$.

Since the grid- and galaxy-based classification are different in the schemes to sample the classification points, the data produced by the two schemes are not qualitatively homogeneous. Because the distribution of galaxies is biased towards high-density environments, a single cubic region in the galaxy-based classification generally includes more galaxies than that in the grid-based classification even if their labels are the same. The three-dimensional extension of galaxy distribution could also be different between them. We argue the influence on our 3D-CNN models by these different sampling schemes in Section 4.2.

By repeating the above procedures, we create 10000 cubic data for each label. The classification points are randomly sampled with replacement. Although the same classification points can be selected again, we randomly change the orientation of the cube every time. In this study, the cubic data consist of a single channel: the number of galaxies, $N_{\rm voxel}$, in each voxel. We scale $N_{\rm voxel}$ linearly in the data. Even when we scale the data logarithmically, the performance of our models hardly changes.⁵⁵5We give the voxel $\log(N_{\rm voxel}+1)$ in the logarithmic case. We discuss the influence by including other physical quantities as additional channels in Section 5.2. For preprocessing the data, we apply the Min-Max normalisation. The maximum value is computed among all voxels in all cubes, and then all voxel values are normalised into $[0,1]$. In the 10000 data cubes for each label, 6400, 1600 and 2000 are used as training, validation and test data. Even if we generate four times more data sets, the performance of our models is not improved.

We expect that the differences in three-dimensional spatial distribution of ambient galaxies would have key information in deducing the cosmic structures. We, therefore, consider three-dimensional neural networks (3D-CNN) to be a feasible classifier of the cosmic structures. Such a classifier would be able to detect and distinguish their diffuse (voids), planar (sheets), filamentary (filaments) and concentrated (knots) distribution of galaxies.

We implement 3D-CNN using the Keras library on TensorFlow. Our network architecture is shown in Table 1. We adopt the Adamax optimiser (Kingma & Ba, 2014) with a learning rate of $0.002$ without decaying and the parameters of $\beta_{1}=0.9$, $\beta_{2}=0.999$ and $\varepsilon=10^{-7}$. The categorical cross-entropy loss function is used. We set a mini-batch size to $512$ and the number of learning epochs $N_{\rm epoch}$ to $30$. We use the model trained at the last epoch $N_{\rm epoch}=30$ to evaluate the resultant accuracy of the model using the test data; the performance of our models does not significantly depend on the epoch when we stop the training as long as $N_{\rm epoch}\gtrsim 10$ (see Section 4).

We find that such a simple 3D-CNN architecture with only two convolutional layers works well enough without significant overfitting in this study (see below), and building deeper networks can result in a tendency of instability on loss values as well as overfitting. Although we have tested various architectures, hyper-parameters, other machine/deep-learning libraries and ways of preprocessing, such alteration for our model only changes the total accuracy by $\lesssim 0.03$ (see Section 5.2). Hence, we do not change the architecture or hyper-parameters throughout this paper unless otherwise stated.

We first examine the intrinsic performance of our 3D-CNN models. For this purpose, we present our results for the grid-based classification and compare them with a previous study. We here randomly select the classification points of the cubic data from the $256^{3}$ grid points, and number densities in the cubes are computed from the halos containing stellar particles. Fig. 5 tabulates the normalised confusion matrix and $F_{\rm 1}$-scores in the case of classification into the four categories. The $F_{\rm 1}$-score is equivalent to Dice coefficient in the case of binary classification, defined as

where $N_{\rm tp}$, $N_{\rm fp}$ and $N_{\rm fn}$ are the numbers of true positive, false positive and false negative classification. In this study, we evaluate the performance of our models with the $F_{\rm 1}$-scores. As shown in Fig. 5, the model can achieve relatively high accuracy for voids and knots with the $F_{\rm 1}=0.83$ and $0.84$. On the other hand, classifying sheets and filaments are less accurate with the lower $F_{\rm 1}$-scores of $0.63$ and $0.67$. In the two classes, $20.2$ and $17.9$ per cent of the sheet voxels are erroneously classified as filaments and voids, and $23.4$ and $11.4$ per cent of the filament voxels are mistaken for knots and sheets. This difficulty in distinguishing sheets and filaments is probably because of their intermediate densities and complex spatial extension (see also Section 5.1). However, the macro-average of the $F_{\rm 1}$-scores is $0.74$,⁶⁶6In all computations in this paper, the macro-averages of $F_{\rm 1}$-scores are little different from the micro-averages. The macro-average is defined as the mean of $F_{\rm 1}$ computed in each class, whereas the micro-average is defined as $F_{\rm 1}$ computed with the means of $N_{\rm tp}$, $N_{\rm fp}$ and $N_{\rm fn}$ among the classes. which means that our model is not useless although we need to be aware of the accuracy according to actual applications of this method.

For the above quaternary classification, Fig. 6 indicates the macro-averages of $F_{\rm 1}$-scores and loss values as functions of learning epoch $N_{\rm epoch}$. The $F_{\rm 1}$-scores and loss values almost converge after the first few learning epochs. Since the $F_{\rm 1}$-scores (loss values) of the validation data are only slightly lower (higher) than those of the training data at later epochs of $N_{\rm epoch}\gtrsim 15$, the 3D-CNN classifier shows no sign of significant overfitting.

Fig. 7 shows the map of predicted labels on the same slice of Fig. 1. Unlike the distribution of the true labels, the boundary of the structures are blurred. This is probably because our cubic data have the side length of $10~{}{\rm Mpc}$, and this length limits the spatial resolution of the prediction. As we mention in Section 3.1, however, decreasing the side length does not improve the accuracy of the prediction. Over-prediction of knot and filament regions are significant, and the knot regions are quite large in the prediction (see also Figs. 12 and 16 in Section 4.3).

Aragon-Calvo (2019) has performed similar cosmic-structure classification with 3D-CNN on a U-Net architecture (e.g. Ronneberger et al., 2015) although his deep-learning model learns DM density fields in large cosmological volumes and simultaneously classifies all spatial voxels therein. In terms of classifying spatial grid points, the situation of his study corresponds to the grid-based classification in ours. His study is, however, based on a binary classification where he ignores voids and merges knots with filaments. The largest difference from ours is that his model predicts the class labels from the DM density fields instead of galaxies. The performance of his model is $F_{\rm 1}=0.78$ and $0.72$ for sheets and filaments/knots. Fig. 8 shows our result for the same binary classification where voids are ignored and knots are merged with filaments. Specifically, we randomly create 10000 data cubes centred on the sheet voxels and another set of 10000 cubes on the filament and knot voxels without distinguishing between the two. In our result shown in Fig. 8, the $F_{\rm 1}$-scores are $0.84$ and $0.83$ for sheets and filaments/knots, and the performance of our model is thus comparable with that of Aragon-Calvo (2019).⁷⁷7Although our simple model may be more accurate than that of Aragon-Calvo (2019) in $F_{\rm 1}$-scores, the accuracy can depend on the resolutions of a simulation and the analysis of DM for the labelling. We use the tensors of velocity gradients (equation 1) to categorise the structures, whereas he does the second-order derivative of DM density fields. This result demonstrates that galaxy distribution can be a substitution for DM density fields in predicting the cosmic structures with the models. It can, therefore, be feasible to classify the cosmic structures in the real Universe by using wide-field survey observations for galaxies such as SDSS. In addition, the above result proves that our method with the simple 3D-CNN classifier trained with the cubic data of galaxy distribution works as accurate as the U-Net architecture learning DM density fields in the previous study.

We stress, however, that the above models in Section 4.1 cannot classify galaxies since they are trained with the grid-based data. The grid-based classification uniformly samples spatial regions for each class label, whereas galaxies distribute with a bias towards high-density regions. For example, few galaxies are found in the middle of void regions, and most of the galaxies labelled as voids reside around boundaries of void regions which are close to sheets. Accordingly, in the galaxy-based classification where the data cubes are centred on galaxies, the cubic regions centred on the void galaxies are generally similar to sheet regions. Sheet and filament galaxies are also subject to the same bias since most of them are found in regions close to filament and knot regions, respectively. To demonstrate the influence of this bias, we classify the galaxy-based data cubes with the model trained for the grid-based classification that is used for Fig. 5. The galaxy-based data cubes are centred on galaxies selected randomly, and we create a set of 10000 data for each label. Fig. 9 shows the result, where the predictions are significantly biased towards contiguous classes with higher densities except for knots. In comparison with Fig. 5, the $F_{\rm 1}$-scores are significantly lower in Fig. 9. For knot galaxies, although the true positive fraction is quite high with $98.0$ per cent, the false positive predictions to knots from sheets and filaments also largely increase to $27.8$ and $69.6$ per cent. As a result, the $F_{\rm 1}$-score of knots becomes lower as well as the other classes. The macro-averaged $F_{\rm 1}$-score among the four classes is $0.49$. Thus, we cannot adopt the model trained with the grid-based classification to galaxies. This implies that it is difficult to classify spatial regions close to boundaries between contiguous classes.

The above models trained with the grid-based data can, however, be useful for other purposes such as classifying spatial regions. For example, it can define extents of void regions and lengths of filament structures in the real Universe. Especially, the size of ‘the super void’ and statistics of the filament lengths are used as tests of the models of cosmology (e.g. Inoue & Silk, 2006, 2007; Sousbie et al., 2008). A point worthy to mention here is that our methods use observable galaxies to classify the cosmic structures, but the class labels are obtained from velocity gradients of unobservable DM (equation 1). This is qualitatively different from the previous classification methods described in Section 1.

Our models taking into account the observational restrictions can be directly applied to observations. Using the models obtained for Fig. 14, we here classify observed galaxies. From the galaxy catalogue of SDSS DR12 for the north side ($100\lesssim$ RA $\lesssim 300~{}{\rm degree}$), we extract galaxies and quasars⁹⁹9Hereafter, we do not distinguish quasars from galaxies in the SDSS data. that have no warning flags for their redshift measurements, and convert their redshifts $z$ to distances $d_{\rm obs}$ by assuming the uniform expansion of the local Universe. We do not take into account uncertainties of the redshift measurements since these are typically $\Delta z/z\sim 10^{-4}$. We use galaxies within a range of $d_{\rm obs}=85$–$115~{}{\rm Mpc}$ and create data cubes whose classification points are centred on all the galaxies within $d_{\rm obs}=90$–$110~{}{\rm Mpc}$. In this narrow distance range, the limits on absolute magnitude are nearly constant with the small variation from $M_{r}=-17.02$ to $-17.46~{}{\rm mag}$. We can, therefore, consider the SDSS sample to be approximately volume-limited, and we use the models built by assuming the simulated galaxies to be at $100~{}{\rm Mpc}$ in Section 4.3.2. Our models can, however, classify galaxies at other distances by changing the absolute magnitude limit $M_{r}$ depending on the distances and retraining the models as long as the distance is not too large. We do not classify galaxies close to the boundaries of the survey area since our models cannot treat such galaxies if their data cubes overlap with the boundaries.

Fig. 18 shows the predicted class labels for the SDSS galaxies within $d_{\rm obs}=90$–$110~{}{\rm Mpc}$. In the quaternary classification (top left), we find $628$ ($9.1$), $1895$ ($27.6$), $2613$ ($38.1$) and $1730$ ($25.2$) galaxies (per cent) in the classes of void, sheet, filament and knot in the plotted region (RA$=130$–$230^{\circ}$ and DEC$=10$–$55^{\circ}$). The galaxies classified as filaments distribute along the structures resembling a web studded with compact groups of the knot galaxies. The sheet galaxies are found around the filaments, and the voids reside in the diffuse inter-filament regions. These features appear to agree with the intuitive recognition of the cosmic structures. Meanwhile, it should be reminded that the models cannot classify sheet and filament galaxies very accurately as shown in the top panel of Fig. 14, and the $F_{\rm 1}$-scores are $0.46$ and $0.48$ for sheet and filament galaxies in the quaternary classification. From the analogy of Figs. 14, 16 and 17, a significant fraction of the galaxies identified as knots can be misclassification of (true) sheets.

In the cases of the ternary (the right panel in Fig. 18) and binary (bottom left) classification, the merged classes are plotted in black. In the ternary classification, $548$ ($8.0$), $1788$ ($26.0$) and $4530$ ($66.0$) galaxies (per cent) are classified as voids, sheets and filaments/knots. The fractions of void and sheet galaxies hardly change from the quaternary classification. In the binary classification, $927$ ($13.5$) and $5939$ ($86.5$) galaxies (per cent) are classified as voids, sheets/filaments/knots. Although the fraction of voids is somewhat higher than those in the cases of the quaternary and ternary classification, the binary case is expected to be more accurate and credible for the voids because of the higher $F_{\rm 1}$-score (Fig. 14).

Here, we compare the accuracy of our 3D-CNN model with that of a simple method not using the deep learning. In the four cosmic-structure classes, local densities of galaxies are expected to typically increase from void, sheet, filament to knot regions. A local density around a galaxy may therefore characterise the cosmic structure to which the galaxy belongs. As is done in Section 4.3.2, we exclude galaxies fainter than $M_{r}=-17.25~{}{\rm mag}$ from the TNG100-1 simulation and shift positions of the galaxies according to their $v_{\rm los}$ by equation (3). We then compute a distance from a galaxy to its fifth nearest neighbour, $r_{5}$. We calculate a local number density of galaxies as $n_{5}=5/(4\pi r_{5}^{3}/3)$ for every single galaxy with $M_{r}<-17.25~{}{\rm mag}$ in the simulation, compute a global number density $\phi$ of galaxies with a given $n_{5}$ in the entire simulation. The top panel of Fig. 19 shows the distribution of $\phi$ as a function of $n_{5}$ for galaxies in each (true) class label. As we expect above, the median values of $n_{5}$ (the vertical dashed lines) increase from voids to knots in the order. It is worthy to mention, however, that their ranges between $\pm 1\sigma$ of $n_{5}$ (the thick parts of the solid lines) significantly overlap with each other, especially those of sheets, filaments and knots.¹⁰¹⁰10Similar arguments have been presented in Hoffman et al. (2012) for DM density. We find that the distance measurement errors by $v_{\rm los}$ can significantly lower $n_{5}$ of galaxies in dense environments.

In the bottom panel of Fig. 19, we normalise the functions of $\phi$ to make their integrals equal to unity. The vertical solid lines indicate the values of $n_{5}$ at which the normalised distribution functions $\tilde{\phi}$ become equal between contiguous classes. We consider these values of $n_{5}$ to be the thresholds between the pairs of contiguous classes, which give the galaxies their predicted labels as illustrated in the bottom panel of Fig. 19. Thus, in this simple analysis, each galaxy has its true label determined by the DM analysis (equation 1) and the predicted label given by the local density $n_{5}$. A difference from the 3D-CNN models is that this prediction based on $n_{5}$ does not take into account the shapes of three-dimensional distribution of galaxies.

Fig. 20 shows the normalised confusion matrix of the above classification. In comparison with the 3D-CNN model (the top panel of Fig. 14), the $F_{\rm 1}$-scores decrease in all classes, and their macro-average is $0.54$. Especially, the misclassification of filaments as knots ($38.2$ per cent) is more than the true positive classification of filaments ($36.7$ per cent). However, the decrease of the $F_{\rm 1}$-scores is relatively small in the voids and knots. Although we define $n_{5}$ at the fifth nearest neighbour, the result of Fig. 20 does not significantly depend on the number of the nearest galaxies to define the local density. If predicted labels are given completely at random, quaternary classification results in $F_{\rm 1}=0.25$ for all classes. So, in this sense, we consider that the classification based on $n_{5}$ is not too inaccurate. Although the local density of galaxies is an essential quantity to characterise the cosmic structures, the 3D-CNN model can improve the accuracy by taking into account the three-dimensional extension of galaxy distribution, especially for sheets and filaments.¹¹¹¹11If the observational restrictions are ignored, we find that the simple classification method based on the local density $n_{5}$ does not classify the simulated galaxies accurately. This is because a large number of faint galaxies are taken into account in this analysis due to the absence of the limiting magnitude, and the number density of galaxies increases generally. Since most of the faint galaxies are satellites, the distances to the fifth nearest neighbours $r_{5}$ can become so short that the fifth neighbours can be included in the systems hosting the galaxies (i.e. $r_{5}\sim 10$–$100~{}{\rm kpc}$). Since the local density $n_{5}$ is strongly affected by their host systems in this case, $n_{\rm 5}$ cannot characterise their environments of the cosmic structures on the scale of $\sim 10~{}{\rm Mpc}$. Even if we use the fiftieth nearest neighbours $n_{50}$, the simple classification method is still significantly inaccurate.

a large number of faint galaxies are taken into account in this analysis, and the number density of galaxies increases generally. Since most of the faint galaxies would be satellites, the distances to the fifth nearest neighbours $r_{5}$ can become significantly shorter.

We have tested various types of CNN architectures. Two-dimensional CNN using projected images of the cubic data are significantly inaccurate. More sophisticated 3D-CNN on neither Residual Networks (ResNet, He et al., 2015) nor U-Net based classification models improve the accuracy significantly. We finally arrived at the simple 3D-CNN classifier described in Section 3.3.

In the cosmological simulation, there are other quantities available in observations besides the number of galaxies, such as stellar mass and colour. As the second and third channels of our cubic data, we add the total stellar mass and $g-i$ colours of galaxies in each voxel. However, we find that these additional channels do not change the performance of our models. Although we have also tested with the higher-resolution simulation of TNG50-1, the results are similar to those with TNG100-1 (see Appendix B).

The method of this study assumes the cosmological simulation to be accurate and statistically compatible with the observed galaxies. The details of baryon components can depend on the sub-grid models in the simulation. However, since our fiducial models only use the spatial distribution of galaxies above the limiting magnitude, our results are expected to be less dependent on such physical models for baryons. Our results can also depend on the accuracy of the cosmic-structure classification to obtain the true labels. In Section 2, we have used the method of Hoffman et al. (2012) with the threshold of $\lambda_{\rm th}=0.44$. If this method and/or the threshold is inaccurate, our models cannot correctly predict the labels by the mislabelling. In addition, the eigenvalues of the velocity-gradient tensors (equation 1) can vary continuously between the classes. If it is the case, the class labels given to a certain fraction of galaxies would be ambiguous. However, there are no methods to evaluate quantitatively the accuracy of the true labels. In this study, we consider that the DM analysis of Hoffman et al. (2012) presents the ‘definitions’ of the cosmic structures. The detailed discussion of the cosmic-structure classification is beyond the scope of this study although we show further analysis for the true labels in Appendix C.

Observations would be improved in the future. In Section 4.3.2, we have considered the two observational restrictions: limiting magnitude and distance measurement error by a proper motion. The former can be mitigated if deeper spectroscopic surveys become available although the latter is unavoidable as long as distances are measured from recession velocities of galaxies. Fig. 21 shows the confusion matrix of the galaxy-based classification, where the limiting magnitude is ignored by taking into account all the subhaloes that contain stellar particles whereas the distance errors are included. In comparison with the result including both restrictions (the top panel of Fig. 14), the performance of the model is a little improved in Fig. 21, and the macro-averaged $F_{\rm 1}$-score is $0.64$. This $F_{\rm 1}$-score is similar to that in the case including neither restriction (the top panel of Fig. 10). Thus, although excluding faint galaxies by the limiting magnitude lowers the accuracy of the models, it can be improved if the limiting magnitude is mitigated in future observations. In addition, we evaluate the influence by the distance errors due to $v_{\rm los}$ by comparing Fig. 21 with the top panels of Fig. 10. Although the model for Fig. 21 includes the effect of $v_{\rm los}$, the $F_{\rm 1}$-scores are similar to those in the top panels of Fig. 10, and the differences are within the fluctuation shown in Table 2 (Appendix D). This may imply that the 3D-CNN model learns the pattern of the redshift distortions by $v_{\rm los}$ in cluster regions, and the model can adjust the prediction to the distance errors. Thus, the distance errors do not make the models inaccurate.