Elastic Coupled Co-clustering for Single-Cell Genomic Data

Clustering aims at grouping a set of objects such that objects in the same cluster are more similar to each other compared to those in other clusters. It has wide applications in many areas, including genomics, where single-cell sequencing technologies have recently been developed. For the analysis of single-cell genomic data, most clustering methods are focused on one data type: SIMLR (Wang et al.,, 2017), SC3 (Kiselev et al.,, 2017), DIMM-SC (Sun et al.,, 2017), SAFE-clustering (Yang et al.,, 2018) and SOUP (Zhu et al.,, 2019) are developed for scRNA-seq data, and chromVAR (Schep et al.,, 2017), scABC (Zamanighomi et al.,, 2018), SCALE (Xiong et al.,, 2019) and cisTopic (Gonzalez-Blas et al.,, 2019) are developed for scATAC-seq data. A more comprehensive discussion is presented in Lin et al., (2019). Some methods are developed for the integrative analysis of single-cell genomic data, including Seurat (Butler et al.,, 2018; Stuart et al.,, 2019), MOFA (Argelaguet et al.,, 2018), coupleNMF (Duren et al.,, 2018), scVDMC (Zhang et al.,, 2018), Harmony (Korsunsky et al.,, 2019), scACE(Lin et al.,, 2019) and MOFA+ (Argelaguet et al.,, 2020). David et al., (2020) presented a more comprehensive discussion on integration of single-cell data across samples, experiments, and types of measurement. In real-world applications, for instance, we may better cluster scATAC-seq data by using the knowledge from scRNA-seq data, or better cluster scRNA-seq data from mouse by inference from human data. This raises a critical question on how can we apply knowledge learned from one dataset in one domain to cluster another dataset from a different domain.

In this paper, we focus on the problem of clustering single-cell genomic data across different domains. For example, we may typically have an auxiliary unlabeled dataset A from one domain (say, scRNA-seq data from human), and a target unlabeled dataset T from a different domain (say, scRNA-seq data from mouse). The two datasets follow different distributions. Target data T may consist of a collection of unlabeled data from which it is hard to learn a good feature representation - clustering directly on T may therefore perform poorly. It may be easier to learn a good feature representation from auxiliary data A, which can be due to its larger sample size or less noise than T. Therefore, incorporating the auxiliary data A, we may achieve better clustering on the target data T. This problem falls in the context of transfer learning, which utilizes knowledge obtained from one learning task to improve the performance of another (Caruana,, 1997; Pan and Yang,, 2009), and it can be considered as an instance of unsupervised transfer learning (Teh et al.,, 2006), since all of the data are unlabeled.

In this work, we propose a novel co-clustering-based transfer learning model to address this problem. A schematic plot of our proposed model is shown in Figure 1. Auxiliary data A and target data T can be regarded as two matrices with cells in the rows and genomic features in the columns. The co-clustering framework (Dhillon et al.,, 2003), which clusters cells and features simultaneously, is utilized in this work. Our proposed approach is composed of two steps: in Step 1, we co-cluster auxiliary data A and obtain the optimal clustering results for cells and features; in Step 2, we co-cluster target data T by transferring knowledge from the clusters of cells and features learned from A. The degree of cluster propagation is elastically controlled by learning adaptively from the data, and we refer to our model as elastic coupled co-clustering (elasticC3). If auxiliary data A and target data T are highly related, the degree of cluster propagation will be higher. On the contrary, if A and T are less related, the degree of knowledge transfer will be lower. The contributions of this paper are as follows:

• •

  To the best of our knowledge, this is the first work introducing unsupervised transfer learning for clustering single-cell genomic data across different data types.

• •

  To ensure wide application, the model proposed in this paper can elastically control the degree of knowledge transfer and is applicable when cluster numbers of cells in auxiliary data and target data are different.

• •

  Our algorithm significantly boosts clustering performance on single-cell genomic data over traditional learning algorithms.

We first use a toy example (Figure 2) to illustrate our method. We regard two $5\times 6$ matrices as the auxiliary data (denoted as A) and the target data (denoted as T), respectively, with all matrix elements being either 1 or 0. Let $X$ and $Y$ be discrete random variables taking values from the sets of cell indexes $\{x_{1},...,x_{n_{\textsc{A}}}\}$ in the auxiliary data A and $\{y_{1},...,y_{n_{\textsc{T}}}\}$ in the target data T, respectively. Let $W$ and $Z$ be discrete random variables for the respective feature spaces of these data, taking values from the sets of feature indexes $\{w_{1},...,w_{k}\}$ and $\{z_{1},...,z_{k}\}$. Take the auxiliary data in Figure 2(a) as an example: $X=x_{i},i=1,\dots,5$ means the selected cell is the $i$-th cell among the $5$ cells and $W=w_{j},j=1,\dots,6$ means the selected feature is the $j$-th feature among the $6$ features. The meanings of $Y$ and $Z$ (Figure 2(c)) are the same as that for $X$ and $W$.

Let $p(X,W)$ be the joint probability distribution for $X$ and $W$, which can be represented by an $n_{\textsc{A}}\times k$ matrix. $p(X=x_{i},W=w_{j})$ represents the probability of the $j$-th gene being active: the $j$-th gene is expressed in scRNA-seq data or the $j$-th genomic region is accessible in scATAC-seq data in the $i$-th cell. The probability is estimated from the observed auxiliary data A, and we have $p(X=x_{i},W=w_{j})=\frac{\textsc{A}_{ij}}{\sum_{u=1}^{n_{\textsc{A}}}\sum_{v=1}^{k}{\textsc{A}_{uv}}},$ where the $\textsc{A}_{uv}$ are elements of auxiliary data observations: $\textsc{A}_{uv}=1$ if the $v$-th feature is active in the $u$-th cell, and $\textsc{A}_{uv}=0$ otherwise. The marginal probability distributions are then expressed as $p(X=x_{i})=\frac{\sum_{j=1}^{k}\textsc{A}_{ij}}{\sum_{u=1}^{n_{\textsc{A}}}\sum_{v=1}^{k}{\textsc{A}_{uv}}}$ and $p(W=w_{j})=\frac{\sum_{i=1}^{n_{\textsc{A}}}\textsc{A}_{ij}}{\sum_{u=1}^{n_{\textsc{A}}}\sum_{v=1}^{k}{\textsc{A}_{uv}}}$. For the target data, $\textsc{T}=(\textsc{T}_{ij})_{n_{\textsc{T}}\times k}$ is the observed data matrix. $q(Y,Z)$ is the joint probability distribution for $Y$ and $Z$. $q(Y)$ and $q(Z)$ are the marginal probabilities calculated similarly to that in the auxiliary data.

Our goal is to group similar cells and features into clusters (Figure 2(b) and Figure 2(e)). Suppose we want to cluster the cells in auxiliary data A and target data T into $N_{\textsc{A}}$ and $N_{\textsc{T}}$ clusters ²²2Often the number of clusters is unknown, and we usually combine other exploratory analysis including visualization with clustering to determine the number of clusters in practice.correspondingly, and cluster the features in A and T into $K$ clusters. Let $X^{\ast}$ and $Y^{\ast}$ be discrete random variables that take values from the sets of cell cluster indexes $\{x_{1}^{\ast},...,x_{N_{\textsc{A}}}^{\ast}\}$ and $\{y_{1}^{\ast},...,y_{N_{\textsc{T}}}^{\ast}\}$, respectively. Let $W^{\ast}$ and $Z^{\ast}$ be discrete random variables that take values from the sets of feature cluster indexes $\{w_{1}^{\ast},...,w_{K}^{\ast}\}$ and $\{z_{1}^{\ast},...,z_{K}^{\ast}\}$, respectively. We use $C_{X}(\cdot)$ and $C_{W}(\cdot)$ to represent the clustering functions for auxiliary data and $C_{X}(x)=x_{i}^{\ast}$ $(i=1,...,N_{\textsc{A}})$ indicates that cell $x$ belongs to cluster $x_{i}^{\ast}$ and $C_{W}(w)=w_{i}^{\ast}$ $(i=1,...,K)$ indicates that feature $w$ belongs to cluster $w_{i}^{\ast}$. For the target data, the clustering functions $C_{Y}(\cdot)$ and $C_{Z}(\cdot)$ are defined in the same way as that for the auxiliary data. The tuples $(C_{X},C_{W})$ and $(C_{Y},C_{Z})$ are referred to as co-clustering (Dhillon et al.,, 2003).

Let $p(X^{\ast},W^{\ast})$ be the joint probability distribution of $X^{\ast}$ and $W^{\ast}$, which can be represented as an $N_{\textsc{A}}\times K$ matrix. This distribution can be expressed as

The marginal probability distributions are then expressed as $p(X^{\ast}=x^{\ast}_{i})=\sum_{j=1}^{K}p(X^{\ast}=x^{\ast}_{i},W^{\ast}=w^{\ast}_{j})$ and $p(W^{\ast}=w^{\ast}_{j})=\sum_{i=1}^{N_{\textsc{A}}}p(X^{\ast}=x^{\ast}_{i},W^{\ast}=w^{\ast}_{j})$. For the target data, $q(Y^{\ast},Z^{\ast})$, $q(Y^{\ast})$ and $q(Z^{\ast})$ are defined and calculated similarly to those for the auxiliary data.

The goal of elastic coupled co-clustering in this work is to find the optimal cell clustering function $C_{Y}$ on the target data T by co-clustering the target data and utilizing the information of $(C_{X},C_{W})$ learned from auxiliary data A (Figure 2(d)).

In this section, we first present our elastic coupled co-clustering (elasticC3) algorithm, and then discuss its theoretical properties.

To evaluate the performance of our algorithm, we conduct experiments on simulated datasets and two single-cell genomic datasets.

In this paper, we have developed elasticC3 for the integrative analysis of multiple single-cell genomic datasets. Our proposed method, elasticC3, was developed under the unsupervised transfer learning framework, where the knowledge learned from an auxiliary data is utilized to improve the clustering results of target data. Our algorithm consists of two separate steps. In Step 1, we cluster both the cells and features (i.e. co-cluster) in the auxiliary data, and in Step 2 we co-cluster the target data by elastically transferring the knowledge learned in Step 1. We prove the convergence of elasticC3 to a local optimum. Our algorithm outperforms other commonly used clustering methods in single-cell genomics. Because the framework of elasticC3 is general, we plan to explore its application to other areas, including text mining.

This work does not present any foreseeable societal consequences and ethical issues.

This work has been supported by the Chinese University of Hong Kong direct grant 2018/2019 No. 4053360, the Chinese University of Hong Kong startup grant No. 4930181, and Hong Kong Research Grant Council Grant ECS No. CUHK 24301419.