Self-supervised Learning for Geospatial
AI: A Survey

Predictive methods employ pretext tasks that are formulated as prediction problems, with objectives derived from the original data instances. Specifically, these methods involve tasks like the reconstruction of corrupted geospatial objects using a subset of available data, or the prediction of auxiliary labels that are extracted from the attributes or structures of geospatial objects. They can be formulated as:

where $f_{\theta}$ and $p_{\phi}$ represent the geospatial encoder and the pretext decoder respectively. $\mathcal{D}$ denotes the target geospatial objects, which can be of any data types. $\mathcal{D}_{c}$ is the context information associated with $\mathcal{D}$, and $\mathcal{D}_{t}$ denotes the prediction objectives, which could either be the original data instance for reconstruction tasks, or additional features excluded from the encoder input for auxiliary label prediction. $\mathcal{L}_{pre}$ is the loss function that measures the prediction error, such as cross-entropy loss or mean squared error (MSE) loss.

Contrastive methods are based on the principle of maximizing agreement between different views generated from the same data instance. Specifically, these methods aim to pull closer the representations of positive view pairs, which are derived from various data augmentation operations of the same data instance, while pushing apart the representations of negative view pairs from different data instances. They can be formulated as:

where $f_{\theta}$ and $p_{\phi}$ represent the geospatial encoder and the pretext decoder respectively. $\mathcal{\tilde{D}}^{1}$ and $\mathcal{\tilde{D}}^{2}$ are two distinct views generated from the target geospatial objects $\mathcal{D}$, which can belong to any geospatial data types, and $\mathcal{\tilde{D}}_{c}^{1}$ and $\mathcal{\tilde{D}}_{c}^{2}$ are the context information associated with these respective views. $\mathcal{L}_{con}$ is the contrastive loss function that quantifies the degree of agreement, typically measured by mutual information estimator [12], such as InfoNCE [31], JS divergence [32] and triplet loss [33].

Graph Neural Networks (GNNs) [34] correspond to a type of neural network architectures designed to process graph-structured data, aiming to capture the complex relationships and structures within the graph. GNNs employ message-passing operations iteratively on the graph, where the representation of a node $v$ is updated through interactions with its neighbors. This process can be expressed as:

where $h_{v}^{(l)}$ indicates the representation of $v$ at layer $l$ and $N(v)$ denotes the neighbors of $v$. $\operatorname{AGG}^{(l)}$ is the message aggregation function at layer $l$, which collects and combines node features, and potentially edge features, from the neighbors, and $\mathcal{F}^{(l)}$ is the function that updates the representation of $v$ based on the aggregated information. For geospatial objects, GNN are frequently utilized to model discrete objects, enabling the capture of complex relationships among them.

Sequential models are designed to process input data composed of sequences, which include domains such as time series, text, audio, and video. Over the past decade, neural network architectures have exhibited exceptional performance in sequence modeling due to their capability of capturing dependencies effectively. The general process can be described as:

where $[x_{1},...,x_{n}]$ denotes the input sequence, $[h_{1},...,h_{n}]$ denotes the hidden representations output by the sequential model $\mathcal{F}$, which is parameterized by $\Theta$. Recurrent Neural Networks (RNNs) accomplish this by recursively processing the current input along with previous elements of the sequence, where the previous elements are encoded into internal hidden states, leading to several model variants, with GRU [35] and LSTM [36] being the most notable ones. In recent years, the Transformer architecture [37] has revolutionized sequence modeling by handling historical sequences in a parallel manner, instead of the recursive approaches. Meanwhile, it demonstrates superiority of modeling pairwise relationships between any two positions in a sequence through self-attention mechanism. For geospatial objects, sequential models are particularly valuable for modeling trajectories or data instances that are built to consider the sequential dependencies.

The evolution of advanced sequential models, especially those based on the Transformer architecture, have marked the milestone in the development of pre-trained language models. One popular paradigm is to adhere to the principles set by BERT [38]. These models [39, 40, 41, 42] leverage large-scale datasets to employ the training objective of masked language modeling (MLM). This process enables the acquisition of rich and transferable representations, which can be fine-tuned for specific tasks with less labeled data. Another paradigm is to employ training with autoregressive language modeling (i.e., next token prediction), exemplified by ChatGPT and its counterparts [43, 44, 45, 46, 47]. These large language models (LLMs) demonstrate strong capabilities in language understanding and reasoning, transforming various tasks into the process of autoregressive language generation. The robust performance of these two paradigms has led to the widespread use of pre-trained language models to encode textual information associated with geospatial data or to adapt their training objectives to develop specialized representations. On the other hand, in scenarios where vision information, such as street view images, is associated with geospatial elements like polylines or polygons, pre-trained visual models are also utilized. These include CNN-based models [48, 49] and recent Transformer-based models [50, 51] trained on large-scale image datasets. When both textual and visual data sources are available, visual-language pre-trained models such as CLIP and its variants [52, 53] are employed to synergize the semantics of the two data modalities, enhancing the interpretability and utility of combined data sources in geospatial applications.

In the following sections, for each data type under consideration, our analysis adopts a structured way. We first present the encoding methods for intrinsic attributes that reflect the fundamental characteristics of the data instance. Then we discuss the context information leveraged to enhance the extraction of meaningful insights and knowledge. Finally, we introduce the downstream applications that benefit from the derived geospatial representations. The overview of applying SSL on geospatial data is illustrated in Fig. 2.

POIs refer to the semantic locations in location-based services that users might be interested in visiting. Examples of POIs include restaurants, stores, schools, etc. Each POI is associated with geographical coordinates $l$ (i.e., geo-location) and usually a number of features $x=\{name,categories,reviews\}$, including the POI’s name, one or multiple categories, and possibly a set of user reviews.

Since the geo-location and features are in different formats, existing methods [54, 5, 55] often employ separate encoders to extract essential information from geo-location [27], categories, and other text features [38]. The encoded features are fused in a subsequent model, such as multi-layer perceptrons (MLP) or attention-based models, to obtain the POI representations [54, 5, 55]. The objective of SSL for POI is to learn an encoder $f_{\theta}$ that can acquire a $d$-dimension latent representation of any input POI $p$, denoted by $\mathbf{e}_{p}=f_{\theta}(p)\in\mathbb{R}^{d}$, such that $p$’s geo-location $l$ and features $x$, as well as $p$’s context are well preserved in $\mathbf{e}_{p}$. Based on this objective, we note that scenarios of (next) POI recommendation [56] are more related to specific tasks with supervised signals rather than the SSL setting. Therefore, we do not focus on these methods in this section.

The intrinsic attributes only include the information of individual POIs. POIs are often located in spatial environments where diverse types of POIs are present at different distance ranges. Besides, users’ check-ins or queries related to POIs indicate the dependency on the urban functions of distinct POIs. Consequently, most SSL research for POIs has been dedicated to modeling the rich context information of POIs, including spatial neighborhoods, check-in sequences, co-query context, and temporal context.

GNN-based methods construct a graph where two POIs are connected by an edge if they are close in spatial. Leveraging the ability of GNNs to acquire the local structure of a graph, these methods encode the spatial neighborhood of a POI into its latent representation. To learn the POI representation in a self-supervised manner, predictive SSL methods have been exploited to predict either the intrinsic attributes of the POI or the relation between POIs. STPA [58] constructs a Delaunay triangulation graph based on the distance between POIs. Each node in the graph is a POI, represented by a one-hot encoding of its category. GNN is applied to acquire a target POI’s representation by aggregating the category information from its neighbors in the Delaunay triangulation graph. Subsequently, a predictive objective is employed to predict the category of the target POI, given its latent representation. For objectives that predict the relations (competitive or complementary) between POIs, the relations between POIs are constructed based on certain heuristic rules. DeepR [59] builds the competitive relations between two POIs if they are close in spatial and frequently co-occur in the same query. DeepR employs a Heterogeneous POI Information Network (HPIN) to represent POI, brands, aspects, and their relations, and a spatial adaptive GNN to acquire the POI representations from neighbors. Given the representations of two POIs, it predicts whether they are in a competitive relation. PRIM [60] constructs competitive relations between different categories of POIs if they frequently co-occur in the same query. Likewise, it constructs complementary relations between POIs in the same category that frequently co-occur in the same query. To predict the relations, PRIM employs the POI representations obtained through a GNN that gives importance to the neighbors based on spatial distance and feature similarity.

Besides, numerous studies leverage the MLM idea applied in pre-trained language models to learn POI representations [61, 55, 54]. The common idea behind MLM-based methods is to construct pseudo sentences from the spatial neighborhoods and then apply MLMs on the pseudo sentences. GeoBERT [61] divides the digital map into grids and proposes two methods to create a pseudo sentence from POIs residing in each grid cell. The first method creates the pseudo sentence by finding the shortest path between the two farthest POIs, passing through all POIs in the same grid cell. The second method returns the ordered sequence of POIs by their distance to the grid center. With the obtained pseudo sentences, GeoBERT treats each POI as a token and adopts the same training mechanism as existing MLMs – predicting a masked POI in the pseudo sentence based on others. SpaBERT [55] creates pseudo sentences by concatenating the names of neighboring POIs. For each POI, SpaBERT finds its neighboring POIs using Geohash and sorts them by their distance to the POI. Subsequently, SpaBERT masks some or all words of a certain POI in a pseudo-sentence and predicts the masked words based on the remaining words in the sentence. In this way, the learned POI representation preserves essential information for generating the words that appear in the nearby POIs, thus capturing the underlying correlations between geo-locations and text. To better leverage the spatial context and support query-POI matching, MGeo [54] designs several geospatial encoders to encode the spatial attributes of a POI, including ID, shape, map position, and relative location to neighboring spatial objects extracted from OpenStreetMap [67]. Subsequently, MGeo employs both predictive and contrastive SSL objectives to train these encoders. The predictive SSL objective minimizes the loss of predicting the masked attribute (e.g., shape) using other attributes. The contrastive objective minimizes the difference between the actual spherical distance and the distance computed by the representations of all pairs of POIs in the same batch. After training the encoders, MGeo fuses the text and spatial information through another MLM that predicts the masked words or spatial attributes of a POI.

Existing studies [68, 69, 70, 71, 72] employ skip-gram or contextual bag-of-word (CBOW), which are two typical predicative objectives, to the check-in sequences. Specifically, C-WARP [68] learns POI representations to predict other POIs within the same check-in sequences. As a result, the co-occurrences of POIs in the same check-in sequences are preserved in the POI representations. Building upon this, CAPE [70] learns POI representations that effectively predict both IDs and texts of other POIs in the same sequences. In this way, the text information in the sequential context is encoded in the learned POI representations. Unlike previous methods that predict all POIs visited within a close time period, DeepMove [71] focuses more on the intent of the trip and considers only the origin and destination POIs of a trip. Utilizing an origin POI, DeepMove predicts other origin POIs with the same destination POI. Consequently, POIs with similar travel intents are projected into a close region in the learned representation space. Hier-CEM [72] extends the check-in sequence context by associating each check-in POI with the ancestor categories. POI2Vec [69] introduces a binary region tree to enhance the modeling of check-in sequences with spatial proximity.

The POI representations acquired from the SSL methods can be applied to various downstream applications. Most POI representations can be used to predict specific POI attributes, such as category [58] and geo-location [54]. Besides, POI presentations can be directly used or used with a subsequent sequential model to predict the next POI visited by a user [68, 69, 70]. The POI representations can also be used to answer POI queries by matching POI representations and the query representation [62]. Furthermore, the presentations of multiple POIs can be used for more complex geospatial data mining tasks. For instance, POI relation prediction methods [59, 60, 74] use the representations of two POIs to predict their competitive or complementary relations. Likewise, the POI representations are pivotal in matching POIs from different databases for entity resolution [76]. In addition, increasing studies have utilized POI embeddings in an aggregated manner for the inference of land use [64] or land use change [65] across years. The aforementioned applications demonstrate that the SSL methods can effectively encode POIs and their contexts, thereby helping to improve the accuracy of various GeoAI tasks.

Road networks are composed of connected road segments, where each segment is characterized as a polyline. As a collection of polylines, road networks are usually concepturalized as a graph. In this regard, two primary graph-based perspectives are adopted in the research. In the first perspective, road segments themselves are treated as nodes, and the connections between these segments are treated as graph edges. In the alternative perspective, road intersections serve as nodes, while the road segments linking these intersections are treated as edges.

The graph-based formulation from these two perspectives naturally represents the topological structure of road networks. Additionally, various attributes on road networks are integrated as node or edge features within the formulation, such as geo-locations, road attributes, and intersection characteristics. As a result, existing studies on SSL for road networks generally follow the principles of SSL for graph data. The objective is to learn geospatial encoder $f_{\theta}$ to obtain node representations $\{\mathbf{e}_{v}\}_{v\in\mathcal{V}}\in\mathbb{R}^{d}$ (i.e., $\{\mathbf{e}_{v}\}_{v\in\mathcal{V}}=f_{\theta}(\mathcal{G}$), where $\mathcal{G}$ and $\mathcal{V}$ denote the road network graph and its nodes, by employing either graph perspective in a self-supervised manner.

Early approaches generally adopt predictive SSL with various training objectives. Initial exploration into this field [77] applies node2vec [78], a self-supervised graph representation learning method based on skip-gram training, directly to road networks. It predicts road segments within a context window derived from random walks sampled on road networks [63]. The derived representations are demonstrated to be effective on several road classification tasks. Building upon this groundwork, IRN2Vec [79] targets at learning representations for intersections by treating intersections as graph nodes. This method refines node2vec by employing shortest-path random walk sampling and selecting positive intersection pairs based on defined road path distances for skip-gram training. Moreover, it leverages intersection-specific attributes (e.g., intersection tags and types), to enrich the selection of positive road segment pairs. Another research line explores GNN with reconstruction-based objectives. For example, RFN [80, 81] applies GNN to both perspectives of the road network graphs, fusing features from both road segments and intersections, including zone categories, road types and intersection angles, to derive representations capable of reconstructing the original graphs. HyperRoad [82] expands upon the vanilla graph structure by constructing a corresponding hypergraph, where hyperedges represent the road segments within the polygons produced through a map segmentation algorithm [83]. GNN is further enhanced by a dual-channel attention mechanism that operates on both the original graph and the hypergraph. This method is trained to not only construct both the original graph and its corresponding hypergraph, but also to perform a hyperedge classification task.

Apart from the topological structure and the attributes associated with road segments and intersections, road networks contain rich context information that can be leveraged to enhance the extraction of semantic knowledge. We introduce three types of context information utilized in existing methods, where road segments are mainly regarded as graph nodes.

The representations of road networks, derived from either predicative or contrastive SSL objectives, provide task-agnostic inputs that can be directly utilized or fine-tuned with geospatial encoder, for downstream applications. For example, they are directly applicable in classifying attributes of road segments or intersections with simple models (e.g., MLP), such as road type [89], lane number [82], and intersection tags [79, 84]. Besides, these representations are similarly utilized to infer the traffic status of road segments, including metrics like average speed [87, 90] and speed limits [81]. They also support the efficient vector computation for road network-based queries, such as calculating shortest path distances [85]. Furthermore, road segment representations serve as the effective start point for training in models that involve map-matched trajectories, for applications such as destination prediction [86] and anomalous sub-trajectory detection [94]. By enabling a diverse range of analytical tasks related to road networks, these representations offer substantial potential to improve understanding and decision-making within the domain of transportation infrastructure.

A trajectory, composed of a sequence of sampled points that represent the path of a moving object, can essentially be conceptualized as a polyline. Based on the definition, the context feature associated with each point in trajectory data instance includes timestamps, along with potentially additional textual content, and semantic tags, etc. The objective of SSL for trajectory data is to learn a representation produced by geospatial encoder $f_{\theta}$ for any given trajetocry $\mathcal{T}$: $\mathbf{e}_{T}=f_{\theta}(\mathcal{T})\in\mathbb{R}^{d}$. The sequential nature of a trajectory, marked by its spatio-temporal point sequence, represents its most fundamental intrinsic attributes. Accordingly, sequential models are used to model trajectory data, effectively capturing the transition patterns and long-term dependencies inherent in this data instance. The surveyed studies for road networks are listed in Table III

In the category of predicative SSL methods, trajectories are typically trained to reconstruct their original sequence from a corrupted version of the input. For example, TCDRL [95] proposes to extract local features within sliding windows applied on trajectory records, such as speeds and rate of turns. These features are then processed using a sequence-to-sequence encoder-decoder architecture [96] based on RNN to reconstruct the local features for each window. t2vec [97] refines this reconstruction process by aligning with the tokenization paradigm in natural language. It partitions the geospatial space into regular and square-sized grids and match the coordinates to their corresponding grids (tokens). Moreover, t2vec introduces downsampling and point distortion in the input sequence, and aims to reconstruct the original trajectory with spatial proximity aware loss that penalizes more for the predicted tokens that deviate significantly from the correct grids. In such an encoder-decoder framework, the vector produced by encoder part can be treated as the trajectory representation.

This framework is enhanced by integrating a variety of modifications in terms of model architecture and loss functions. Specifically, model adaptations include refinements via variatioanl inference [98] or self-attention mechanisms [99]. Geo-Tokenizer [100] reduces the number of grids to be trained by representing a location as a combination of multiple shared grids at several granular scales. It then utilizes the objective of predicting the grids for the next token at each scale in a hierarchical way. Furthermore, AdvTraj2vec [101] aims to learn more robust trajectory representations through adversarial training. It adds perturbations to the token embeddings of the input sequence, with the magnitude of these perturbations guided by generative adversarial network (GAN) [102] to ensure the effects are neither too large or too small. E²DTC incorporates self-training via soft cluster assignments [103] as an auxiliary loss into the reconstruction process. STPT [104] performs a sub-trajectory discrimination loss that differentiates whether pairs of sub-trajectory representations originate from the same source.

In addition to considering spatial dimension associated with trajectories, several SSL methods leverage temporal dimension into the reconstruction process. The method in [105] expands the square-sized spatial partitions to 3D spatio-temporal grids for each point. It accordingly adapts the reconstruction loss to account for temporal effects, imposing heavier penalties for reconstructed results with larger temporal discrepancies. Similarly, RSTS [106] also employs 3D spatio-temporal grids and applies a linear combination of spatial and temporal distances for reconstructed results as the final loss. Besides, DeepTEA [107] utilizes Convolutional-LSTM [108] to model historical traffic conditions reflected in holistic trajectories, thus enhancing the variational inference by providing additional hints into dynamic patterns for each trajectory.

In the category of contrastive SSL methods, standard mutual information maximization objectives are applied to produce similar/dissimilar representations for views derived from the same/different trajectories with various data augmentation operations. CL-Tsim [109] primarily uses point distortion to generate positive pairs of trajectories for contrastive learning. Expanding on this, TrajCL [110] introduces three additional operations, namely point masking, trajectory truncating and trajectory simplification, to enhance the diversity of the patterns for positive pairs. Besides, it proposes a dual-feature self-attention-based encoder to process not only the grid sequence, but also the spatial attributes for points, such as coordinates, angles and lengths. KGTS [111] further implements GNN to consider the interactions of neighboring grids, and includes grid deletion and movement operations to both the entire trajectories and partial trajectories to create positive pairs.

In addition to the spatial and temporal attributes intrinsic to trajectories, we discuss two distinct scenarios where trajectories are modeled under specific constraints and characteristics with further context information.

Several predicative SSL methods for map-matched trajectories utilize objectives similar to those used in trajectories with grid partitions, such as the reconstruction of road segments [112, 113] and next road prediction [127]. Besides, tailored objectives are devised to effectively incorporate the knowledge within road networks. For example, JGRM [114] aims to recover the road segments masked from the complete path. It also considers the original point sequence and adopts another objective of differentiating whether pairs of representations are derived from the same trajectory with both the road segment sequence and the point sequence.

For contrastive SSL methods applied to map-matched trajectories, ST2Vec [128] adopts co-attention mechanism to merge spatial and temporal embeddings, followed by LSTM sequence modeling. This model employs energy-based margin functions (i.e., triplet loss) to enforce higher similarities for positive pairs which follow similar routes. GRLSTM [115] combines GNN and graph embedding techniques [129] to handle sequence inputs and augments LSTM with residual connections to generate trajectory representations. It employs similar triplet loss at both the point level and the trajectory level. In addition, PIM [116] treats trajectories that share the same source and destination as positive pairs, and aims to maximize mutual information over these pairs. MMTEC [117] utilizes both a discrete sequence model based on Transformer, and a continuous model formulated as neural controlled differential equation to generate representations from two views. It applies a different contrastive learning objective of maximizing entropy coding between two views.

Moreover, both contrastive and predictive SSL can be simultaneously employed with a single framework. HMTRL [118] employs the strategies to predict road attributes and traverse time for road segments, as well as employing full trajectory and its sub-trajectory for contrastive learning. START [119] incorporates trajectory pattern into GNN aggregation and enhances Transformer with time-aware self-attention mechanism. Furthermore, it utilizes masked road segment recovery as the predicative task, and enhances its contrastive learning aspect by employing four distinct data augmentation operation –trajectory trimming, masking, temporal shifting, and dropout– to generate positive pairs. Similarly, LightPath [120] utilizes masked road path recovery as its predictive task. For its contrastive aspect, LightPath achieves pairwise matching through the use of dual encoders and varied dropping ratios to generate positive pairs from the same trajectories.

Similar to the representations derived for road networks, trajectory representations can be directly utilized or fine-tuned for downstream applications. These representations, regardless of their specific context information, facilitate a variety of operations due to their vectorized format, including similarity computation [97, 133] and clustering [103, 95]. Moreover, they can be utilized to support diverse applications, such as transportation mode prediction [100], driver status inference [104], and anomalous trajectory detection [98]. In scenarios involving map-matched trajectories, these representations prove particularly valuable in applications focused on path analysis, such as travel time estimation [119], path ranking [120], and destination prediction [117]. In addition, semantic trajectory representations trained with SSL methods are typically fine-tuned to enhance performance in next location prediction [121], and can be uniquely applied in trajectory user-linking task [124, 123]. These applications demonstrate the broad utility and adaptability of SSL for trajectories in advancing the understanding and analysis of mobility patterns and intelligent transportation systems.

Among various data modalities, POIs and imagery data, including remote sensing (RS) images and SVIs, are most commonly utilized to reflect the intrinsic properties of each region. POIs capture socioeconomic factors, while images reflect the overall physical appearance from a human or aerial perspective. Several studies have explored these data sources to learn region representations in a self-supervised manner, sometimes with minimal consideration of contextual information.

In a pioneering study on POI-based region representation learning,  [145] treats each region as an “image” where some pixels are filled with POIs. This approach allows regions to be processed by a CNN, with POI category information, represented by one-hot encoding, serving as “pixel values”. The model is trained with the objective of a triplet loss. For each anchor region, its augmentation generated by random removal, addition, and shifting of POIs serves as a positive sample, while negative samples are non-overlapping regions or augmentations with larger perturbations (hard negative samples). UrbanCLIP [160] utilizes RS images and LLM to learn region representations. For each image, it generates a detailed description using a pre-trained LLM, where detailed and specific prompts oriented to urban infrastructures are found to be beneficial. The method then trains an image encoder and a text encoder using contrastive learning and auto-regressive text generation. Building upon this work, UrbanVLP [161] employs both SVIs and RS images. It proposes to filter the LLM-generated text by a quality metric CycleScore, to avoid low-quality text generation. This approach fuses remote sensing and street view images within each region, contrasting them with generated texts at both image and token levels. In addition, MMGR [159] proposes to fuse RS images and POIs for learning intrinsic attributes of urban regions. It extends the idea in [13] by carrying out contrastive learning between multiple augmentations of each image, as well as between images and POIs using the method proposed in [64].

Incorporating diverse sources of context information to model the semantics of urban regions has become a standard practice. The contextual information often comes from connectivity patterns exhibited in human trajectories, spatial proximity, temporal patterns, and other types of property similarities (e.g., from land use data and knowledge graphs). The primary incentive for utilizing context information is its ability to significantly enhance the quality of learned region representations, especially when the expressiveness of data modalities representing intrinsic attributes, such as POIs, is limited. We organize the subsequent content based on the data modalities utilized to learn representations for urban regions, allowing for a focused discussion on how different types of data contribute to the modeling process.

Later, ZE-Mob [149] defines several human mobility events pertaining to regions, time, and movement mode (i.e., departure/arrival). In this way, region representations can be learned similarly in Word2vec [63] by skip-gram objective. Besides, it enriches the objective by integrating the importance of region origin-destination pairs based on popularity and distance. GMEL [150] constructs a region adjacency graph and subsequently employs two graph attention networks to model the two types of representations for regions functioning as travel origins and destinations. The model is trained with the pretext tasks of predicting human flow and predicting in/out flow. MGFN [151] regards human movements in each time slot as a mobility graph and defines several graph distance measures to cluster the graphs into several mobility patterns, e.g., distance between mean or variance of edge weights and human flow imbalance. A hierarchical clustering method is then applied to distill these into a reduced number of region graphs that represent specific mobility patterns. Within each mobility pattern graph, message passing is performed. Besides, inter-pattern attention mechanisms are employed to fuse various mobility patterns for the final region representations. The model is trained with the objective of reconstructing region-level human mobility patterns.

Another line of research extensively emoplys GNNs. MVURE [152] utilizes human trajectories, POIs, and user check-ins to construct four different graph views. It then employs graph attention networks to encode each view and implements cross-view information sharing with attention mechanisms as well as multi-view fusion using adaptively weighted combination of different views. The model is trained through objectives of reconstructing human mobility patterns and region relatedness reflected by POIs and check-ins. HREP [140] further considers several types of graph edges using human mobility data, POIs, and geographic context. This method constructs source and target edges based on human trajectories, POI edges derived from the similarities of POIs shared by different regions, and geographic context edges based on spatial adjacency. Using this heterogeneous graph structure, region representations are learned through attention-based GNN method by three objectives: a geographic proximity loss, a mobility reconstruction loss, and a POI correlation reconstruction loss. Additionally, HREP employs prompt learning (prefix-tuning) to adapt the learned representations for various downstream urban analytical tasks, a technique adapted from natural language processing. In addition, HUGAT [153] utilizes POI check-in trajectories and land use data for region representation learning. It defines five types of nodes, such regions and POI categories, and two types of edges representing spatial and temporal relations. This method then constructs a heterogeneous information network and designs five meta-paths to involve interesting relationships between regions, such as identifying regions that are popular destinations at the same time. A heterogeneous graph attention network [163] is employed to derive region representations by minimizing the difference between estimated and actual mobility patterns of regions, check-in distributions, and land use distribution. Region2Vec [141] utilizes inter-region human trajectories, spatial adjacency, and POI distributions as three similarity measures to construct a multi-graph for regions. Specifically, human mobility similarity is reflected by the co-occurrence frequency between regions, and POI distribution similarity is measured using the embeddings from a POI knowledge graph. This method utilizes GNN and fusion layers based on attention mechanisms to generate comprehensive region representations, with the training objectives of reconstructing human mobility patterns, preserving geospatial adjacency, and POI distribution similarity.

The third line of research employing POIs and human trajectories focuses on the application of contrastive learning paradigm. ReMVC [142] is a dual-view approach integrating POIs and human mobility data. For the POI view, it uses region-level POI category proportions as raw features, and performs data augmentation through random POI insertion, deletion, and replacement. These augmented POI views of regions serve as positive samples, whereas POI data from differing regions are used as negative samples for the contrastive learning. For the human mobility view, it constructs two heatmaps to represent the source and destination patterns of each region, followed by the augmentation of Gaussian noise injection to form positive samples for the contrastive learning. Furthermore, an inter-view contrastive learning objective is employed to ensure that representations of a region from different perspectives are similar, while the representations of different regions are distinguishable. ReCP [143] develops a fusion technique for multiple information views from an information theory perspective. Apart from maximizing the mutual information (consistency) shared between different views, this method implements a dual prediction strategy to minimize the conditional entropy between representations from different views, thereby reducing the inconsistency between views.

Tile2Vec [154] generates representations based on RS images patches for square-shaped regions. It employs a CNN model to encode remote sensing images and a triplet loss to ensure that patches which are geographically adjacent are also close in the embedding space. RegionEncoder [155] leverages RS images, POIs, and human trajectories for learning region representations. Initially, it employs a denoising autoencoder to extract representations from RS images. Following this, a region graph is constructed, utilizing region-level POI features as node attributes and human trajectory data to define inter-region connectivity through edges. The model is trained with three specific objectives: a reconstruction loss for the denoising autoencoder, a reconstruction loss for human mobility patterns, and a binary cross-entropy loss used by a discriminator to verify whether RS image representations and region graph node representations correspond to the same geographical region. The study in  [158] adopts a contrastive learning strategy to maximize the similarity of representations derived from RS images that are geographically adjacent and those that have similar POI distributions. The representations learned from the two pathways, including spatial proximity and POI distribution similarity, of each region are adaptively fused using learnable weights in downstream tasks. Urban2Vec [156] derives the initial region representations by averaging all the SVI representations obtained from a CNN-based model for each region. It subsequently fine-tunes this image encoder with a spatial proximity-based triplet loss to bring SVIs that are spatially close together in the embedding space. Besides, this method integrates the POIs by further generating the POI representation of a region that encapsulate all the words associated with the POIs in each region. It introduces another triplet loss designed to merge POI information into the region representations by minimizing the distance between each region’s representation and its corresponding POI representation. M3G [157] extends Urban2Vec by simultaneously utilizing SVIs, POIs, and human trajectories. The techniques for handling SVIs and POIs are similar to those in Urban2Vec. Additionally, M3G enhances the model by conducting region-level contrastive learning, selecting other regions that are either spatially close or connected through human mobility as positive samples. The model is trained using a combination of triplet losses: region-SVI, region-POI, and region-region, which consider both spatial proximity and human mobility connections.

As urban regions serve as a critical analytical scale for various urban analyses and prediction tasks, the learned region representations are used in a diverse range of downstream tasks. Commonly, these region embeddings are utilized by integrating them as frozen inputs into shallow task predictors, such as MLP, for making task-specific inferences. Additionally, several studies have enhanced the utilization of region representations in downstream tasks through advanced methods like prompt learning [140] and adaptive multi-view fusion [158]. From the perspective of downstream tasks, the predominant tasks involve predicting various socioeconomic indicators in cities. These include region-level attributes such as land use/urban function/land cover [149, 142, 143], population density [156, 147], house prices [155, 160], average income [157, 148], check-in counts [137, 139], crime rates [151, 148], service call volumes [144], GDP [159, 160], nighttime lighting [161], takeaway order volumes [158], health indices [154], and so on. Moreover, region representations are extensively utilized for tasks like similar region search[145, 146] and region or land use clustering [141] in a fully unsupervised manner, which has significant practical implications for real-world urban planning and management. Furthermore, region representations can benefit studies on human mobility in cities, such as predicting human flow and bike flow [150, 153].

The application of SSL-based region representations has proven to be diverse and effective across many critical urban analytical tasks, leading to substantial real-world benefits. Such applications not only improve the accuracy of urban analyses but also support more informed decision-making in urban planning and policy-making. For example, accurate predictions of socioeconomic indicators can inform better resource allocation, enhance public services, and facilitate targeted interventions in sectors like healthcare, education, and transportation. Additionally, the capability to identify and cluster similar regions supports strategic site selection, the development of cohesive urban zones, optimization of land use, and promotion of sustainable development. As urban environments continue to expand and evolve, the role of robust region representations in tackling complex urban challenges will become increasingly crucial, paving the way for smarter, more resilient cities.