Leveraging Two Types of Global Graph for Sequential Fashion Recommendation

Personalized fashion recommendation aims to model the personalized fashion preference of users. Most previous works (He et al., 2016; He and McAuley, 2016b; Yu et al., 2018; He and McAuley, 2016c; Kang et al., 2017; Liu et al., 2017) focus on emphasizing visual representation of fashion items in building the recommender system as visual information is considered much more important in fashion domain than in other domains. Despite the effectiveness of incorporating visual feature, existing works widely overlook the unique user behaviors in fashion domain and fail to explore more effective behavior patterns from the interaction history of users. The sparsity issue is also not properly addressed in previous works. However, the issue is particularly significant in fashion domain as the item set is way larger compared to other domains such as books or music (Hu et al., 2015).

In literature, matrix factorization (MF) (Rendle et al., 2012) is one of the most simple and effective non-sequential recommendation methods. However, MF, as well as other non-sequential methods (Wang et al., 2019; He et al., 2020, 2017), only models the u-i interactions but is not able to take i-i transition into account. In comparison, session-based methods usually solely explore the item transition patterns for the recommendation. In recent years, Deep Neural Networks (DNNs) have been widely applied in session-based recommendation for its capability of modeling sequential data. Various types of DNNs, e.g., Recurrent Neural Networks (RNNs), Convolutional Neural Networks (CNNs), have been applied in developing advanced session-based recommenders (Hidasi et al., 2015; Li et al., 2017; Quadrana et al., 2017; Tang and Wang, 2018). In the recent years, GNNs have drawn special attention and some GNN-based methods have achieved state-of-the-art performance in session-based recommendation tasks (Wu et al., 2020).

Compared with above two types of methods, sequential recommendation methods model the user’s general preference and also analyze the latest action(s) of him (Fang et al., 2019; Quadrana et al., 2018). Markov Chain (MC) is an effective tool to model the action sequence of users and infer the next one. Combined with Matrix Factorization (MF), Rendel et al. (Rendle et al., 2010) propose the Factorized Personalized Markov Chain (FPMC) to capture both the sequential patterns and the long-term user preference. Despite of its simplicity and efficiency  (Feng et al., 2015; Wang et al., 2015; He and McAuley, 2016a), it only applies the first-order MC and fail to model any high-order user-item or item-item dependency or connectivity, which still limits the overall recommendation performance.

With the rapid development and tremendous success of GNNs in many application domains (Scarselli et al., 2008; Kipf and Welling, 2016; Ding et al., 2021), recommendation approaches based on GNNs (Wu et al., 2020; Guo et al., 2020) have achieved state-of-the-art performance in various sub-tasks, such as implicit feedback-based general recommendation and session-based recommendation (He et al., 2020; Wang et al., 2020). Graph is a natural data structure for most of data in recommender systems. For example, the user and item interactive relationship can be modeled with a bipartite graph. Another main contribution of GNN in recommender system is its ability of capturing the high-order connectivity, which can help inject the CF signal into the embedding learning process, thereby achieving significant improvements on performance (Wang et al., 2019; He et al., 2020). Wang et al. (Wang et al., 2019; He et al., 2020) proposed the Neural Graph Collaborative Filtering (NGCF) method for the implicit feedback-based recommendation. NGCF builds the user-item interaction graph first and then conducts multi-layer embedding propagation on the graph to refine the embeddings of users (or items). It has achieved very competitive performance over all CF-based recommendation methods, as well as its variant LightGCN (He et al., 2020).

GNNs have also been applied in session-based recommendation to capture the transition patterns in sessions. The majority of existing methods build the graphs based on the sequence of items within the same session and then apply GNNs to capture transitions among items, which are claimed to be complex and difficult to be captured by previous conventional sequential methods (Wu et al., 2019a). Methods with such a framework include SR-GNN (Wu et al., 2019a), FGNN (Qiu et al., 2019), GC-SAN (Xu et al., 2019), A-PGNN (Wu et al., 2019b). In summary, different efforts have been made in modeling the high-order connectivity of either user-item interaction or item-item transition. However, so far no work has tried to modeling both yet. Since the sparsity issue is particularly severe in sequential fashion recommendation, leveraging more contextual information by high-order connectivity, for both u-i interaction and i-i transition, would be significantly valuable.

The problem studied in this paper is sequential fashion recommendation. Let $\mathcal{U}=\{u_{1},u_{2},...u_{N_{U}}\}$ denote the whole user set and $\mathcal{I}=\{i_{1},i_{2},...i_{N_{I}}\}$ denote the whole fashion item set, where $N_{U}$ and $N_{I}$ are the total number of users and fashion items respectively. Each user $u$ is interacted with a sequence of fashion items in chronological order, denoted as $\textbf{s}^{u}=[s_{1}^{u},s_{2}^{u},...,s_{t}^{u},...,s_{|s^{u}|}^{u}]$, $s_{t}^{u}\in\mathcal{I}$. The objective of sequential fashion recommendation is to predict the next action and make recommendations accordingly, in other words, to predict the probability of item $i$ to be the next pick of $u$ after picking $l$: $y_{u,l,i}:=p(i=s_{t}^{u}|l=s_{t-1}^{u},u)$.

Personalized sequential recommendation aims to predict the triplet score among the user, previously interacted item and target item $y_{u,l,i}$. It can be formulated as a personal transition cube prediction problem (Rendle et al., 2010), where one own transition matrix is learned for each user. Due to the limited observations for estimating the transition cube, factorizing methods are adopted to factorize the transition cube into a linear combination of both user-item (u-i) interaction and item-item (i-i) transition (Rendle et al., 2010): $y_{u,l,i}=y_{u,i}+y_{l,i}$. Following the typical sequential recommendation approach FPMC (Rendle et al., 2010), we describe a user (or an item) with an embedding vector and model the u-i interaction and i-i transition with inner-product as:

$\boldsymbol{e}^{U}_{u}\in\mathbb{R}^{d}$ is the embedding of user $u$, $\boldsymbol{e}^{IU}_{i}\in\mathbb{R}^{d}$ is the embedding of item $i$ specifically for the modeling of the interaction with users. $\boldsymbol{e}^{U}_{u}$ and $\boldsymbol{e}^{IU}_{i}$ are looked up from the embedding table of users $\boldsymbol{E}^{U}\in\mathbb{R}^{N_{U}\times d}$ and that of items $\boldsymbol{E}^{IU}\in\mathbb{R}^{N_{I}\times d}$, where $d$, $N_{U}$ and $N_{I}$ are the the embedding dimensionality, user set length, and item set length respectively. Likewise, $\boldsymbol{e}^{L}_{l}\in\mathbb{R}^{d}$ is the embedding of item $l$, to specifically represent the anchor (last interacted) items, $\boldsymbol{e}^{IL}_{i}\in\mathbb{R}^{d}$ is the embedding of item $i$ that is specifically for the item transition modeling. The corresponding embedding tables are $\boldsymbol{E}^{L}\in\mathbb{R}^{N_{I}\times d}$ and $\boldsymbol{E}^{IL}\in\mathbb{R}^{N_{I}\times d}$. Note that although all the $\boldsymbol{E}^{IU}$, $\boldsymbol{E}^{L}$ and $\boldsymbol{E}^{IL}$ are for item embedding lookup, they are individually initialized and optimized for different purposes.

After the factorization operation, the overall prediction score is divided into two parts: 1) the interaction probability score between the given user and the target item, corresponding to the user’s personalized long-term static preference, and 2) the transition probability score between the previously interacted item and the target item, corresponding to the user’s short-term dynamic preference.

Even though FPMC is a concise and effective method, it has the following inherent limitations. First, the user and item representative capability is limited since the model learns the user/item representations solely based on the dot-product score of specific interaction (transition) pair while overlooking the global contextual information (i.e., all interaction or transition pairs). For the u-i interaction, all the items (users) a user (an item) has interacted with can be treated as the features of the it (Wang et al., 2019). Such features can enhance the user (item) representations. Likewise, for the i-i transition, all the items that an item connected (either as predecessors or successors) can be treated as the contextual features, which can be used to enhance the short-term preference modeling.

Second, FPMC only considers first-order relations, i.e., the direct u-i interaction and first-order Markov Chain transition between item and item, while overlooking the higher-order relations. In fact, higher-order relations are prevalent and meaningful in modeling both u-i interaction and i-i transition. As shown in Figure 2 (we will present the details of the graph construction later), $u_{1}$ and $u_{2}$ have a second-order connection since both of them have interacted with item $i_{1}$. Consequently, $u_{1}$ and $u_{2}$ may share some common preferences and could serve as CF features for each other. Similarly, the higher-order Markov Chain transitions also provide rich contextual information to improve the modeling of the i-i transition. Based on above considerations of learning better user and item representations, we propose to leverage two types of global graph to exploit more higher-order relations.

Graph Construction: We demonstrate a toy example of sequential fashion recommendation in Figure 2 (a), where there are two users ($u_{1}$ and $u_{2}$) and each of them has a sequence of interacted fashion items. The following (both u-i and i-i) graph construction illustrations are based on this example. To build the u-i interaction graph, we follow the previous GNN-based recommendation approaches (Wang et al., 2019; He et al., 2020) and build a bipartite graph, as shown in Figure 2 (b). Two types of nodes, namely user nodes and item nodes, as well as the interactions (edges) between the users and items form the graph. The first-order connection is built upon the interaction relationship between users and items. Higher-order connectivity can be derived by combining two or more edges consecutively. For example, $u_{1}$ and $u_{2}$ can be connected in a second-order relation through their connection with item $i_{1}$. Such a second-order connection is meaningful as it suggests common choice and preference. Similarly, items $i_{1}$ and $i_{4}$ are also connected in second-order as they are both interacted with $u_{2}$, which means that the two items may be related in some dimensions, such as with similar style.

Figure 2 (c) illustrates the constructed i-i transition graph for the toy example and is associated with the first-order and second-order adjacent matrices. Higher-order connectivity can be derived by combining multiple consecutive connections. For example, $i_{1}$ and $i_{3}$ are connected in second-order by the middle node $i_{2}$ since $[i_{1},i_{2},i_{3}]$ is a sequence of $u_{i}$. Moreover, $i_{4}$ and $i_{2}$ also have a second-order connection by the middle node $i_{1}$ since $i_{4}$ transits to $i_{1}$ in the interaction sequence of $u_{2}$ and $i_{1}$ transits to $i_{2}$ in the sequence of $u_{1}$. This example indicates that our global graph can model cross-session i-i transitions. For simplicity, we just demonstrate the partial transitions from anchor to target, and the transitions from target to anchor are similar. Note that our design not only models the transition directions but also has better representative capability since we further specify the item representation with respect to its different situations.

After the information propagation over two graphs, we therefore obtain the updated user and item representations. The prediction of the score for the target item $i$ to be the next pick by user $u$ after item $l$ can be computed as:

We apply the pairwise BPR loss (Rendle et al., 2012) to train the model, treating all observed triplets as positive samples and unobserved triplets as negatives. For each positive sample $(u,l,i)$, we randomly sample one negative triplet $(u,l,j)$ with same user $u$ and anchor item $l$. The BPR loss is specifically calculated by:

where $\mathcal{O}={(u,l,i,j)|u\in\mathcal{U},l=s^{u}_{t-1},i=\textbf{s}^{u}_{t},t\in[0,|\textbf{s}^{u}|]},j\notin\textbf{s}^{u}$. $\sigma(\cdot)$ is the sigmoid function; $\Theta$ denotes all trainable parameters in the model, which include user and item embeddings. $\lambda$ is the hyper-parameter controlling the regularization strength.

We evaluate our proposed model based on two real-world e-commerce datasets: iFashion (Chen et al., 2019) and Amazon review datasets (McAuley et al., 2015).

iFashion is a large-scale fashion dataset generated based on the e-commerce platform Taobao¹¹1www.taobao.com. The original dataset contains over three million users and four million items, where each user is interacted with tens of items in chronological order. To make the datasets applicable to all sequential recommendation methods, we crop two sub-datasets which consist of different minimal fixed-length user interaction sequences of 7 and 10 respectively. For each sub-dataset, we randomly sample 50,000 sequences from the interaction sequence list, whose length must be longer than the predefined minimal sequence length. Since one user may have multiple interaction sequences in the dataset, the user number is less than the sequence number. The generated dataset is named as iFashion-SR-7 and iFashion-SR-10 respectively, where iFashion-SR denotes iFashion for Sequential Recommendation.

Amazon review dataset is a large-scale review dataset which consists of the purchase records and reviews of users on Amazon²²2www.amazon.com. We use the Clothing-Shoes-Jewelry subset data (2014 version) to generate our dataset for sequential recommendation. Since the long-tail situation is severe in this dataset, we filter out the items with less than five interactions at the very first step. Similar with the preprocessing for iFashion dataset, we generate two sub-datasets for users with interacted sequence length longer than the minimal predefined length of 7 and 10 respectively.

For each data setting, we split the interaction sequences into three parts: (1) the last interaction in each sequence is used for testing; (2) the second to last interaction in each sequence is used for validation; and (3) all remaining interactions are used for training. The statistics of the datasets are shown in Table 1. For methods modeling one-step transitions (such as FPMC), the previous one item of each interaction is employed to build a pair. For methods need to model the whole historical sequence before the target item (such as SR-GNN), we generate short sub-sequences with the sliding window strategy (Tang and Wang, 2018) from the original sequences.

Baselines: We compare the proposed methods with several competitive and relevant baselines:

• •

  MF (Rendle et al., 2012) factorizes the u-i interaction matrix and is optimized with the BPR loss.

• •

  FMC (Rendle et al., 2010) focuses on modeling the sequential dynamics by factorizing the i-i transition matrix.

• •

  FPMC (Rendle et al., 2010) models both the personalized u-i interaction and the ‘global’ i-i transition by MF and FMC respectively.

• •

  NGCF (Wang et al., 2019) leverages a bipartite graph to model the u-i interaction, which is a typical GCN-based recommendation model.

• •

  LightGCN (He et al., 2020) is the light version of NGCF and utilizes a light graph convolution kernel.

• •

  GRU4REC (Hidasi et al., 2015) is an RNN-based session-based recommendation method which models the interaction sequences with GRUs.

• •

  Caser (Tang and Wang, 2018) is a session-based recommendation method which treats the embedding of fixed length of historical interacted items as an “image” and models the interaction sequences with CNNs.

• •

  SR-GNN (Wu et al., 2019a) is a GNN-based method which leverages the session-level i-i transition graph to enhance the session representation.

Table 2 lists the properties of these methods in terms of whether being personalized or sequence-aware and whether considering the user-item or the item-item information propagation. It offers an intuitive illustration of the difference of those methods.

Table 3 shows the overall performance of all compared methods on the iFashion-SR and Amazon-Fashion datasets. We have the following observation based on the results.

(2) Compared to the basic FPMC, which also models both user-item interaction and item-item transition, our method achieves around 10% improvement in terms of NDCG for both settings due to the incorporation of two types of global graph.

(3) LightGCN shows the best performance among all baselines on both settings, while the simple MF and FPMC are competitive too according to the results. Such results may imply that the collaborative signals (CF) in this dataset are quite significant. It also shows that modeling u-i interaction is effective on this dataset.

(4) Session-based methods, including GRU4REC, Caser and SR-GNN, do not show superior performance as expected. The reasons are multi-fold. First, most of them (except Caser) ignore user information in the model. Second, the training samples are less for them as they require to input multiple historical interactions. Third, they ignore the effect among the connected users and items in higher orders.

(2) Overall, session-based methods perform better than methods such as FMC, which suggests that the i-i transition signals are harder to explore for this dataset. Since session-based methods are better in exploring the dependency among items by considering the item transition sequences rather than the pairs in the model, they achieve more desired performance.

(3) As the i-i transition signals are less effective to model than the u-i interaction, our DGSR, exploring more high-order i-i transitions, might affect the exploring o u-i patterns instead, which degrades the overall performance of DGSR.

In this part, we conduct several ablation studies to discuss more detailed technical components in DGSR. The ablation study focuses on three aspects: 1) the effect of incorporating two types of graph; 2) the effect of the layer numbers ($k$) in each graph; and 3) the effect of embedding size.

From the results we can observe that: First, either incorporating u-i graph or i-i graph can improve the performance on iFashion-SR dataset. On Amazon-Fashion, the u-i graph seems not helpful from the comparison between the models with/without u-i graph (the first two rows), while the i-i graph brings significant improvement (comparing first and third rows). As discussed in the above section, modeling i-i transition is hard on Amazon-Fashion, which may even degrade the performance when combining with the u-i interaction modeling (comparing results of MF, FMC and FPMC in Table 3). When we further strengthen the u-i modeling, it leads to more conflicts between u-i and i-i modeling. However, when we apply both graphs, the performance of the method on all settings boosts; this is obvious on Amazon-Fashion that the i-i transition modeling significantly improve the overall recommendation performance.

From the results we have the following observations. 1) The performance of all three basic models are improved by leveraging various graphs. The best layer number setting for different models are different. 2) For the FMC model, the performance increases with the increase of graph layers. This demonstrates that higher-order connectivity is important to improve the i-i transition pattern modeling. 3) For MF+UI and FPMC+UI, the best performance is achieved when one-layer information propagation is conducted, which shows that stacking more graph layers may over-smooth the representation of users and items and degrade the performance instead. 4) When leveraging both graphs, we can see that the performance steadily improves with the increase in layer number, which shows that the u-i graph and i-i graph organically corporate with each other and contribute to the final interaction prediction modeling.

In this section, we first discuss the sparsity problem by showing the performance of DGSR and other three baselines on testing samples with different levels of sparsity. From Figure 5, we can see that generally, all methods perform better on dense samples and worse on sparse ones, which is consistent with our hypothesis. The DGSR method outperforms the others in the whole density range, but the superiority is more significant on sparse data (items interacted lower than 10). Such a result demonstrates that our proposed DGSR method can effectively tackle the data sparsity problem as claimed.

We further analyze the higher-order connectivity in the data and the specific effect of it on the prediction and recommendation. We illustrate two cases in Figure 6 and show the target user, the previously interacted item, the target item, as well as the multiple connectivities related to the three subjects in each case. From the examples, we can see that our model is able to rank the target item higher in both cases with the two graphs. By analyzing more details, we find that the target items in both cases are connected with other user and items by interaction or transition relationships. In the first case, the target item b5b2ab is connected to another user 0d0f6a with the u-i graph, who picks item 232012. As item 232012 is also picked by the target user f6e920, the target user and target item are connected in third-order. With the message passing and aggregation operations, the representation of the target user and item will be closer, so that the final interaction probability predicted by the model will be higher. The higher-order connectivity in i-i graph works in a similar way, which can help improve the prediction of the transition probability from the anchor item to the target item. In general, we can see that with two graphs, the target item in two cases can be ranked higher.