session-based recommendation with hypergraph attention

Session-based Recommendation with Hypergraph Attention Networks

Jianling Wang∗ Kaize Ding†† Ziwei Zhu∗ James Caverlee∗

AbstractSession-based recommender systems aim to improve recom-mendations in short-term sessions that can be found acrossmany platforms. A critical challenge is to accurately modeluser intent with only limited evidence in these short ses-sions. For example, is a flower bouquet being viewed meantas part of a wedding purchase or for home decoration? Suchdifferent perspectives greatly impact what should be recom-mended next. Hence, this paper proposes a novel session-based recommendation system empowered by hypergraphattention networks. Three unique properties of the proposedapproach are: (i) it constructs a hypergraph for each sessionto model the item correlations defined by various contex-tual windows in the session simultaneously, to uncover itemmeanings; (ii) it is equipped with hypergraph attention lay-ers to generate item embeddings by flexibly aggregating thecontextual information from correlated items in the session;and (iii) it aggregates the dynamic item representations foreach session to infer the general purpose and current need,which is decoded to infer the next interesting item in the ses-sion. Through experiments on three benchmark datasets, wefind the proposed model is effective in generating informa-tive dynamic item embeddings and providing more accuraterecommendations compared to the state-of-the-art.

1 Introduction

Recommendation systems are ubiquitous, acting asan essential component in online platforms to helpusers discover items of interest. In many practicalscenarios, a recommendation system needs to inferthe next interaction for a user based only on theshort-term prior interactions within a particular session.Previous approaches that learn a static model of users[10, 11, 22, 31, 34] or rely on long-term user behaviors[9, 17, 28, 30] are not well-suited for such scenariosand could lead to poor predictive power. In contrast,session-based recommendation has attracted increasingattention, with recent approaches showing promisingperformance in inferring user interests in these (oftenanonymous) short-term sessions [16, 19, 21, 23, 32].

A critical issue is how items are treated in suchsession-based recommendation approaches. The indi-vidual items can reveal user intent, but they only pro-vide limited evidence. To illustrate, consider the exam-ple in Figure 1. In different sessions, the same flowerbouquet can be viewed differently, i.e., as part of a wed-ding party purchase, an option for home decoration, or

∗Texas A&M University, {jlwang, zhuziwei, caver-

lee}@tamu.edu†Arizona State University, [email protected]

Figure 1: An example with three sessions. The sameflower bouquet appears in each session, but with adifferent purpose in each. The righthand side showscontextual windows of size 2 and 3 including this flowerbouquet. Items in the same contextual window arecorrelated; and different contextual windows may havedifferent levels of importance for characterizing an item.

a gift for Mother’s Day. However, if we independentlyconsider Flower Bouquet A, it might be viewed as ex-actly the same item across sessions. In a sense, themeaning of an item (and what it reveals about userintent) could be inferred from contextual windows,each of which contains a set of consecutive items show-ing up together within a session. In Session C, bouquetA is likely for a wedding party since the user clicks itright between other wedding-related items, while thebouquet in Session A may be a Mother’s Day gift sinceit is clicked along with items specified for “Mom.”

Hence, we propose to exploit these contextual win-dows to model session-wise item representations thatcan robustly capture user intent with only limited ev-idence available in short sessions. However, there areseveral key challenges in eliciting the user intent sig-nal among items from the contextual windows in eachsession: (1) conventional graph structures and graphneural networks [2, 3, 7, 8, 15] are designed to modelthe pairwise connections between items, which are notalways sufficient since we need to consider contextualwindows connecting various numbers of items rangingfrom two to many. For example, for Session B in Fig-ure 1, the pairwise linkage between bouquet A and B isnot enough to reveal that bouquet A is meant for homedecoration. But such evidence could be inferred by an-alyzing the triadic relations among both the bouquets

Copyright c© 2021 by SIAMUnauthorized reproduction of this article is prohibited

and a vase as defined by Contextual Window 1. To thisend, we adopt the hypergraph [1, 4, 6, 29] structure tomodel the correlations amongst items in different con-textual windows within each session. In a session hy-pergraph, each node denotes an item, and a hyperedgeconnects the collection of items that show up togetherwithin a contextual window. This hypergraph structuresupports capturing correlations among items defined bycontextual windows, which could be arbitrary-order de-pending on the usage scenario; (2) while propagatingand aggregating the user intent evidence within a ses-sion hypergraph, some items are informative, but oth-ers may not be. Moreover, different contextual windowsmay bring in different levels of evidence for how itemsare represented. Thus, a key challenge is how to care-fully highlight the informative items on each hyperedgeand also emphasize the evidence from hyperedges withlarger impacts.

To tackle the aforementioned challenges, we proposeSHARE: a Session-based Hypergraph Attention Net-work for REcommendation. Specifically, the proposedSHARE has three unique characteristics:

• First, it captures contextual information with slid-ing windows, whereby items appearing in the samewindow are connected with a hyperedge. By ap-plying multiple sliding windows on the session se-quence, it is able to model a session consideringcontextual windows of varying sizes simultaneouslyin the hypergraph structure.

• Second, it incorporates a carefully-designed hyper-graph attention network to extract the user intentevidence from contextual windows, which is ableto pay more attention on the informative items(nodes) and also emphasize the evidence from con-textual windows (hyperedges) with larger impacts.

• Third, the session-wise item embeddings resultingfrom a stack of hypergraph attention layers canthen be fed into a self-attention layer to inferboth the general intent and current interests in thesession, which are decoded jointly to generate thenext-item recommendation for this session.

With experiments on three benchmark datasets forsession-based recommendation, we show that such ahypergraph-based approach is better suited than con-ventional graphs for modeling item correlations withinsessions. The proposed SHARE is effective in predictingthe next interesting items, and significantly outperformsthe state-of-the-art in session-based recommendation.

2 Related Work

Session-based Recommendation. Given sequentialuser behaviors in a session, session-based recommen-

dation systems aim to infer the subsequent behavior.Since these sessions usually exclude login informationand take place in the short-term (e.g., less than a halfhour) where user intent is local to the session, previousworks that elicit user preferences from long-term historicbehaviors or generate user embeddings based on theiridentification [11, 28, 30] are not well-suited to providingrecommendations for these anonymous short-term ses-sions. Though matrix factorization-based and neighbor-based methods can be applied for session-based recom-mendation [18, 24], they ignore the transitional patternsbetween items. Thus, FPMC [23] has been proposed toextend matrix factorization with Markov Chains, whichis able to predict the next item considering the tran-sition from the last item. With the advancement ofneural networks, different neural mechanisms have beenapplied to handle sequential session data. In [12, 13],the authors adopt recurrent neural networks to learnsequential patterns from all sessions and infer the nextitems with the output of the last layer. NARM [16] uti-lizes an attention layer to aggregate items in the sessionand capture the main purpose of each session. Fur-thermore, STAMP [19] designs another attention com-ponent to emphasize the short-term interest in the ses-sion. To capture the local dependencies between items,SR-GNN [32] applies the conventional graph neural net-work to model the transitions between items in a sessiongraph. In contrast, we propose to utilize hypergraphsand an attention mechanism to model item correlationsand user intent with the contextual windows in session-based recommendation.

Graph-based Recommendation. Since graphs canbe a good fit for modeling the interactions between item-item, user-user, or user-item, there are works in develop-ing recommendation systems centering around differentgraph structures. Translation-based recommendationsystems [9, 20] treat all the items as nodes and users asthe connections between items consumed subsequently.These models embed users and items into a similarspace by minimizing the translation loss, which requiresclear user identification (which is often not available insession-based scenarios). Recently, several efforts ap-ply newly introduced Graph Neural Networks (GNNs)for recommendation [2, 31, 33], in which different GNNmodels have been designed for representation learningin graph structured data. GCMC [2] consists of neuralgraph autoencoders to reconstruct the user-item ratinggraph. And in NGCF [31], the authors propose to con-struct a user-item bipartite graph and utilize multiplegraph neural layers to capture the multi-hop collabora-tive signals between users and items. In social recom-mendation, the relations between users can be exploited


with the social graph [5, 32]. For example, DiffNet [5]adopts a Graph Convolutional Network to model thediffusion of user embeddings among their social con-nections. Unlike SHARE, these methods are built onconventional graph structures (not hypergraphs). Andthey are not designed for the special characteristics ofsession-based recommendation, like sequential user ac-tions in short-lived sessions without user identification.

3 The Proposed Model - SHARE

In this section, we propose a novel session-based rec-ommendation model with Hypergraph Attention Net-works to exploit the contextual windows within eachindividual session. In the following, we begin with theproblem setup and then structure the design of SHAREaround three research questions: (i) Given that itemsfalling into the same contextual window are correlated,how do we construct a hypergraph for each session tomodel the correlations among items from a variety ofcontextual windows simultaneously? (ii) Consideringthese contextual windows, how do we update an itemembedding with the user intent evidence that propa-gates in and across the contextual windows? (iii) Withthese session-wise item embeddings, how can we inferthe next interesting item by extracting both the generalinterest and current need in the session?

3.1 Problem Setup. In session-based recommenda-tion [16, 21, 32], given the sequence of items which havebeen interacted within a session, the goal is to predictthe next item. Let I = {i1, i2, ..., iN} denote the set ofN unique items in the system. These items start witha set of embeddings {i1, i2 ..., iN}, each of which is atrainable embedding associated with the item ID. Ananonymous session s consisting of a sequence of t ac-tions (i.e., the items interacted within the session) canbe denoted by s = [is,1, is,2,..., is,t], in which the itemsare sorted in chronological order and is,p represents thepth item interacted within the session. A session-basedrecommendation system should predict the next possi-ble action (i.e., is,t+1) based on the previous t actions.That is, we want to generate the preference scores forall the items in I based on the sequence of actions in ses-sion s. Then the top-K preferred items can be treatedas candidates for recommendation.

3.2 Session Hypergraph Construction. With asequence of interacted items in session s, we proposeto model the session as a hypergraph before exploitingthe item correlations in different (possibly, overlapping)contextual windows. Let Gs = {Vs, Es} denote thehypergraph constructed from session s, in which thenode set Vs consists of all the unique items appearing in

this session. Each hyperedge e ∈ Es will connect all theitems falling into the specific contextual window. As inFigure 2, we can apply a sliding window with size won the item sequence for the session to identify all ofthe contextual windows of size w in this session, withwhich the items appearing in the same window can betreated as items falling into the same contextual windowand thus will be connected with a hyperedge. Hence,to exploit contextual windows of varying sizes in thesession, we can apply sliding windows of varying sizes onthe item sequence. Let Ews represent the collection of allthe hyperedges constructed with such a sliding windowof size w on session s. Then we gather hyperedges basedon different sliding windows together to be the set ofhyperedges Es for session s with Es = E2s ∪ E3s ∪ ...EWs ,in which W is the maximum size of contextual windowsthat we consider in the model. We repeat this process toconstruct the unique hypergraph for each of the sessions.

3.3 Hypergraph Attention Networks. With thehypergraph structure, we want to learn the represen-tations for nodes (items) considering the correlationsamong nodes (items) defined by different hyperedges. Inthe following, we introduce a novel hypergraph attentionnetwork (HGAT) which is able to aggregate the user in-tent evidence that propagates in and across contextualwindows while updating the node representations in ahypergraph. There are recent works extending neuralnetworks from a conventional graph to a hypergraph bygeneralizing the convolution operation [1, 6, 29], withwhich the neighboring node features will be firstly ag-gregated to the common hyperedges and then propa-gated to the node. They usually treat nodes equallywhile characterizing the hyperedge or use a pre-definedweight matrix to model the importance of informationpropagated via different hyperedges. However, in ses-sion hypergraphs, for nodes belonging to the same con-textual window, some of them may be informative inrevealing user intent, but others may not be. And theevidence propagating via different hyperedges (contex-tual windows) may bring in different levels of impacts toan item. Thus, we require a new representation learningprocess for hypergraphs to capture these special needs.

To tackle this problem, we propose to generalizethe attention mechanism for hypergraphs so that wecan highlight the informative items on each hyperedgeand also emphasize the evidence from hyperedges withlarger impacts. In the following, we will explain thenode representation learning process with the proposedhypergraph attention layer (see Figure 2) in two steps:

Node to Hyperedge. Since information can propa-gate among neighboring nodes via hyperedges, they arethe key factor for node representation learning in hyper-


Hypergraph Attention Layers (HGAT)

E3sE2

s EWs

…

1 2 3 9…

Gs = (Es, Vs)Gs = (Es, Vs)Gs = (Es, Vs)

Sliding Window

s

…

1

2

3

4

4

9

8

…

…

1

2

3

4

4

9

8

…

1

2

3

4

4

9

8

…

w = 2

1

2

3

4

4

9

8

…

…

1

2

3

4

4

9

8

…

1

2

3

4

4

9

8

…

w = 3

1

2

3

4

4

9

8

…

Session Hypergraph

1

2

3

4

4

9

8…

6

3

1

…

Top-K Ranked

ListSession-wise

Item embeddings

13

4

1

8

…

…

…

…

✏1

✏2

✏m

↵11↵14

↵19 ↵18

↵21

↵22

↵23

↵m4

↵m5

�11

�12

�13

�14

�22

�23

�25

�q8 �q9

e(l)1

e(l)2

e(l)m n(l)

q

n(l)1

n(l)2

9

2

4

5q

2

1

l = 1

Session Hypergraph Construction Next-item Prediction

Self-Attention

Softmax

13

4

1

8

…

…

…

…

✏1

✏2

✏m

↵11↵14

↵19 ↵18

↵21

↵22

↵23

↵m4

↵m5

�11

�12

�13

�14

�22

�23

�25

�q8 �q9

e(l)1

e(l)2

e(l)m n(l)

q

n(l)1

n(l)2

9

2

4

5q

2

1

l = 2

13

4

1

8

…

…

…

…

✏1

✏2

✏m

↵11↵14

↵19 ↵18

↵21

↵22

↵23

↵m4

↵m5

�11

�12

�13

�14

�22

�23

�25

�q8 �q9

9

2

4

5q

2

1

l = L

…

0e(L)1

e(L)2

e(L)m

n(L)1

n(L)2

n(L)q

Figure 2: The structure of SHARE: It applies multiple sliding windows to capture the contextual informationto construct a hypergraph. With a well-designed HGAT network, it is able to generate the item embeddingsrevealing their meanings in the specific session. The sequence of session-wise item embeddings is fed into theself-attention layer to generate an embedding for the session, which is decoded for the preference scores on items.

graph. With the special hypergraph structure, insteadof directly updating each node with the the neighboringnode information, we firstly treat each hyperedge as aninterlayer between nodes and aggregate all the informa-tion propagating via the hyperedge.

We take the graph Gs constructed by session s asan example. The operation can be applied on hyper-

graphs for other sessions. Let {n(0)1 ,n

(0)2 , ...,n

(0)p } =

{i1, i2, ..., ip} denote the node input for the first HGATlayer, which is the initial item embeddings for the set ofunique items {i1, i2, ..., ip} appearing in Gs. Since somenodes on a hyperedge are informative but others maynot be, we should pay varying attention on the informa-tion from these nodes while aggregating them together.

Let m(1)t∼j denote the information propagating via hy-

peredge εj from node t on the (1)th HGAT layer. Wecan aggregate the information from each of the nodesconnected by εj with the attention operation to gener-

ate the representation e(1)j as:

(3.1)

e(1)j =

∑t∈Nj

m(1)t∼j m

(1)t∼j = αjtW

(1)1 n

(0)t

αjt =S(W

(1)

1 n(0)t ,u(1))∑

f∈NjS(W

(1)

1 n(0)f ,u(1))

in which Nj denotes all the nodes connected by hyper-edge εj and u(1) represents a trainable node-level con-

text vector for the (1)th HGAT layer. W(1)1 and W

(1)

1

are the transform matrices and αjt denotes the atten-tion score of node t on hyperedge εj . We use a functionS(·, ·) to calculate the similarity between the node em-bedding and context vector. Empirically, we use ScaledDot-Product Attention to calculate the attention scores[14, 26], which is defined as:

(3.2) S(a,b) =aTb√D

where D is the dimension size and can be used fornormalization while calculating the similarity scores.

Hyperedge to Node. To update the embeddingfor a node, we need to aggregate the contents fromall its connected hyperedges. Similarly, we utilize theattention mechanism to perform the aggregation inorder to model the significance of different hyperedges.

Let m(1)j→t denote information (user intent evidence)

from hyperedge εj to node t. Given the set of hyperedgesYt which are connected to node t, its update embeddingis calculated as:

(3.3)

n(1)t =

∑j∈Yt

m(1)j→t m

(1)j→t = βtjW

(1)2 e

(1)j

βtj =S(W2e

(1)j ,W

(1)3 n

(0)t )∑

f∈YtS(W

(1)

2 e(1)f ,W

(1)3 n

(0)t )

where W(1)2 , W

(1)

2 and W(1)3 are the trainable matrices

used to transform the vector before calculating theattention scores for the (1)th HGAT layer. βtj indicatesthe impact of hyperedge εj on node t. As in the laststep, we use the Scaled Dot-Product Attention formuladefined by Equation 3.2 to calculate S. The resulting

n(1)t can be treated as the updated embedding for note t

by aggregating information from its neighboring regionin the hypergraph.

High-order Propagation. While a single HGAT layercan capture the information from direct neighbors, weconstruct a Hypergraph Attention Network by stackingmultiple HGAT layers to model the multi-hop high-order information propagation in the hypergraph. Inthis Hypergraph Attention Network, the output of the(l − 1)th HGAT layer is the input for the lth layer.Thus the outputs (i.e., node embeddings) from the lastlayer (L) inherit the contextual information from all theprevious layers and can be used to characterize the items


in this session. For each of the session hypergraph, suchan Hypergraph Attention Network is able to generatesession-wise item embeddings to reflect user intent inthe session, by highlighting the informative items oneach hyperedge and also emphasizing the evidence fromhyperedges with larger impacts.

3.4 Next-Item Prediction. To infer the next in-teresting item, our goal is to generate an embeddingwhich can encode both the general interest and the cur-rent need of the session. We adopt the idea from self-attention [14, 26] to achieve this goal. Since the generalinterest can be uncovered by the aggregation of all theitems and the current need is revealed by the last item,we want to aggregate the items in the session while pay-ing more attention on those items which are highly cor-related to the last one. Thus, we can treat the last itemin the session as the query and the sequence of itemsin the session as both keys and values, leading to thedesign of the self-attention layer in Figure 2.

For a session interacted with [is,1, is,2, ..., is,t] se-quentially, we will lookup the corresponding node em-beddings from the output of the hypergraph attention

networks to get [n(L)s,1 ,n

(L)s,2 , ...,n

(L)s,t ]. We will transform

the item embeddings with WQ, WK and WV to gen-erate the query vectors, key vectors, and value vectorscorrespondingly. Then we can aggregate the sequenceof embeddings with:

(3.4)

hs =∑i≤t

σtiWV n(L)s,i

σti =S(WQn

(L)s,t ,WKn

(L)s,i )∑

j≤t S(WQn(L)s,t ,WKn

(L)s,j )

in which the attention scores S is defined in Equation3.2. According to observations in previous research[14, 32], the order of items is less likely to be related tothe general interest in short-term sequences and couldintroduce noise in modeling such short-term sequences.We omit the order information while handling the itemsequences in session-based recommendation.

With this carefully-designed self-attention layer, theresulting hs encodes both the general interest and thecurrent need for session s. Then we compute themultiplication between hs and the latent factor of itemv to predict the preference score of session s on vusing psv = hT

s iv. Let ps = [ps1, ps2, ..., psN ] denotethe predicted preference scores of session s on all ofthe N items in the system, which will be processedwith a Softmax layer to generate the final scores suchthat: ys = softmax(ps). We use a one-hot vectorys = [ys1, ys2, ..., ysN ] to denote the ground-truth itemof session s. During the training phrase, we calculate

# Sessions(Training)

# Sessions(Testing)

# ItemsAvg. Lengthof Sessions

Y ooChoose1/64 369,859 55,898 16,766 6.16Y ooChoose1/4 5,917,745 55,898 29,618 5.71Diginetica 719,470 60,858 43,097 5.12

Table 1: Dataset Statistics.

the cross-entropy loss for all training sessions Strain

with L = −∑s∈Strain

∑Nv=1 ysv log ysv. Then we can

train the model using back-propagation. In the testingphase, given an unseen session, we can construct a newhypergraph and feed it into the hypergraph attentionnetwork. And then we can calculate its preference scoreson all the items and recommend the items ranked amongthe top.

4 Experiments

In this section, we conduct several experiments toevaluate how the proposed SHARE model performs insession-based recommendation.

4.1 Data. We adopt two public datasets that havebeen widely used to evaluate session-based recommen-dation: YooChoose and Diginetica [13, 16, 21, 32]. Yoo-choose contains sessions of click events from an onlineretailer in Europe and was originally published as partof the 2015 RecSys Challenge. Diginetica contains ses-sions of product transaction data from an online retailerand was released as part of the 2016 CIKM Cup.

We keep sessions with length longer than 1 anditems which appear in at least 5 sessions. For Yoo-choose, we test on all the sessions happening on the lastday in the dataset while training all the sessions be-fore that. Only items appearing in the training set areconsidered. As for Diginetica, we split the dataset anduse the sessions happening in the last 7-days for testing.Furthermore, we adopt a standard sequence preprocess-ing method used in previous work [19, 25, 32] to generatethe session sequences and labels. Then we get the finalDiginetica dataset as described in Table 1. However, itis not necessary to train on the entire set of training se-quences from YooChoose since it is extremely large andtraining only on a fraction of recent sessions can leadto better prediction performance based on the experi-mental results in [25]. As in [16, 19, 32], we sort all ofthe training sequences generated from YooChoose, andretrieve the most recent 1/64 and 1/4 to be the trainingsamples in Y ooChoose1/64 and Y ooChoose1/4 (listed inTable 1). Note that Y ooChoose1/64 and Y ooChoose1/4share the same set of testing samples. In addition, forfair comparison, the training samples and testing sam-ples in all of the three datasets are exactly the same asin [16, 19, 21, 32].


Datasets S-Pop MF FPMC KNN GRU4Rec+ NARM STAMP RPNet SR-GNN SHARE

Y ooChoose1/64Hit@20 30.44 31.31 45.62 51.60 67.84 68.32 68.74 69.13 70.57† 71.51∗

MRR@20 18.35 12.08 15.01 21.81 29.00 28.63 29.67 30.24 30.94† 31.45∗Y ooChoose1/4

Hit@20 27.08 3.40 51.86 52.31 69.11 69.73 70.44 70.71 71.36† 72.25∗MRR@20 17.75 1.57 17.50 21.70 29.22 29.23 30.00 31.03 31.89† 32.11∗

DigineticaHit@20 21.06 5.24 26.53 35.75 46.16 49.70 45.64 47.79 50.73† 52.73∗

MRR@20 13.68 1.98 6.95 11.57 14.69 16.17 14.32 17.66† 17.59 18.05∗

Table 2: Comparison of Different Models. All the results are in percentage (%). The best performing method ineach column is boldfaced, and the second best method is marked with †. ∗ indicates that the improvement of thebest result is statistically significant compared with the next-best result with p < 0.05.

4.2 Experimental Setup

4.2.1 Evaluation Metrics. We aim to evaluate howeach model performs in predicting the next item in eachsession. Thus, the experiments follow the leave-one-out setting with one ground-truth item to be tested foreach session in the test set. As in previous works forsession-based recommendation [16, 21, 32], we adoptboth Mean Reciprocal Rank (MRR@K) and Hit Rate(Hit@K) as evaluation metrics. Given the ranked list ofitems predicted for each session, Hit@K measures theprobability that the ground-truth item is within thetop-K. Let rs denote the ranking of the ground-truthitem for session s. Then, Hits@K = 1 if rs ≤ K andHits@K = 0 otherwise. As for MRR@K, it measuresthe average ranking of the ground-truth items amongthe lists. That is MRRs@K = 1

rsif rs ≤ K otherwise

MRRs@K = 0. Then we take the average values ofMRR and Hit Rate over all the sessions in the test setand report the results.

4.2.2 Baselines.

• S-Pop: This simple baseline recommends the mostpopular items based on their popularity in thecurrent session. Ties are broken up using thepopularity values based on the whole training set.

• KNN [24]: This method recommends items whichare most similar to items clicked in the currentsession. Each item is represented with a binaryvector indicating all the sessions it appears in andcosine similarity is used to define their similarity.

• MF [22]: This matrix factorization-based model istrained with Bayesian personalized ranking loss.

• FPMC [23]: Building on top of both MatrixFactorization and Markov Chains, it is able to inferthe next item based on sequential behaviors.

• GRU4Rec+ [12]: This baseline utilizes Gated Re-current Unit (GRU) to model sequential patterns,with a new ranking loss functions tailored to RNNsin session-based recommendation.

• NARM [16]: Besides learning the sequential be-havior with GRU, it also includes an attention layerto extract the session’s main purpose, which arecombined together to infer the preference scores.

• STAMP [19]: This model combines both generalinterests from long-term memory and current inter-ests from the short-term memory of the last-clicksin session-based recommendation.

• RPNet [21]: Considering the repeat consump-tion in each session, this model can recommendunclicked items in the explore mode while recom-mending repeated items in the repeat mode.

• SR-GNN [32]: This is the state-of-the-art forsession-based recommendation with graph neuralnetworks. It models session sequences as graph-structured data and designs a graph neural net-work to capture the transition between items. Rec-ommendation is made with the composition of thewhole session and the last click.

4.2.3 Parameter Tuning. All of the experimentsare conducted on a server machine equipped with a 12GB Titan Xp GPU. We use Adam as the optimizer witha learning rate set to be 0.001 for all datasets. The batchsize for Y ooChoose1/64 is 100. Since there are moretraining sessions in both Y ooChoose1/4 and Diginetica,we set their batch size to be 300. For a fair comparison,the dimension of the item embedding is set to be 100 asin the baseline methods. We grid search for the dropoutrate in {0.1, 0.2, 0.3, 0.4, 0.5, 0.6} and L2 regularizationin {10−5, 10−4, 10−3, 10−2, 10−1}. The dropout is setto be 0.3 and the L2 regularization is set to be 10−6 forall the datasets. We train each model for 50 epochs oruntil its performance does not improve for the validationset after 5 epochs. We fine-tune the maximum size ofthe sliding window in {2, 3, 4, 5, 6} and the number ofHGAT layer in {1, 2, 3, 4} for different datasets. As inprevious works, we randomly sample 10% of the trainingsessions as validation for parameter tuning.

4.3 Overall Evaluation. To compare SHARE withthe baseline models, we summarize the overall results


MethodsY ooChoose1/64 Y ooChoose1/4 Diginetica

Hit MRR Hit MRR Hit MRR@10 @20 @10 @20 @10 @20 @10 @20 @10 @20 @10 @20

(1) w/o Hypergraph 59.79 70.45 29.46 30.21 60.13 70.79 29.77 30.52 37.57 50.52 16.17 17.06(2) + GAT 60.25 70.88 29.77 30.59 60.60 71.22 30.54 31.29 38.36 51.54 17.75 17.61(3) + HyperGCN 60.93 71.29 30.65 31.37 60.98 71.73 30.58 31.33 39.32 52.39 16.83 17.73SHARE 61.13∗ 71.51∗ 30.67∗ 31.45∗ 61.52∗ 72.25∗ 30.96∗ 32.11∗ 39.52∗ 52.73∗ 17.12∗ 18.05∗

Table 3: Ablation Test Result. All the results are in percentage (%). ∗ indicates that the improvement of thebest result is statistically significant compared with the next-best result with p < 0.05.

in Table 2. As in previous work [16, 19, 32], we reportMRR and Hit Rate at K=20 for the overall comparisonwith the baselines. We find that SHARE outperformsall of the baselines under each of the metrics for session-based recommendation over all of the datasets.

Starting from the simplest method, since MF isunable to capture sequential information, it performspoorly in session-based recommendation. Integratingwith Markov Chains, FPMC can improve MF, but itis still not a good fit for modeling the short-term userbehaviors and performs worse than KNN. With ad-vances in neural networks, we observe that GRU4Rec+achieves much better results than the non-neural mod-els in sequential pattern modeling. However, in compar-ison to GRU4Rec+, we see that NARM and STAMP arebetter suited to extract the general interests from eachshort-term anonymous session by aggregating itemswith weighted attention scores. Emphasizing the itemclicked most recently helps STAMP achieve better per-formance than NARM. Furthermore, by predicting andmodeling the repeat consumption in sessions, RPNetcan outperform the attention-based and GRU-basedmodels in this recommendation scenario.

As the state-of-the-art for session-based recommen-dation, SR-GNN outperforms RPNet, verifying the ef-fectiveness of modeling item transitions with appropri-ate graph structure to obtain accurate user represen-tations in sessions. However, the conventional graphstructure is insufficient in capturing the correlationsamongst items across different contextual windows.Since SHARE is specifically designed with this challengein mind, we observe that it achieves significant improve-ments compared with SR-GNN. Further, we observethat the performance of SHARE on Y ooChoose1/64 iseven better than SR-GNN on Y ooChoose1/4. ThoughY ooChoose1/64 and Y ooChoose1/4 contain differentnumbers of training sessions, they are tested on exactlythe same set of sessions. That is, with much less train-ing data, SHARE can recommend more accurately thanthe state-of-the-art model trained with more data.

4.4 Ablation Analysis. To examine the design ofthe proposed model, we conduct an ablation test to com-

1 2 3 4 5 6 7Yoochoose

60.25

60.50

60.75

61.00

61.25

61.50

Hit@

10(%

)

1 2 3 4 5 6 7Diginetica

37.50

38.00

38.50

39.00

39.50

Figure 3: Size of contextual windows vs. Hit Rate.

pare SHARE with three of its variants and report theresults in Table 3. In Model (1), we remove the Hyper-graph attention network from SHARE, meaning that aset of static item embeddings are directly fed into theself-attention layer for different sessions. Instead of uti-lizing the hypergraph structure, Model (2) constructs aconventional graph for each session based on pairwiseitem co-occurrence within the session and use a GraphAttention Networks (GAT) [27] to generate the dynamicitem embeddings. In Model (3), we train a hypergraphconvolution networks [6] instead of HGAT network togenerate the dynamic item embeddings. To comparethem under different conditions, we report their per-formance under both K = 10 and K = 20. Overall,SHARE can outperform all of its variants for both Kvalues, indicating the effectiveness of its design.

Without the hypergraph component, though it in-cludes the self-attention layer to learn the sequentialpatterns, Model (1) achieves the weakest results sinceit does not consider the item correlations within eachsession and uses the same embedding for an item acrossdifferent sessions. Then in Model (2), building on topof Model (1), we adopt the conventional graph struc-ture and GAT networks [27] to obtain dynamic itemrepresentations considering the consecutive transitionsof items in each session. This model can outperformmodel (1), illustrating the necessity of modeling the in-session item correlations with a graph-based structure.Next, to take advantage of the hypergraph structure formodeling the complex item correlations in each session,we construct a hypergraph for each session followingthe same procedure as in SHARE. But in Model (3), wegenerate the item embedding with hypergraph convolu-


0 1 2 3 4Yoochoose

60.25

60.50

60.75

61.00

61.25

61.50

Hit@

10(%

)

0 1 2 3 4Diginetica

37.50

38.00

38.50

39.00

39.50

Figure 4: Number of HGAT Layers vs. Hit Rate.

tional networks [6], which aggregates information fromthe neighboring hyperedges with the convolutional op-eration. It can achieve better performance than usingconventional graphs, which shows the effectiveness ofthe hypergraph structure and the necessity of modelingthe contextual windows for item representation learn-ing. However, it still falls behind SHARE since it doesnot consider the informativeness of nodes and the dif-ferent impacts from contextual windows.

4.5 Study of SHARE. In this section, we conductfurther studies on three core factors in SHARE: size ofthe contextual window, depth of information propaga-tion and length of the sessions. In the following, due tothe limit of space, as Y ooChoose1/64 and Y ooChoose1/4shares the same set of testing sessions, we show the re-sults trained with Y ooChoose1/4 to uncover the pat-terns for YooChoose.

Size of contextual window. We first examine theperformance of SHARE by varying the maximum sizeof contextual windows that we consider while construct-ing the hypergraphs (in Section 3.2). We use the 1-layerHGAT network to avoid the influence of high-order in-formation propagation. Note that maximum size = 1is similar to the case that no hypergraph is constructedfor each session, meaning that items use static embed-dings across sessions. As in Figure 3, for YooChoose,at the beginning, the performance of SHARE improvesas contextual windows with larger size are incorporatedinto the hypergraphs. In Diginetica, the best choice ofwindow size is smaller that in YooChoose. The reasoncould be that the length of sessions is shorter in Digi-netica, and thus we need only consider smaller contex-tual windows. These observations show that contextualwindows can have significant contribution on character-izing items in each session and the proposed frameworkis effective in modeling various contextual windows si-multaneously.

High-order Propagation. We can stack multipleHGAT layers to model the information flowing amongitems with high-order connections in hypergraph. Tovisualize the high-order information propagation, whilesetting the maximum window size to be constant (i.e.,

1 2 3 4 5 6 7 8 9 10YooChoose

68.0

70.0

72.0

74.0

Hit@

20(%

)

SHARESR-GNN

1 2 3 4 5 6 7 8 9 >10Diginetica

44.0

46.0

48.0

50.0

52.0

54.0

56.0SHARESR-GNN

Figure 5: Session Length vs. Recommendation Quality.

2), we show the resulting Hit Rate in Figure 4 by varyingthe number of HGAT layers in SHARE. In each dataset,starting with a single HGAT layer, we find that theperformance is improved by stacking one more layer,indicating the importance of modeling the informationvia high-order connections. For Diginetica, stackingmore than two layers will worsen the performance sincethe sessions in this dataset are short in general. Usingtoo many layers will bring in noise for the representationlearning process. Meanwhile, the optimized number oflayers for YooChoose is larger than that for Diginetica.Thus, we conclude that SHARE can capture both thedirect and high-order connections among items in thehypergraphs and lead to accurate recommendation.

Short vs. Long Sessions. To fully understand howSHARE performs in modeling sessions under differentcircumstances, we group the sessions based on theirlength and test how SHARE and SR-GNN perform foreach group of sessions. As in Figure 5, SHARE pro-vides better recommendations for all session lengths incomparison with SR-GNN. Both models can make moreaccurate predictions for short action sequences than thelonger ones, since they omit the order information whichmay be important for long sessions with adequate userbehaviors. However, since the percentage of long ses-sions is low in real-world datasets for session-based rec-ommendation, the main purpose is to boost the recom-mendation for sessions with fewer items. Indeed, theproposed framework is suitable for such practical sce-narios (i.e., short-term sessions) characterized by only alimited number of sequential actions.

5 Conclusion and Future Work

To recommend the next interesting items in short-termsessions, we are motivated to exploit the correlationsamong items within various contextual windows in eachsession to better model their dynamic meanings acrosssessions. In this work, we propose a novel session-basedrecommendation system – SHARE – which is empow-ered by the hypergraph structure and hypergraph at-tention networks. With experiments on three real-worldbenchmark datasets, we find that the proposed SHAREis able to generate effective session-wise item embed-


dings and thus provide more accurate recommendationcompared with baseline models. While in this work wemainly studying how to generate session-wise item rep-resentations, in the future we want to explore how tocapture the item dynamics along time and over differentdomains in session-based recommendation. Meanwhile,we are also interested in extending the proposed hyper-graph attention networks for representation learning inother research areas.

References

[1] S. Bai, F. Zhang, and P. H. Torr, Hypergraphconvolution and hypergraph attention, arXiv preprintarXiv:1901.08150, (2019).

[2] R. v. d. Berg, T. N. Kipf, and M. Welling,Graph convolutional matrix completion, arXiv preprintarXiv:1706.02263, (2017).

[3] K. Ding, J. Li, R. Bhanushali, and H. Liu, Deepanomaly detection on attributed networks, in SDM,2019.

[4] K. Ding, J. Wang, J. Li, D. Li, and H. Liu, Be morewith less: Hypergraph attention networks for inductivetext classification, in EMNLP, 2020.

[5] W. Fan, Y. Ma, Q. Li, Y. He, E. Zhao, J. Tang,and D. Yin, Graph neural networks for social recom-mendation, in WWW, 2019.

[6] Y. Feng, H. You, Z. Zhang, R. Ji, and Y. Gao,Hypergraph neural networks, in AAAI, 2019.

[7] J. Gilmer, S. S. Schoenholz, P. F. Riley,O. Vinyals, and G. E. Dahl, Neural message passingfor quantum chemistry, in ICML, 2017.

[8] W. Hamilton, Z. Ying, and J. Leskovec, Inductiverepresentation learning on large graphs, in NeurIPS,2017.

[9] R. He, W.-C. Kang, and J. McAuley, Translation-based recommendation, in RecSys, 2017.

[10] X. He, Z. He, X. Du, and T.-S. Chua, Adversarialpersonalized ranking for recommendation, in SIGIR,2018.

[11] X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T.-S.Chua, Neural collaborative filtering, in WWW, 2017.

[12] B. Hidasi and A. Karatzoglou, Recurrent neuralnetworks with top-k gains for session-based recommen-dations, in CIKM, 2018.

[13] B. Hidasi, A. Karatzoglou, L. Baltrunas, andD. Tikk, Session-based recommendations with recur-rent neural networks, in ICLR, 2016.

[14] W.-C. Kang and J. McAuley, Self-attentive sequen-tial recommendation, in ICDM, 2018.

[15] T. N. Kipf and M. Welling, Semi-supervised clas-sification with graph convolutional networks, arXivpreprint arXiv:1609.02907, (2016).

[16] J. Li, P. Ren, Z. Chen, Z. Ren, T. Lian, andJ. Ma, Neural attentive session-based recommendation,in CIKM, 2017.

[17] J. Li, Y. Wang, and J. McAuley, Time intervalaware self-attention for sequential recommendation, inWSDM, 2020.

[18] G. Linden, B. Smith, and J. York, Amazon. comrecommendations: Item-to-item collaborative filtering,IEEE Internet computing, 7 (2003), pp. 76–80.

[19] Q. Liu, Y. Zeng, R. Mokhosi, and H. Zhang,Stamp: short-term attention/memory priority modelfor session-based recommendation, in KDD, 2018.

[20] R. Pasricha and J. McAuley, Translation-basedfactorization machines for sequential recommendation,in RecSys, 2018.

[21] P. Ren, Z. Chen, J. Li, Z. Ren, J. Ma, andM. de Rijke, Repeatnet: A repeat aware neural recom-mendation machine for session-based recommendation,in AAAI, 2019.

[22] S. Rendle, C. Freudenthaler, Z. Gantner, andL. Schmidt-Thieme, Bpr: Bayesian personalizedranking from implicit feedback, in UAI, 2009.

[23] S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme, Factorizing personalized markov chains fornext-basket recommendation, in WWW, 2010.

[24] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl,Item-based collaborative filtering recommendation algo-rithms, in WWW, 2001.

[25] Y. K. Tan, X. Xu, and Y. Liu, Improved recurrentneural networks for session-based recommendations, inProceedings of the 1st Workshop on Deep Learning forRecommender Systems, 2016.

[26] A. Vaswani, N. Shazeer, N. Parmar, J. Uszko-reit, L. Jones, A. N. Gomez, L. Kaiser, andI. Polosukhin, Attention is all you need, in NeurIPS,2017.

[27] P. Velickovic, G. Cucurull, A. Casanova,A. Romero, P. Lio, and Y. Bengio, Graph atten-tion networks, 2018.

[28] J. Wang and J. Caverlee, Recurrent recommenda-tion with local coherence, in WSDM, 2019.

[29] J. Wang, K. Ding, L. Hong, H. Liu, and J. Caver-lee, Next-item recommendation with sequential hyper-graphs, in SIGIR, 2020.

[30] J. Wang, R. Louca, D. Hu, C. Cellier, J. Caver-lee, and L. Hong, Time to shop for valentine’s day:Shopping occasions and sequential recommendation ine-commerce, in WSDM, 2020.

[31] X. Wang, X. He, M. Wang, F. Feng, and T.-S.Chua, Neural graph collaborative filtering, in SIGIR,2019.

[32] S. Wu, Y. Tang, Y. Zhu, L. Wang, X. Xie,and T. Tan, Session-based recommendation with graphneural networks, in AAAI, 2019.

[33] L. Zheng, C.-T. Lu, F. Jiang, J. Zhang, and P. S.Yu, Spectral collaborative filtering, in RecSys, 2018.

[34] Z. Zhu, J. Wang, and J. Caverlee, Improving top-krecommendation via jointcollaborative autoencoders, inWWW, 2019.


session-based recommendation with hypergraph attention

Documents