Download - Graph Learning in Financial Networks
![Page 1: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/1.jpg)
Graph Learning in Financial Networks
Jiaxuan YouStanford University
Collaboration with Tianyu Du, Fan-yun Sun, Jure Leskovec
![Page 2: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/2.jpg)
Financial Networks
§ Financial Networks: Describe financial entities and their connections
Jiaxuan You, Stanford University 2Image credit: The Political Economy of Global Finance: A Network Model Image credit: https://dailyblockchain.github.io/
International banking• Nodes: Countries• Edges: Capital flows
Bitcoin transactions• Nodes: BTC wallets• Edges: Transactions
![Page 3: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/3.jpg)
Graph Learning in Financial Networks
§ Goal: A graph learning framework for financial networks§ Applications: Fraud detection, Anti-money laundering, Anomaly detection§ Solution: Graph representation learning!
Jiaxuan You, Stanford University 3
Input: Financial networks
Node-level: Fraudsters, …
Edge-level: fraudulent/anomalous transactions, …
Output: Predictions
Subgraph-level: Money laundering subnetworks, …
Graph Neural Network
![Page 4: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/4.jpg)
Why Graph Representation?
§ Transaction-based approach§ “On 01/03, Client 𝐴 sends Company 𝐵 $500”§ Build models based on transaction attributes§ Issues: ignore the context of a transaction
§ Graph-based approach§ Represent transactions as a dynamic graph§ Predictions are made based on the entire graph§ Benefits:
§ Represents a transaction with a broader context§ Requires fewer feature-engineering
Jiaxuan You, Stanford University 4
$400, 01/05
bankclient company
$100, 01/01
$500, 01/03
$200, 01/02
$200, 01/02
$100, 01/06
![Page 5: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/5.jpg)
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
5Jiaxuan You, Stanford UniversityGraph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
![Page 6: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/6.jpg)
$400, 01/05
ROLAND: GNN for Financial Networks
6
bankclient company
$100, 01/01
$500, 01/03
$200, 01/02
$200, 01/02
Will there be a transaction? YesWhat is the amount? $500When will it happen? 01/03…Is it a fraud? NoIs it money laundering? Yes…
Graph Neural NetworksFinancial networks Learning Signals & Application
$100, 01/06
bankclient company
Self-supervised(from raw data)
Supervised(from external sources)
bankclient company
NN
NN
NN
NN
NN
NN
?
§ ROLAND framework:§ Transform financial networks as GNN
computational graphs§ Learning from diverse signals
Jiaxuan You, Stanford University
J. You, T. Du, J. Leskovec. ROLAND: Graph Neural Networks for Dynamic Graphs, in submission
node embeddings
![Page 7: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/7.jpg)
ROLAND Model: From Static to Dynamic GNN
Jiaxuan You, Stanford University 7
GNN Layer 1
Graph !
Pred "
GNN Layer 2
#(")
#($)
a) A static GNN with modern architectural design options
Skip connections
LayerconnectivityGNN Layer
GNN Layer
GNN Layer
Intra-layer Design: 4 dims
MLP Layer
MLP Layer
Messagepassinglayers
Post-processlayers
MLP Layer
MLP Layer
Pre-processlayers
Linear
BatchNorm
Dropout
Activation
Attention
Aggregation
Transformation
GNN Layer
GNN Layer
GNN Layer
MLP Layer
MLP Layer
Post-processlayers
MLP Layer
MLP Layer
Pre-processlayers
GNN Layer
GNN Layer
GNN Layer
Linear
BatchNorm
Dropout
Activation
Attention
Aggregation
Aggregation
Attention
Activation
Dropout
BatchNorm
LinearGNN Layer 1
Snapshot !%&"
Pred "%
GNN Layer 2
Embedding update
Embedding update
#$%&"(")
#%&"(")
#$%&"($)
#%&"($)
GNN Layer 1
Snapshot !%
Pred "%'"
GNN Layer 2
Embedding update
Embedding update
#$%(")
#%(")
#$%($)
#%($)
#%&"(")
#%&"($)
b) Extend static GNNs to dynamic GNNs
……
Hierarchical Node state #% = {#%" , … , #%(()}
Question: Can we adapt a SOTA static GNN to dynamic prediction tasks?
t+1t+2 t+3
This transaction is fraudulent
SOTAStatic GNN
![Page 8: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/8.jpg)
ROLAND Model: From Static to Dynamic GNN
Jiaxuan You, Stanford University 8
§ Idea: Recurrently update node embeddings at each layer
§ Introduce a new module to a static GNN:
§ Benefits:§ Simple and effective§ Benefit from the SOTA designs of a static GNN
Static GNN Dynamic GNN
Embedding update
Input:§ Previous embeddings from the
the same layer§ Current embeddings from the
previous layerOutput: Updated embeddings
GNN Layer 1
Graph !
Pred "
GNN Layer 2
GNN Layer 1
Snapshot !!"#
Pred "!
GNN Layer 2
Embedding update
Embedding update
GNN Layer 1
Snapshot !!
Pred "!$#
GNN Layer 2
Embedding update
Embedding update
……
![Page 9: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/9.jpg)
ROLAND Training: Efficient Training
§ Incremental training: § Only keep these in GPU
§ GNN Model 𝐺𝑁𝑁!§ Historical node states 𝐻!"#§ Incoming new graph snapshot 𝐺!§ Efficient and work well in practice
§ Meta training:§ Train a meta-GNN that can quickly adapt
to new data§ Benefits: ROLAND does not need to be
frequently retrained9Jiaxuan You, Stanford University
Snapshot!!"#
"!
!##!"#
Evaluate
!##!
Train
Time !
"$!
State%!
Snapshot!!
"!$#
!##!
Evaluate
!##!$#
Train
"$!$#
…
Time ! + #
…
Live-update evaluation pipeline
Only this part in GPU
![Page 10: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/10.jpg)
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
10Jiaxuan You, Stanford UniversityGraph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
![Page 11: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/11.jpg)
ROLAND Implementation: GraphGym
§ GraphGym offers a standardized pipeline for graph learning§ Design Space of Graph Neural Networks (NeurIPS 2020 spotlight)§ Now officially part of PyG 2.0! (Stay tuned for my next talk :)
11
General GNN design space
Clean interface for comparing designs
Jiaxuan You, Stanford University
J. You, R. Ying, J. Leskovec. Design Space of Graph Neural Networks, NeurIPS 2020
§ We implement the financial prediction pipeline with GraphGym§ We adopt many optimal design choices from GraphGym
![Page 12: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/12.jpg)
An Example Financial Dataset
Jiaxuan You, Stanford University 12
§ Transactions between companies captured by a bank clearing system§ 336 million transactions, 6 million accounts, 6 years of daily data
![Page 13: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/13.jpg)
§ Task setting: Supervised learning§ Unbalanced labels: 2% of all the transactions are fraudulent§ Random dataset split: 80% training, 10% validation, 10% testing§ Metric: AUC on test set (random guess has AUC 0.5, higher is better)
§ Best non-deep learning baseline§ AUC: ~0.8
§ Simple GNN implementation using out-of-the-box PyG§ AUC: 0.81
§ ROLAND framework works much better!§ AUC: 0.94
Task 1: Classify Fraud Transactions
13Jiaxuan You, Stanford University
![Page 14: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/14.jpg)
Task 2: Forecast Future Transactions
§ Task setting: Self-supervised/Unsupervised learning§ Rolling prediction: On each day, use all the historical information to predict the
transactions on the next day§ Metric: Mean reciprocal ranking (MMR) of ground-truth future transactions
Jiaxuan You, Stanford University 14
ROLAND significantly outperforms SOTA baselines
![Page 15: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/15.jpg)
Analysis: ROLAND’s Performance over Time
Jiaxuan You, Stanford University 15
ROLAND maintains a robust performance over transaction pattern changes
# of retraining epochs
Almost no retraining needed!
Test MRRRecall@1
(week)day of the year
Transaction amount
![Page 16: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/16.jpg)
Analysis: Interpreting ROLAND’s Predictions
§ Question: What kind of transactions the model paid attention to?
§ Method: Inspect edges' attention scores of the last GNN layer.§ Select edges with low/high attention scores.§ Get the feature value distribution of these
edges
Jiaxuan You, Stanford University 16
Domestic -> Abroad
Non-financial àHousehold
Higher attention to large amount transactions
![Page 17: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/17.jpg)
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
17Jiaxuan You, Stanford UniversityGraph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
![Page 18: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/18.jpg)
§ Goal: Detect anomalous transactions without external supervision§ Combine Graph Representation Learning with Outlier Detection
Application: Anomaly Detection
18
Input: Financial networks
node embeddings
edge embeddings
Jiaxuan You, Stanford University
![Page 19: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/19.jpg)
Self-supervised objectives
19
Real world Indicators (from baselgovernance.org)
Objectives for GNN
investment funds sent to countries of concern Predict country
Large cash deposits into (company) accounts Predict amount
unusual subject of transaction Predict subject of transaction(link prediction)
high volume of transactions within a short period predict acculumated amount
multiple individuals sending funds to the one beneficiary predict # transactions
Jiaxuan You, Stanford University
“Indicators and patterns of money laundering or terrorist financing”
![Page 20: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/20.jpg)
Objectives for Anomaly Detection
20
t+1t+2 t+3
Self-supervised Objectives● Objective 1: predict amount● Objective 2: predict recipient● Objective 3: predict country● …
dim
|N|
Outlier Detection Objectives
Jiaxuan You, Stanford University
§ Solution: ROLAND framework + diverse objectives
![Page 21: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/21.jpg)
Quantitative Evaluation for Anomaly Detection
● Task: Detect anomalous transactions without supervision● Goal: Simulate real-world anomaly detection use cases○ Corrupt the dataset to see if ROLAND can extract them○ Report precision/recall of corrupted edges detected by ROLAND
○ Ways of corrupting the dataset:■ Amount corruption (~1% transactions)■ Country corruption (~1% transactions)■ Recipient corruption (~1% transactions) → Rewire !
21
A B
C D
A B
C D
Jiaxuan You, Stanford University
![Page 22: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/22.jpg)
Quantitative Evaluation for Anomaly Detection+14.7% Precision +14.2% Recall
Jiaxuan You, Stanford University
Deep learning baseline:
Rule-based baseline: +172% Precision +171% Recall
![Page 23: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/23.jpg)
Jiaxuan You, Stanford University 23
Normal accountsPotential anomalous accounts
![Page 24: Graph Learning in Financial Networks](https://reader031.vdocuments.site/reader031/viewer/2022012508/618448f0fbcb2538ae0bc469/html5/thumbnails/24.jpg)
Overview of This Talk
ROLAND: GNN for Financial
Networks
Results & Insights from
ROLAND
Application: Anomaly Detection
24Jiaxuan You, Stanford University
Graph Neural Networks
bankclient company
NN
NN
NN
NN
NN
NN
Resources: ROLAND (in submission)Design Space for Graph Neural NetworksLearn more about GNN: PyG.org