optimal transport in imaging sciences
DESCRIPTION
Slides of the presentation at the conference "Optimal Transport to Orsay", Orsay, France, June 18-22, 2012.TRANSCRIPT
![Page 1: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/1.jpg)
1
Optimal Transportin Imaging Sciences
Gabriel Peyré
www.numerical-tours.com
![Page 2: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/2.jpg)
Statistical Image Models
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Colors distribution: each pixel � point in R3
![Page 3: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/3.jpg)
Statistical Image Models
Input image Modified image
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Colors distribution: each pixel � point in R3
![Page 4: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/4.jpg)
Other applications: Texture synthesisTexture segmentation
Statistical Image Models
Input image Modified image
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Colors distribution: each pixel � point in R3
![Page 5: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/5.jpg)
Overview
• Wasserstein Distance
• Sliced Wasserstein Distance
• Color Transfer
• Wasserstein Barycenter
• Gaussian Texture Models
![Page 6: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/6.jpg)
Discrete measure:
Discrete Distributions
Xi � Rd�
i
pi = 1µ =N�1�
i=0
pi�Xi
![Page 7: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/7.jpg)
Xi
Quotient space:
Discrete measure:
Constant weights: pi = 1/N .Point cloud
RN�d/�N
Discrete Distributions
Xi � Rd�
i
pi = 1µ =N�1�
i=0
pi�Xi
![Page 8: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/8.jpg)
Xi
Quotient space: A�ne space:
Discrete measure:
Constant weights: pi = 1/N . Fixed positions Xi (e.g. grid)HistogramPoint cloud
{(pi)i \�
i pi = 1}RN�d/�N
Discrete Distributions
Xi � Rd�
i
pi = 1µ =N�1�
i=0
pi�Xi
![Page 9: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/9.jpg)
Discretized image f � RN�d
N = #pixels, d = #colors.fi � Rd = R3
From Images to Statistics
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 10: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/10.jpg)
Discretized image f � RN�d
N = #pixels, d = #colors.fi � Rd = R3
f µf� Needs an estimator:� Modify f by controlling µf .
From Images to Statistics
Disclamers: images are not distributions.
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 11: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/11.jpg)
Discretized image f � RN�d
N = #pixels, d = #colors.
Point cloud discretization:
fi � Rd = R3
f µf� Needs an estimator:
µf =�
i
�fi
� Modify f by controlling µf .
From Images to Statistics
Xi
Disclamers: images are not distributions.
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 12: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/12.jpg)
Discretized image f � RN�d
N = #pixels, d = #colors.
Point cloud discretization:
fi � Rd = R3
Histogram discretization:
f µf� Needs an estimator:
µf =�
i
�fi
� Modify f by controlling µf .
µf =�
i
pi�Xi
pi =1
Zf
�
j
�(xi � fj)Parzen windows:
From Images to Statistics
Xi
Disclamers: images are not distributions.
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 13: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/13.jpg)
Discretized image f � RN�d
N = #pixels, d = #colors.
Point cloud discretization:
fi � Rd = R3
Histogram discretization:
f µf� Needs an estimator:
µf =�
i
�fi
� Modify f by controlling µf .
µf =�
i
pi�Xi
pi =1
Zf
�
j
�(xi � fj)Parzen windows:
Today’s focus
From Images to Statistics
Xi
Disclamers: images are not distributions.
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 14: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/14.jpg)
µX =�
i
�Xi
Colors: 3-DVector X � RN�d
(image, coe�cients, . . . )
Optimal Transport Distances
R
GB
Grayscale: 1-D
![Page 15: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/15.jpg)
µX =�
i
�Xi
Colors: 3-D
�⇥ ⇥ argmin���N
�
i
||Xi � Y�(i)||p
Vector X � RN�d
(image, coe�cients, . . . )
Optimal Transport Distances
R
GB
Grayscale: 1-D
Optimal assignment:
![Page 16: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/16.jpg)
µX =�
i
�Xi
Colors: 3-D
�⇥ ⇥ argmin���N
�
i
||Xi � Y�(i)||p
Wasserstein distance: Wp(µX , µY )p =�
i
||Xi � Y��(i)||p
�� Metric on the space of distributions.
Vector X � RN�d
(image, coe�cients, . . . )
Optimal Transport Distances
R
GB
Grayscale: 1-D
Optimal assignment:
![Page 17: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/17.jpg)
µX =�
i
�Xi
Colors: 3-D
�⇥ ⇥ argmin���N
�
i
||Xi � Y�(i)||p
Wasserstein distance:
Projection on statistical constraints:ProjC(f) = Y � ��
Wp(µX , µY )p =�
i
||Xi � Y��(i)||p
�� Metric on the space of distributions.
C = {f \ µf = µY }
Vector X � RN�d
(image, coe�cients, . . . )
Optimal Transport Distances
R
GB
Grayscale: 1-D
Optimal assignment:
![Page 18: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/18.jpg)
Computing Transport Distances
Xi
Yi
Explicit solution for 1D distribution (e.g. grayscale images):
�� sorting the values, O(N log(N)) operations.
![Page 19: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/19.jpg)
Higher dimensions: combinatorial optimization methods
Hungarian algorithm, auctions algorithm, etc.
Computing Transport Distances
Xi
Yi
Explicit solution for 1D distribution (e.g. grayscale images):
�� sorting the values, O(N log(N)) operations.
�� O(N5/2 log(N)) operations.
�� intractable for imaging problems.
![Page 20: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/20.jpg)
Higher dimensions: combinatorial optimization methods
Hungarian algorithm, auctions algorithm, etc.
µ =�
i
pi�Xi ⇥ =�
i
qi�Yi
�� Wp(µ, �)p solution of a linear program.
Arbitrary distributions:
Computing Transport Distances
Xi
Yi
Explicit solution for 1D distribution (e.g. grayscale images):
�� sorting the values, O(N log(N)) operations.
�� O(N5/2 log(N)) operations.
�� intractable for imaging problems.
![Page 21: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/21.jpg)
Overview
• Wasserstein Distance
• Sliced Wasserstein Distance
• Color Transfer
• Wasserstein Barycenter
• Gaussian Texture Models
![Page 22: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/22.jpg)
Approximate Sliced Distance
Key idea: replace transport in Rd by series of 1D transport. Xi
�Xi, ��Projected point cloud: X� = {�Xi, �⇥}i.
[Rabin, Peyre, Delon & Bernot 2010]
![Page 23: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/23.jpg)
(p = 2)
SW (µX , µY )2 =�
||�||=1W (µX� , µY� )
2d�
Approximate Sliced Distance
Key idea: replace transport in Rd by series of 1D transport. Xi
�Xi, ��Sliced Wasserstein distance:
Projected point cloud: X� = {�Xi, �⇥}i.
[Rabin, Peyre, Delon & Bernot 2010]
![Page 24: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/24.jpg)
(p = 2)
Theorem: E(X) = SW (µX , µY )2 is of class C1 and
SW (µX , µY )2 =�
||�||=1W (µX� , µY� )
2d�
�E(X) =�
��Xi � Y⇥�(i), �� � d�.
Approximate Sliced Distance
Key idea: replace transport in Rd by series of 1D transport. Xi
�Xi, ��Sliced Wasserstein distance:
Projected point cloud: X� = {�Xi, �⇥}i.
[Rabin, Peyre, Delon & Bernot 2010]
where �� � �N are 1-D optimal assignents of X� and Y�.
![Page 25: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/25.jpg)
(p = 2)
Theorem: E(X) = SW (µX , µY )2 is of class C1 and
SW (µX , µY )2 =�
||�||=1W (µX� , µY� )
2d�
�E(X) =�
��Xi � Y⇥�(i), �� � d�.
Approximate Sliced Distance
Key idea: replace transport in Rd by series of 1D transport. Xi
�Xi, ��Sliced Wasserstein distance:
Projected point cloud: X� = {�Xi, �⇥}i.
[Rabin, Peyre, Delon & Bernot 2010]
where �� � �N are 1-D optimal assignents of X� and Y�.
�� Possible to use SW in variational imaging problems.�� Fast numerical scheme : use a few random �.
![Page 26: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/26.jpg)
Sliced Assignment
X is a local minima of E(X) = SW (µX , µY )2
�� �� � �N , X = Y � �
Theorem:
![Page 27: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/27.jpg)
Stochastic gradient descent of E(X):E�(X) =
�
���
W (X�, Y�)2Step 1: choose � at random.
Step 2: X(�+1) = X(�) � ��E�(X(�))
Sliced Assignment
X is a local minima of E(X) = SW (µX , µY )2
�� �� � �N , X = Y � �
Theorem:
![Page 28: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/28.jpg)
Stochastic gradient descent of E(X):E�(X) =
�
���
W (X�, Y�)2Step 1: choose � at random.
Step 2:
X(�) converges to C = {X \ µX = µY }.
X(�+1) = X(�) � ��E�(X(�))
Sliced Assignment
X is a local minima of E(X) = SW (µX , µY )2
�� �� � �N , X = Y � �
Theorem:
![Page 29: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/29.jpg)
Stochastic gradient descent of E(X):E�(X) =
�
���
W (X�, Y�)2Step 1: choose � at random.
Step 2:
X(�) converges to C = {X \ µX = µY }.
X(�+1) = X(�) � ��E�(X(�))
Sliced Assignment
X is a local minima of E(X) = SW (µX , µY )2
�� �� � �N , X = Y � �
Theorem:
X(�) � ProjC(X(0))Numerical observation:
Final assignment
![Page 30: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/30.jpg)
Overview
• Wasserstein Distance
• Sliced Wasserstein Distance
• Color Transfer
• Wasserstein Barycenter
• Gaussian Texture Models
![Page 31: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/31.jpg)
Input color images: fi � RN�3.
Color Histogram Equalization
⇥i =1N
�
x
�fi(x)
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
f0
µ1
f1
µ0
f0
![Page 32: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/32.jpg)
Optimal assignement:
Input color images: fi � RN�3.
min���N
||f0 � f1 ⇥ �||
Color Histogram Equalization
⇥i =1N
�
x
�fi(x)
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
f0
µ1
f1
µ0
f0
![Page 33: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/33.jpg)
Transport:
Optimal assignement:
Input color images: fi � RN�3.
min���N
||f0 � f1 ⇥ �||
T : f0(x) � R3 �� f1(�(i)) � R3
Color Histogram Equalization
⇥i =1N
�
x
�fi(x)
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
f0
µ1
f1
µ0
f0
T
![Page 34: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/34.jpg)
Transport:
Equalization:
Optimal assignement:
Input color images: fi � RN�3.
min���N
||f0 � f1 ⇥ �||
T : f0(x) � R3 �� f1(�(i)) � R3
f0 = T (f0) �� f0 = f1 � �
Color Histogram Equalization
⇥i =1N
�
x
�fi(x)
T
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
f0
T (f0)
µ1
f1
µ0 µ1
f0
T
![Page 35: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/35.jpg)
Solving is computationally untractable.
and is close to f0
f0 solves minf
E(f) = SW (µf , µf1)
Sliced Wasserstein Transfert
Approximate Wasserstein projection:
min���N
||f0 � f1 ⇥ �||
![Page 36: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/36.jpg)
Solving is computationally untractable.
and is close to f0
f (0) = f0
(Stochastic) gradient descent:
At convergence: µf0= µf1
f (�+1) = f (�) � ���E(f (�))
f0 solves minf
E(f) = SW (µf , µf1)
f (�) � f0
Sliced Wasserstein Transfert
Approximate Wasserstein projection:
min���N
||f0 � f1 ⇥ �||
![Page 37: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/37.jpg)
Input image f0
Target image f1
Transfered image f0
Color Transfert
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 38: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/38.jpg)
Transfered image f1
Color Exchange
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
X ⇥� Y
Y ⇥� X
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
X ⇥� Y
Y ⇥� X
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
X ⇥� Y
Y ⇥� X
J. Rabin Wasserstein Regularization
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
X ⇥� Y
Y ⇥� X
J. Rabin Wasserstein Regularization
Input image f0 Target image f1
Transfered image f0
![Page 39: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/39.jpg)
Overview
• Wasserstein Distance
• Sliced Wasserstein Distance
• Color Transfer
• Wasserstein Barycenter
• Gaussian Texture Models
![Page 40: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/40.jpg)
µ�
µ1
µ3
W2(µ1, µ� )
W2(µ
2, µ
� )
W2 (µ
3 , µ �)�
i
�i = 1Barycenter of {(µi, �i)}Li=1:
µ� � argminµ
L�
i=1
�iW2(µi, µ)2
Wasserstein Barycenterµ2
![Page 41: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/41.jpg)
µ�
µ1
µ3
W2(µ1, µ� )
W2(µ
2, µ
� )
W2 (µ
3 , µ �)
If µi = �Xi , then µ� = �X�
X� =�
i
�iXi
�� Generalizes Euclidean barycenter.
�
i
�i = 1Barycenter of {(µi, �i)}Li=1:
µ� � argminµ
L�
i=1
�iW2(µi, µ)2
Wasserstein Barycenterµ2
![Page 42: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/42.jpg)
µ� exists and is unique.
µ�
µ1
µ3
Theorem:if µ0 does not vanish on small sets,
W2(µ1, µ� )
W2(µ
2, µ
� )
W2 (µ
3 , µ �)
If µi = �Xi , then µ� = �X�
X� =�
i
�iXi
�� Generalizes Euclidean barycenter.
�
i
�i = 1Barycenter of {(µi, �i)}Li=1:
µ� � argminµ
L�
i=1
�iW2(µi, µ)2
Wasserstein Barycenter
[Agueh, Carlier, 2010]
µ2
![Page 43: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/43.jpg)
t �� µt is the geodesic path.
µt � argminµ
(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2
Special Case: 2 Distributions
Case L = 2:
µ1
µ2µ�
![Page 44: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/44.jpg)
µ =�N
k=1 �Xi(k),Discrete point clouds:
Assignment: �⇥ ⇥ argmin���N
N�
k=1
||X1(k)�X2(�(k))||2
µ� =�N
k=1 �X�(k),
t �� µt is the geodesic path.
X� = (1� t)X1(k) + tX2(��(k))
µt � argminµ
(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2
Special Case: 2 Distributions
Case L = 2:
µ1
µ2µ�
where
![Page 45: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/45.jpg)
� i = 1, . . . , L, µi =�
k �Xi(k)
Linear Programming Resolution
µ�
µ1
µ2
µ3
Discrete setting:
![Page 46: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/46.jpg)
� i = 1, . . . , L, µi =�
k �Xi(k)
C(k1, . . . , kL) =�
i,j �i�j ||Xi(ki)�Xj(kj)||2
Linear Programming Resolution
L-way interaction cost:
µ�
µ1
µ2
µ3
Discrete setting:
![Page 47: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/47.jpg)
� i = 1, . . . , L, µi =�
k �Xi(k)
L-way coupling matrices:
�= 1
�= 1
�= 1
C(k1, . . . , kL) =�
i,j �i�j ||Xi(ki)�Xj(kj)||2
P � PL
Linear Programming Resolution
L-way interaction cost:
µ�
µ1
µ2
µ3
Discrete setting:
![Page 48: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/48.jpg)
� i = 1, . . . , L, µi =�
k �Xi(k)
L-way coupling matrices:
�= 1
�= 1
�= 1
Linear program:
C(k1, . . . , kL) =�
i,j �i�j ||Xi(ki)�Xj(kj)||2
minP�PL
�
k1,...,kL
Pk1,...,kLC(k1, . . . , kL)
P � PL
Linear Programming Resolution
L-way interaction cost:
µ�
µ1
µ2
µ3
Discrete setting:
![Page 49: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/49.jpg)
� i = 1, . . . , L, µi =�
k �Xi(k)
L-way coupling matrices:
�= 1
�= 1
�= 1
Linear program:
Barycenter:
X�(k1, . . . , kL) =�L
i=1 �iXi(ki)
C(k1, . . . , kL) =�
i,j �i�j ||Xi(ki)�Xj(kj)||2
minP�PL
�
k1,...,kL
Pk1,...,kLC(k1, . . . , kL)
P � PL
µ� =�
k1,...,kL
Pk1,...,kL�X�(k1,...,kL)
Linear Programming Resolution
L-way interaction cost:
µ�
µ1
µ2
µ3
Discrete setting:
![Page 50: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/50.jpg)
µX that solves
minX
�
i
�i SW (µX , µXi)2
Sliced Wasserstein Barycenter
Sliced-barycenter:
![Page 51: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/51.jpg)
Gradient descent:
µX that solves
minX
�
i
�i SW (µX , µXi)2
Ei(X) = SW (µX , µXi)2
X(�+1) = X(�) � ⇥�
L�
i=1
�i�Ei(X(�))
Sliced Wasserstein Barycenter
Sliced-barycenter:
![Page 52: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/52.jpg)
Gradient descent:
Disadvantage:Non-convex problem
� local minima.
Advantages:
µX that solves
minX
�
i
�i SW (µX , µXi)2
Ei(X) = SW (µX , µXi)2
X(�+1) = X(�) � ⇥�
L�
i=1
�i�Ei(X(�))
Sliced Wasserstein Barycenter
µX is a sum of N Diracs.
Sliced-barycenter:
Smooth optimization problem.
![Page 53: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/53.jpg)
Gradient descent:
Disadvantage:Non-convex problem
� local minima.
Advantages:
µX that solves
µ1
minX
�
i
�i SW (µX , µXi)2
Ei(X) = SW (µX , µXi)2
X(�+1) = X(�) � ⇥�
L�
i=1
�i�Ei(X(�))
Sliced Wasserstein Barycenter
µX is a sum of N Diracs.
Sliced-barycenter:
Smooth optimization problem.
µ2µ3
![Page 54: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/54.jpg)
Raw imagesequence
Application: Color Harmonization
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
![Page 55: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/55.jpg)
Raw imagesequence
ComputeWassersteinbarycenter
Application: Color Harmonization
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
![Page 56: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/56.jpg)
Raw imagesequence
ComputeWassersteinbarycenter
Project onthe barycenter
Application: Color Harmonization
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Raw image sequence
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Sliced Wasserstein Projection on the barycenter
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Sliced Wasserstein Projection on the barycenter
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
Wasserstein Barycenter Sliced Wasserstein Barycenter Experimental results Applications Conclusion
Color transfer
Color harmonization of several images. Step 1: compute Sliced-Wasserstein Barycenter of color statistics;. Step 2: compute Sliced-Wasserstein projection of each image onto theBarycenter;
Sliced Wasserstein Projection on the barycenter
J. Rabin – GREYC, University of Caen Approximate Wasserstein Metric for Texture Synthesis and Mixing
![Page 57: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/57.jpg)
Overview
• Wasserstein Distance
• Sliced Wasserstein Distance
• Color Transfer
• Wasserstein Barycenter
• Gaussian Texture Models
![Page 58: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/58.jpg)
Exemplar f0
Texture Synthesis
Problem: given f0, generate f“random”
perceptually “similar”
![Page 59: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/59.jpg)
analysis Probabilitydistributionµ = µ(p)
Exemplar f0
Texture Synthesis
Problem: given f0, generate f“random”
perceptually “similar”
![Page 60: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/60.jpg)
analysis synthesisProbabilitydistributionµ = µ(p)
Exemplar f0
Outputs f � µ(p)
Texture Synthesis
Problem: given f0, generate f“random”
perceptually “similar”
![Page 61: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/61.jpg)
analysis synthesisProbabilitydistributionµ = µ(p)
Exemplar f0
Outputs f � µ(p)
Gaussian models: µ = N (m,�), parameters p = (m,�).
Texture Synthesis
Problem: given f0, generate f“random”
perceptually “similar”
![Page 62: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/62.jpg)
Input exemplar:d = 1 (grayscale), d = 3 (color)
N1
N2
N3
Images Videos
f0 � RN�d
Gaussian Texture Model
N1
N2
![Page 63: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/63.jpg)
Input exemplar:d = 1 (grayscale), d = 3 (color)
N1
N2
N3
Images Videos
Gaussian model:
m � RN�d,� � RNd�Nd
X � µ = N (m,�)
f0 � RN�d
Gaussian Texture Model
N1
N2
![Page 64: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/64.jpg)
Input exemplar:d = 1 (grayscale), d = 3 (color)
N1
N2
N3
Images Videos
Gaussian model:
m � RN�d,� � RNd�Nd
X � µ = N (m,�)
� highly under-determined problem.Texture analysis: from f0 � RN�d, learn (m,�).
f0 � RN�d
Gaussian Texture Model
N1
N2
![Page 65: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/65.jpg)
Input exemplar:d = 1 (grayscale), d = 3 (color)
N1
N2
N3
Images Videos
Gaussian model:
m � RN�d,� � RNd�Nd
X � µ = N (m,�)
Texture synthesis:given (m,�), draw a realization f = X(�).
� highly under-determined problem.
� Factorize � = AA� (e.g. Cholesky).� Compute f = m + Aw where w drawn from N (0, Id).
Texture analysis: from f0 � RN�d, learn (m,�).
f0 � RN�d
Gaussian Texture Model
N1
N2
![Page 66: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/66.jpg)
Stationarity hypothesis: X(· + �) � X(periodic BC)
Spot Noise Model [Galerne et al.]
![Page 67: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/67.jpg)
Stationarity hypothesis: X(· + �) � X(periodic BC)
Block-diagonal Fourier covariance:y(�) = �(�)f(�)y = �f computed as
where f(�) =�
x
f(x)e2ix1⇥1�
N1+
2ix2⇥2�N2
Spot Noise Model [Galerne et al.]
![Page 68: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/68.jpg)
Stationarity hypothesis: X(· + �) � X(periodic BC)
Block-diagonal Fourier covariance:y(�) = �(�)f(�)y = �f computed as
where f(�) =�
x
f(x)e2ix1⇥1�
N1+
2ix2⇥2�N2
Maximum likelihood estimate (MLE) of m from f0:
� i, mi =1N
�
x
f0(x) � Rd
Spot Noise Model [Galerne et al.]
![Page 69: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/69.jpg)
Stationarity hypothesis: X(· + �) � X(periodic BC)
Block-diagonal Fourier covariance:
�i,j =1N
�
x
f0(i + x)�f0(j + x) � Rd�d
y(�) = �(�)f(�)y = �f computed as
where f(�) =�
x
f(x)e2ix1⇥1�
N1+
2ix2⇥2�N2
MLE of �:
Maximum likelihood estimate (MLE) of m from f0:
� i, mi =1N
�
x
f0(x) � Rd
Spot Noise Model [Galerne et al.]
![Page 70: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/70.jpg)
Stationarity hypothesis: X(· + �) � X(periodic BC)
Block-diagonal Fourier covariance:
�i,j =1N
�
x
f0(i + x)�f0(j + x) � Rd�d
y(�) = �(�)f(�)y = �f computed as
where f(�) =�
x
f(x)e2ix1⇥1�
N1+
2ix2⇥2�N2
�� �� �= 0, �(�) = f0(�)f0(�)� � Cd�d
� is a spot noise �� �� �= 0, �(�) is rank-1.
MLE of �:
Maximum likelihood estimate (MLE) of m from f0:
� i, mi =1N
�
x
f0(x) � Rd
Spot Noise Model [Galerne et al.]
![Page 71: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/71.jpg)
� Cd � C
Input f0 � RN�3 Realizations f
Example of Synthesis
Synthesizing f = X(�), X � N (m,�):
�� �= 0, f(�) = f0(�)w(�)
� Convolve each channel with the same white noise.
w � N (N�1, N�1/2IdN )
![Page 72: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/72.jpg)
Input distributions (µ0, µ1) with µi = N (mi,�i).
E0 E1
Ellipses: Ei =�x � Rd \ (mi � x)���1
i (mi � x) � c�
Gaussian Optimal Transport
![Page 73: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/73.jpg)
Input distributions (µ0, µ1) with µi = N (mi,�i).
�0,1 = (�1/21 �0�
1/21 )1/2
E0 E1
W2(µ0, µ1)2 = tr (�0 + �1 � 2�0,1) + ||m0 �m1||2,
T
T (x) = Sx + m1 �m0
Theorem: If ker(�0) � Im(�1) = {0},
S = �1/21 �+
0,1�1/21
where
Ellipses: Ei =�x � Rd \ (mi � x)���1
i (mi � x) � c�
Gaussian Optimal Transport
![Page 74: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/74.jpg)
Geodesics between (µ0, µ1): t � [0, 1] �� µt
Wasserstein Geodesics
(W2 is a geodesic distance)µt = argmin
µ(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2
Variational caracterization:
![Page 75: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/75.jpg)
Geodesics between (µ0, µ1): t � [0, 1] �� µt
Optimal transport caracterization:
µ1µ0
µt = ((1� t)Id + tT )�µ0
Wasserstein Geodesics
(W2 is a geodesic distance)µt = argmin
µ(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2
Variational caracterization:
µ�
![Page 76: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/76.jpg)
Geodesics between (µ0, µ1): t � [0, 1] �� µt
Optimal transport caracterization:
µ1µ0
µt = ((1� t)Id + tT )�µ0
Gaussian case:µ0
µ1
Wasserstein Geodesics
(W2 is a geodesic distance)µt = argmin
µ(1� t)W2(µ0, µ)2 + tW2(µ1, µ)2
Variational caracterization:
µ�
� the set of Gaussians is geodesically convex.
µt = Tt�µ0 = N (mt,�t)
mt = (1� t)m0 + tm1
�t = [(1� t)Id + tT ]�0[(1� t)Id + tT ]
![Page 77: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/77.jpg)
Geodesic of Spot NoisesTheorem:i.e. �i(�) = f [i](�)f [i](�)�.
f [t] = (1� t)f [0] + tg[1]
g[1](�) = f [1](�)f [1](�)�f [0](�)|f [1](�)�f [0](�)|
Then � t � [0, 1], µt = µ(f [t])Let for i = 0, 1, µi = µ(f [i]) be spot noises,
![Page 78: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/78.jpg)
tf [0]f [1]
Geodesic of Spot Noises
0 1
Theorem:i.e. �i(�) = f [i](�)f [i](�)�.
f [t] = (1� t)f [0] + tg[1]
g[1](�) = f [1](�)f [1](�)�f [0](�)|f [1](�)�f [0](�)|
Then � t � [0, 1], µt = µ(f [t])Let for i = 0, 1, µi = µ(f [i]) be spot noises,
![Page 79: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/79.jpg)
OT Barycenters
µ1 µ3
µ2
![Page 80: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/80.jpg)
OT Barycenters
µ1 µ3
µ2
![Page 81: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/81.jpg)
OT Barycenters
µ1 µ3
µ2
![Page 82: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/82.jpg)
Optimal transportEuclidean Rao
2-D Gaussian Barycenters
![Page 83: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/83.jpg)
Inpu
t
Spot Noise Barycenters
![Page 84: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/84.jpg)
Inpu
t
Spot Noise Barycenters
![Page 85: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/85.jpg)
Exe
mpl
ars
Dynamic Textures Mixing
19/05/12 Static and dynamic texture mixing using optimal transport
1/1localhost/Users/gabrielpeyre/Documents/Dropbox/slides/2012-‐‑05-‐‑20-‐‑siims-‐‑textures/…/index.html
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start
images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
Abstract.- This paper tackles static and dynamic texture mixing by combining the statistical properties of an input set of images or videos. We focus on spot noisetextures that follow a stationary and Gaussian model which can be learned from the given exemplars. From here, we define, using optimal transport, the distancebetween texture models, derive the geodesic path, and define the barycenter between several texture models. These derivations are useful because they allow theuser to navigate inside the set of texture models, interpolating a new texture model at each element of the set. From these new interpolated models, new texturescan be synthesized of arbitrary size in space and time. Numerical results obtained from a library of exemplars show the ability of our method to generate newcomplex realistic static and dynamic textures.
Results following the geodesic path:
M A T L A B C O D E
1
24
5
6
0
3
7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
19/05/12 Static and dynamic texture mixing using optimal transport
1/1localhost/Users/gabrielpeyre/Documents/Dropbox/slides/2012-‐‑05-‐‑20-‐‑siims-‐‑textures/…/index.html
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start
images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
Abstract.- This paper tackles static and dynamic texture mixing by combining the statistical properties of an input set of images or videos. We focus on spot noisetextures that follow a stationary and Gaussian model which can be learned from the given exemplars. From here, we define, using optimal transport, the distancebetween texture models, derive the geodesic path, and define the barycenter between several texture models. These derivations are useful because they allow theuser to navigate inside the set of texture models, interpolating a new texture model at each element of the set. From these new interpolated models, new texturescan be synthesized of arbitrary size in space and time. Numerical results obtained from a library of exemplars show the ability of our method to generate newcomplex realistic static and dynamic textures.
Results following the geodesic path:
M A T L A B C O D E
1
24
5
6
0
3
7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
19/05/12 Static and dynamic texture mixing using optimal transport
1/1localhost/Users/gabrielpeyre/Documents/Dropbox/slides/2012-‐‑05-‐‑20-‐‑siims-‐‑textures/…/index.html
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start
images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
Abstract.- This paper tackles static and dynamic texture mixing by combining the statistical properties of an input set of images or videos. We focus on spot noisetextures that follow a stationary and Gaussian model which can be learned from the given exemplars. From here, we define, using optimal transport, the distancebetween texture models, derive the geodesic path, and define the barycenter between several texture models. These derivations are useful because they allow theuser to navigate inside the set of texture models, interpolating a new texture model at each element of the set. From these new interpolated models, new texturescan be synthesized of arbitrary size in space and time. Numerical results obtained from a library of exemplars show the ability of our method to generate newcomplex realistic static and dynamic textures.
Results following the geodesic path:
M A T L A B C O D E
1
24
5
6
0
3
7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
19/05/12 Static and dynamic texture mixing using optimal transport
1/1localhost/Users/gabrielpeyre/Documents/Dropbox/slides/2012-‐‑05-‐‑20-‐‑siims-‐‑textures/…/index.html
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%Start
images%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
<%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%>
Abstract.- This paper tackles static and dynamic texture mixing by combining the statistical properties of an input set of images or videos. We focus on spot noisetextures that follow a stationary and Gaussian model which can be learned from the given exemplars. From here, we define, using optimal transport, the distancebetween texture models, derive the geodesic path, and define the barycenter between several texture models. These derivations are useful because they allow theuser to navigate inside the set of texture models, interpolating a new texture model at each element of the set. From these new interpolated models, new texturescan be synthesized of arbitrary size in space and time. Numerical results obtained from a library of exemplars show the ability of our method to generate newcomplex realistic static and dynamic textures.
Results following the geodesic path:
M A T L A B C O D E
1
24
5
6
0
3
7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
[0] [1] [2]f f f
0 1 2 3 4 5 6 7
[See web page]
![Page 86: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/86.jpg)
Conclusion
Image modeling with statistical constraints�� Colorization, synthesis, mixing, . . .
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 87: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/87.jpg)
�� Fast sliced approximation.Wasserstein distance approach
Conclusion
Image modeling with statistical constraints�� Colorization, synthesis, mixing, . . .
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization
![Page 88: Optimal Transport in Imaging Sciences](https://reader037.vdocuments.site/reader037/viewer/2022103001/55866a12d8b42a45488b469c/html5/thumbnails/88.jpg)
�� Fast sliced approximation.Wasserstein distance approach
Extension to a wide rangeof imaging problems.
in out�� Color transfert, segmentation, . . .
Conclusion
14 Anonymous
P (⇥⇥) P (⇥� ) P (⇥� c) P (⇥⇥) P (⇥� ) P (⇥� c)
P (⇥⇥) P (⇥� ) P (⇥� c) P (⇥⇥) P (⇥� ) P (⇥� c)
Fig. 4. Left: example of natural image segmentation. Below each image is displayed the 2-D histogram of the image (in the space of the two dominant color eigenvectors as providedby a PCA) for the whole domain � (which is P (⇥⇥)) and over the inside and outside thesegmented region.
Image modeling with statistical constraints�� Colorization, synthesis, mixing, . . .
Optimal transport framework Sliced Wasserstein projection Applications
Application to Color Transfer
Source image (X )
Style image (Y )
Sliced Wasserstein projection of X to styleimage color statistics Y
Source image after color transfer
J. Rabin Wasserstein Regularization