only for maxus Алгоритмы для задачи матирования...m. sindeev, a....
TRANSCRIPT
![Page 1: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/1.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Алгоритмы для задачи
матирования
Юрий Гитман
Video Group CS MSU Graphics & Media Lab
![Page 2: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/2.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Table of content
Introduction
Guided Filter
PatchMatch
Closed-form Matting
Alpha Flow
Conclusion
2
![Page 3: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/3.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Matting problem Common statement
3
Фон
Карта Прозрачности
Объект
Грубая разметка (Trimap)
Исходное изображение
![Page 4: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/4.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Sum of Absolute Differences Overall rank
Average (small trimap)
Average (large trimap)
Average (user-defined trimap)
SVR Matting (Support Vector Regression)
5.2 6.3 4.8 4.5
Weighted Color and Texture Matting 5.5 4.5 6.5 5.4
Shared Matting 6.1 6.0 7.5 4.9
Global Sampling Matting 7.3 5.5 8.8 7.8
Segmentation-based Matting 7.7 8.0 7.3 7.9
Fast Automatic Matting 7.8 7.1 8.1 8.1
Improved Color Matting 8.2 7.9 7.8 9.0
LSR Matting (Local Spline Regression)
9.0 10.4 6.9 9.6
Global Sampling Matting (filter version)
9.1 8.4 9.8 9.3
KNN Matting (K-Nearest Neighbor) 9.7 11.1 10.5 7.4
Learning-based Matting 10.1 10.3 9.4 10.6
LMSPIR Matting 10.2 9.4 10.9 10.3
Shared Matting (real-time) 10.3 10.4 10.4 10.3
Closed-form Matting 10.5 10.1 9.1 12.4
State of the art Image Matting
4
![Page 5: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/5.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Problem of papers on Video matting
Да, статей по matting’у во времени не мало, но многие из них опираются на существование идеального оптического потока
Михаил Ерофеев
5
![Page 6: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/6.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
В предыдущих сериях
6 X. Bai, J. Wang, D. Simons, “Towards Temporally-coherent Video Matting,” in IEEE Mirage, 2011
![Page 7: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/7.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Содержание
7
Introduction
Guided Filter
PatchMatch
Closed-form Matting
Alpha Flow
Conclusion
![Page 8: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/8.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
New edge-preserving filter
K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
Guidance
8
![Page 9: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/9.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
New edge-preserving filter
9
Guided Filter
K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
![Page 10: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/10.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
New edge-preserving filter
10
Bilateral Filter
K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
![Page 11: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/11.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
New edge-preserving filter
11
Gaussian Guided Filter
K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
![Page 12: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/12.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Idea of Filtering
12 K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
Для каждого окна :
Коэффициенты и , определяются так, чтобы наилучшим образом соответствовать исходному изображению
![Page 13: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/13.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Steps of the Algorithm
13 K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
![Page 14: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/14.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Application to Matting
14
Binary Mask Source Image
K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
![Page 15: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/15.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Application to Matting
15 K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
Guided Filter Source Image
![Page 16: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/16.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Application to Matting
16 K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
Joint bilateral Filter Source Image
![Page 17: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/17.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Application to Matting
17 K. He, J. Sun, X. Tang, “Guided Image Filtering,” in ECCV, 2010
Domain Transform Source Image
![Page 18: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/18.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Содержание
18
Introduction
Guided Filter
PatchMatch
Closed-form Matting
Nonlocal Matting
Alpha Flow
Conclusion
![Page 19: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/19.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Fast k nearest
neighbors search (KNN)
19 C. Barnes, E. Shechtman, A. Finkestein, D. Goldman, “PatchMatch.
A Randomized Correspondence Algorithm for Structural Image Editing,” in ACM Transactions on Graphics (TOG), 2009
![Page 20: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/20.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Пример работы алгоритма
20
Поиск потока из нижнего изображения в верхнее. Hue компонента соответствует углу
Magnitude компонента длине вектора
C. Barnes, E. Shechtman, A. Finkestein, D. Goldman, “PatchMatch. A Randomized Correspondence Algorithm for Structural Image
Editing,” in ACM Transactions on Graphics (TOG), 2009
![Page 21: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/21.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Детали алгоритма (1)
21 C. Barnes, E. Shechtman, A. Finkestein, D. Goldman, “PatchMatch.
A Randomized Correspondence Algorithm for Structural Image Editing,” in ACM Transactions on Graphics (TOG), 2009
![Page 22: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/22.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Детали алгоритма (2)
22 C. Barnes, E. Shechtman, A. Finkestein, D. Goldman, “PatchMatch.
A Randomized Correspondence Algorithm for Structural Image Editing,” in ACM Transactions on Graphics (TOG), 2009
![Page 23: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/23.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Содержание
23
Introduction
Guided Filter
PatchMatch
Closed-form Matting
Nonlocal Matting
Alpha Flow
Conclusion
![Page 24: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/24.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Предположение о природе
изображений
Основное уравнение matting’а:
Предположим, что коэффициенты
локально постоянны (окрестности 3×3 в авторской реализации)
24 A. Levin, D. Lischinski, Y. Weiss, “A Closed-form Solution to Natural Image Matting,” in IEEE Pattern Analysis and Machine Intelligence
(PAMI), 2008
![Page 25: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/25.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Matting Laplacian (1)
Предположив локальную линейность , мы можем построить функционал, экстремум которого будет решением:
Тогда
25
A. Levin, D. Lischinski, Y. Weiss, “A Closed-form Solution to Natural Image Matting,” in IEEE Pattern Analysis and Machine Intelligence
(PAMI), 2008
![Page 26: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/26.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Matting Laplacian (2)
26
, где
A. Levin, D. Lischinski, Y. Weiss, “A Closed-form Solution to Natural Image Matting,” in (PAMI), 2008
![Page 27: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/27.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Results (1)
27 A. Levin, D. Lischinski, Y. Weiss, “A Closed-form Solution to Natural Image Matting,” in IEEE Pattern Analysis and Machine Intelligence
(PAMI), 2008
Input image Bayesian Matting Poisson matting Closed-form matting Scribbles
![Page 28: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/28.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Results (2)
28 A. Levin, D. Lischinski, Y. Weiss, “A Closed-form Solution to Natural Image Matting,” in IEEE Pattern Analysis and Machine Intelligence
(PAMI), 2008
Input image Trimap Bayesian matting Scribbles Closed-form matting
![Page 29: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/29.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Results (3)
29 A. Levin, D. Lischinski, Y. Weiss, “A Closed-form Solution to Natural Image Matting,” in IEEE Pattern Analysis and Machine Intelligence
(PAMI), 2008
Input image Matting result Foreground reconstruction
Background reconstruction
New background
![Page 30: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/30.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Содержание
30
Introduction
Guided Filter
PatchMatch
Closed-form Matting
Nonlocal Matting
Alpha Flow
Conclusion
![Page 31: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/31.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Предположение о природе
движения в видео
31 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 32: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/32.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Шаги Алгоритма (1)
1. Инициализация альфа потока значениями оптического потока для RGB
2. Обработка occlusions: построение цепочек (суперпикселей во времени)
3. Перерасчет значений прозрачности
32 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 33: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/33.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Шаги Алгоритма (2)
4. Фильтрация (Guided Filter), чтобы подавить артефакты в виде больших полупрозрачных областей (в конце этот шаг может быть пропущен)
5. Вычисление альфа потока
6. Переход на шаг 2
33 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 34: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/34.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Вычисление оптического
потока (1)
Первое и третье слагаемое оптимизируются
34
попеременно
M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 35: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/35.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Вычисление оптического
потока (2)
Первое слагаемое:
PatchMatch-based Motion Estimation
Второе слагаемое:
Решение линейной системы уравнений (если я не ошибаюсь)
Вообще, говоря условие гладкости может быть включено и в PatchMatch [Besse 2012]
35 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 36: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/36.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Обработка occlusions
Оптический поток вычисляется в обе стороны
Если вектора в обоих направлениях приблизительно совпадают (если я не ошибаюсь), то они образуют ненаправленное ребро в нашем графе
Попробуем соединять последовательные вектора в цепочки
36 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 37: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/37.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Алгоритм построения
цепочек
Цепочки должны начинаться и заканчиваться в occlusion’ах
На каждом шаге будем «жадно» выбирать две цепочки, которые соединяем, пока цена (приращение оптимизируемой функции) не станет отрицательна
Каждая цепочка дает вклад в зависимости от дисперсии цветов пикселей вдоль нее
37 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 38: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/38.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Распределение длин
цепочек (Avg = 2–4)
38 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 39: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/39.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Иллюстрация объединения
цепочек
39 M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013
![Page 40: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/40.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Перерасчет значений
прозрачности
40
Энергия, которую мы оптимизируем на этом шаге:
, где Lt matting Laplacian
для полученных суперпикселей
![Page 41: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/41.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Содержание
41
Introduction
Guided Filter
PatchMatch
Closed-form Matting
Nonlocal Matting
Alpha Flow
Conclusion
![Page 42: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/42.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Matting algorithms
comparison by M. Erofeev (1)
42
![Page 43: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/43.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus Matting algorithms
comparison by M. Erofeev (1)
43
![Page 44: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/44.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Литература
1. M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video Matting,” Asian Conference on Computer Vision (ACCV), 2013.
2. K. He, J. Sun, X. Tang, “Guided Image Filtering,” in European Conference on Computer Vision (ECCV), 2010, pp. 1–14.
3. C. Barnes, E. Shechtman, A. Finkestein, D. Goldman, “PatchMatch. A Randomized Correspondence Algorithm for Structural Image Editing,” in ACM Transactions on Graphics (TOG), 2009, vol. 28, p. 24.
4. F. Besse, C. Rother, A. Fitzgibbon, J. Kautz, “PMBP: PatchMatch Belief Propagation for Correspondence Field Estimation,” in British Machine Vision Conference (BMVC), 2012.
5. A. Levin, D. Lischinski, Y. Weiss, “A Closed-form Solution to Natural Image Matting,” in IEEE Pattern Analysis and Machine Intelligence (PAMI), 2008, vol. 30, pp. 228–242.
44
![Page 45: Only for Maxus Алгоритмы для задачи матирования...M. Sindeev, A. Konushin, and C. Rother, “Alpha Flow for Video 37 Matting,” Asian Conference on Computer](https://reader034.vdocuments.site/reader034/viewer/2022042414/5f2f2c9dc72f5f0be40a3ef5/html5/thumbnails/45.jpg)
CS MSU Graphics & Media Lab (Video Group) www.compression.ru/video/
Only for Maxus
Литература
6. P. Lee, Y. Wu, “Nonlocal Matting,” in IEEE Computer Vision and Pattern Recogntion (CVPR), 2011, pp. 2193–2200.
7. X. Bai, J. Wang, D. Simons, “Towards Temporally-coherent Video Matting,” in IEEE Mirage, 2011, pp. 63–74.
8. I. Choi, M. Lee, Y.W. Tai, “Video Matting Using Multi-Frame Nonlocal Matting Laplacian,” in European Conference on Computer Vision (ECCV), 2012.
45