An Inter-Frame Prediction Technique Combining Template Matching Prediction and Block Motion Compensation for High Efficiency Video Coding

Wen-Hsiao PengChun-Chi Chen

Circuits and Systems for Video Technology, 2013 IEEE Transactions on

Outline

• Introduction• Background• Bi-prediction Combining TMP and BMC• Analysis LS and LMS• Experiment Results• Conclusion

Introduction

• Inter prediction combines MVs from– TMP– BMC

for Overlapped Block Motion Compensation.

• Prediction performance of OBMC close to that of bi-prediction.– without having to signal the template MV

Introduction

• TMP generally outperforms SKIP prediction.

• TMP is inferior to block-based motion compensation.

• Another MV is required to best complement the template MV.

Introduction

• A key issue in video coders with motion-compensated prediction is how to trade off effectively between– accuracy of the motion field representation– required overhead

• Based on HEVC version 6.0

• Achieve the bitrate reduction.

Outline

• Introduction• Background

– Template Matching Prediction– Block Motion Compensation– SKIP and Merge-SKIP– Signal Model– Prediction Error Surface

• Bi-prediction Combining TMP and BMC• Analysis LS and LMS• Experiment Results• Conclusion

Template Matching Prediction

• Obtains the MV at a current pixel by finding, in the reference frames, the best match for a template region composed of its surrounding reconstructed pixels.

Block Motion Compensation

• The frames are partitioned in blocks of pixels and each block is predicted from a block of equal size in the reference frame.

Comparsion

True motion BMC TMP

SKIP and Merge-SKIP

• SKIP– H.264/AVC

• Merge-SKIP– Weighted sum

Signal Model

• Tao et al [19]– .

• Zheng et al [24]– .

[19] B. Tao and M. T. Orchard, “A parametric solution for optimal overlapped block motion compensation,” IEEE Trans. on Image Processing, vol. 10, no. 3, pp. 341–350, Mar. 2001.[24] W. Zheng, Y. Shishikui, M. Naemura, Y. Kanatsugu, and S. Itoh,“Analysis of space-dependent characteristics of motion- compensated frame differences based on a statistical motion distribution model,” IEEE Trans. on Image Processing, vol. 11, no. 4, pp. 377–386, Apr. 2002.

Signal Model

• Mean-sqaured prediction error– .

• Tao et al [19]– .

• Zheng et al [24]– .

[19] B. Tao and M. T. Orchard, “A parametric solution for optimal overlapped block motion compensation,” IEEE Trans. on Image Processing, vol. 10, no. 3, pp. 341–350, Mar. 2001.[24] W. Zheng, Y. Shishikui, M. Naemura, Y. Kanatsugu, and S. Itoh,“Analysis of space-dependent characteristics of motion- compensated frame differences based on a statistical motion distribution model,” IEEE Trans. on Image Processing, vol. 11, no. 4, pp. 377–386, Apr. 2002.

Signal Model

• Block MV, vb , and block center, sc– vb = v(sc)

– .

• Template MV, vt , and template center, st– vt = v(st)– .

Signal Model

Tao’s model Zheng’s model

Prediction Error Surface

Prediction Performance Comparsion

• Encoding 50 frames

Outline

• Introduction• Background• Bi-prediction Combining TMP and BMC

– Overlapped Block Motion Compensation– Least Square Solution– Least Mean-Square Solution

• Analysis LS and LMS• Experiment Results• Conclusion

Bi-prediction Combining TMP and BMC

• Predictor is computed as a weighted average of two reference blocks.– Template MV, vt– Block MV, vb

• TMP can better compensate for the movement of the top-left area of a prediction block.

• BMC is thus aimed at reducing further the prediction residual in the remaining area.

Overlapped Block Motion Compensation

• The weighting can be pixel adaptive.– .

– ω is indicating their likelihood

• The problem is to determine the OBMC weights so that the resulting predictor would produce a minimal residual.– .

Overlapped Block Motion Compensation

• How to minimize the prediction residual by a suitable choice of the block MV and OBMC weights.– .

• The approaches to solve the problem– Least Squares Approach– Least Mean-Square Approach

Least Square Solution

• Rely on an iterative algorithm to solve for the optimal weights.

1. Estimating Block MVs :

• .

2. Adapting OBMC Weights :

• .

• It’s convergence to a possibly local minimum is usually between 5 to 10 iterations.

Least Mean-Square Solution

• Introduce statistical signal models.

• Given that every block is to be predicted using OBMC based on two MVs– defaulting to the true MV – MV sampling the motion field at some point sb– determine a set of OBMC weights

Least Mean-Square Solution

• Transform the problem of minimizing ξ into that of minimizing its expected value E[ξ].– .

1. Fixing sb determine the :

• .

2. Find the optimal sb that yields the global minimum :

• .

Outline

• Introduction• Background• Bi-prediction Combining TMP and BMC• Analysis LS and LMS• Experiment Results• Conclusion

Analysis LS and LMS

• . indicates the likelihood of vt being the true motion of a pixel at s relative to the other hypothesis vb.

• Template MV is not as reliable for compensating pixels in the upper-left area as predicted by the theoretical results.

Tao’s model Zheng’s model LS solution

Analysis LS and LMS

• So, we would expect to drop to zero (or, equivalently, to increase to unity)

without amendment with amendment Multiple reference frames

Results

• Reductions in mean-square error

Outline

• Introduction• Background• Bi-prediction Combining TMP and BMC• Analysis LS and LMS• Experiment Results• Conclusion

Experiment ResultsRandom Access High Efficiency

Random AccessMain

Low-Delay B High Efficiency

Low-Delay B Main

Experiment Results

Experiment Results

Experiment Results

Outline

• Introduction• Background• Bi-prediction Combining TMP and BMC• Analysis LS and LMS• Experiment Results• Conclusion

Conclusion

• We proposed a bi-prediction scheme that combines BMC and TMP predictors through OBMC.

• TMP is inferior to BMC, but is, in general, superior to SKIP prediction.

• The data dependency complicates the pipeline design and hinders parallel processing.

• The proposed method restricted the use of TB-mode to 2Nx2N PUs only.