with variables and , where denotes the nuclear norm (sum of singular values), denotes the sum of absolute value of the entries (not the -norm in the matrix sense), and is a regularization parameter. This formulation is also known as principal component pursuit (PCP) method in the literature and can be solved by utilizing the augmented Lagrange multiplier (ALM) method.
Additive perturbation is not a realistic assumption in video foreground-background decomposition. One immediate consequence of the additive model is that the recovered foreground object would not have the correct color. We forgo the additive assumption and instead propose a formulation of an overlaying model, which acknowledges that the foreground object is overlaid on top of the background and is occluding it (rather than simply being added).
The goal is to find matrices (with low-rank structure) and (with sparse structure) such that the overlaying model is satisfied. A plausible formulation of such problem can be written as
where is a regularization parameter and is a penalty parameter. The indicator function ensures . The algorithm to solve this problem is not immediately obvious. The details of how this probelm is solved can be found in the related paper.
A frame of the resulting background and mask for different variants of the overlaying model are shown in this figure. Additionally, last row shows the duality gap between the split variables and is an indicator for the convergence of the algorithm showing that all proposed methods converge within few tens of iterations. Histogram of the recovered mask for all different methods is shown in following figure. As we can observe, the non-convex priors lead to a separation of the values.
The matlab implementation is provided here.
A. Khalilian-Gourtani, S. Minaee, and Y. Wang. "Masked-RPCA: Moving Object Detection with an Overlaying Model", IEEE Open Journal of Signal Processing, vol. 1, 2020. pdf
A. Khalilian-Gourtani, S. Minaee, and Y. Wang. "Masked-RPCA: Sparse and low-rank decomposition under overlaying model and application to moving object detection." arXiv preprint, 2019. pdf