In this work we propose algorithms to learn the locations of static occlusions and reason about both static and dynamic occlusion scenarios in multi-camera scenes for 3D surveillance (e.g., reconstruction, tracking). We will show that this leads to a computer system which is able to more effectively track (follow) objects in video when they are obstructed from some of the views. Because of the nature of the application area, our algorithm willbe under the constraints of using few cameras (no more than 3) that are configured wide-baseline.
Our algorithm consists of a learning phase, where a 3D probabilistic model of occlusions is estimated per-voxel, per-view over time via an EM-style framework. In this framework, at each frame the visual hull of the foreground objects (people) is computed via a Markov Random Field that integrates the occlusion model. The model is then updated at each frame using this solution, providing an iterative process that can accurately estimate the occlusion model by accumulating temporal information and overcome the few-camera constraint. We demonstrate the application of such a model to a number of areas, including visual hull reconstruction, 3D tracking, and the reconstruction of the occluding structures themselves.