Extremely crowded scenes present unique challenges to video analysis that cannot be addressed with conventional approaches. We present a novel statistical framework for modeling the local spatio-temporal motion pattern behavior of extremely crowded scenes. Our key insight is to exploit the dense activity of the crowded scene by modeling the rich motion patterns in local areas, effectively capturing the underlying intrinsic structure they form in the video. In other words, we model the motion variation of local space-time volumes and their spatial-temporal statistical behaviors to characterize the overall behavior of the scene. We demonstrate that by capturing the steady-state motion be- havior with these spatio-temporal motion pattern models, we can naturally detect unusual activity as statistical deviations. Our experiments show that local spatio-temporal motion pattern modeling offers promising results in real-world scenes with complex activities that are hard for even human observers to analyze.
Anomaly Detection in Extremely Crowded Scenes Using Spatio-Temporal Motion Pattern Models
L. Kratz and K. Nishino,
in Proc. of IEEE Conference on Computer Vision and Pattern Recognition CVPR ‘09, pp1446-1453, Jun., 2009.
[ paper ][ project ]
Also see the following paper which introduces a better crowd flow model:
Going With the Flow: Pedestrian Efficiency in Crowded Scenes
L. Kratz and K. Nishino,
in Proc. of European Conference on Computer Vision ECCV’12, Part IV, pp558-572, Oct., 2012.
[ paper ][ video mov/avi ]
Overview
Our key insight is to exploit the dense local motion patterns created by the excessive number of subjects and model their spatio-temporal relationships, representing the underlying intrinsic structure they form in the video. For this, we first model the variations of local spatio-temporal motion patterns to describe common behavior within the scene, i.e., the crowd flow. Specifically, we construct motion-pattern distributions that capture the variations of local spatio-temporal motion patterns to compactly represent the video volume. We then derive a distribution-based HMM that describes natural motion transitions within local video regions.
We improve our framework by constructing a coupled HMM that models the spatial relationship of mo- tion patterns surrounding each video region.
Results
Unfortunately, we are not allowed to show the results in video format.
Our crowd flow model enables the detection of anomalous local motion in the scene as statistical deviations computed from its predictive likelihood. In this example, station officers walking against the flow of people are detected as conducting anomalous movements.
Having no motion in space-time locations where there should be specific crowd flow will also be detected as anomalous events. Here people getting stuck in ticket turnstiles are detected as having anomalous motion.
