Motion-shape-based deep learning approach for divergence behavior detection in high-density crowd

• Main Authors: Farooq, M.U., Saad, M.N.M., Khan, S.D.
• Format: Article
• Published: Springer Science and Business Media Deutschland GmbH 2022
• Online Access: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85101784657&doi=10.1007%2fs00371-021-02088-4&partnerID=40&md5=fba320c5864e7d239105a8037d70e2f9; http://eprints.utp.edu.my/33132/

We propose a novel method of abnormal crowd behavior detection in surveillance videos. Mainly, our work focuses on detecting crowd divergence behavior that can lead to serious disasters like a stampede. We introduce a notion of physically capturing motion in the form of images and classify crowd behavior using a convolution neural network (CNN) trained on motion-shape images (MSIs). First, the optical flow (OPF) is computed, and finite-time Lyapunov exponent (FTLE) field is obtained by integrating OPF. Lagrangian coherent structure (LCS) in the FTLE field represents crowd-dominant motion. A ridge extraction scheme is proposed for the conversion of LCS-to-grayscale MSIs. Lastly, a supervised training approach is utilized with CNN to predict normal or divergence behavior for any unknown image. We test our method on six real-world low- as well as high-density crowd datasets and compare performance with state-of-the-art methods. Experimental results show that our method is not only robust for any type of scene but also outperform existing state-of-the-art methods in terms of accuracy. We also propose a divergence localization method that not only identifies divergence starting (source) points but also comes with a new feature of generating a �localization mask� around the diverging crowd showing the size of divergence. Finally, we also introduce two new datasets containing videos of crowd normal and divergence behaviors at the high density. © 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH, DE part of Springer Nature.