1. bookVolume 32 (2022): Issue 1 (March 2022)
Journal Details
License
Format
Journal
eISSN
2083-8492
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
access type Open Access

A Data Association Model for Analysis of Crowd Structure

Published Online: 31 Mar 2022
Volume & Issue: Volume 32 (2022) - Issue 1 (March 2022)
Page range: 81 - 94
Received: 16 May 2021
Accepted: 19 Oct 2021
Journal Details
License
Format
Journal
eISSN
2083-8492
First Published
05 Apr 2007
Publication timeframe
4 times per year
Languages
English
Abstract

The paper discusses a non-deterministic model for data association tasks in visual surveillance of crowds. Using detection and tracking of crowd components (i.e., individuals and groups) as baseline tools, we propose a simple algebraic framework for maintaining data association (continuity of labels assigned to crowd components) between subsequent video-frames in spite of possible disruptions and inaccuracies in tracking/detection algorithms. Formally, two alternative schemes (which, in practice, can be jointly used) are introduced, depending on whether individuals or groups can be prospectively better tracked in the current scenario. In the first scheme, only individuals are tracked, and the continuity of group labels is inferred without explicitly tracking the groups. In the second scheme, only group tracking is performed, and associations between individuals are inferred from group tracking. The associations are built upon non-deterministic estimates of memberships (individuals in groups) and estimates obtained directly from the baseline detection and tracking algorithms. The framework can incorporate any detectors and trackers (both classical or DL-based) as long as they can provide some geometric outlines (e.g., bounding boxes) of the crowd components. The formal analysis is supported by experiments in exemplary scenarios, where the framework provides meaningful performance improvements in various crowd analysis tasks.

Keywords

Bazzani, L., Cristani, M. and Murino, V. (2012). Decentralized particle filter for joint individual-group tracking, IEEE Conference on Computer Vision and Pattern Recognition, Providence, USA, pp. 1886–1893, DOI: 10.1109/CVPR.2012.6247888.10.1109/CVPR.2012.6247888 Search in Google Scholar

Benfold, B. and Reid, I. (2011). Stable multi-target tracking in real-time surveillance video, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, USA, pp. 3457–3464, DOI: 10.1109/CVPR.2011.5995667.10.1109/CVPR.2011.5995667 Search in Google Scholar

Berclaz, J., Fleuret, F., Turetken, E. and Fua, P. (2011). Multiple object tracking using k-shortest paths optimization, IEEE Transactions on Pattern Analysis and Machine Intelligence 33(9): 1806–1819, DOI: 10.1109/TPAMI.2011.21.10.1109/TPAMI.2011.2121282851 Search in Google Scholar

Bochinski, E., Senst, T. and Sikora, T. (2018). Extending IOU based multi-object tracking by visual information, 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Auckland, New Zealand, pp. 1–6, DOI: 10.1109/AVSS.2018.8639144.10.1109/AVSS.2018.8639144 Search in Google Scholar

Ciaparrone, G., Luque Sanchez, F., Tabik, S., Troiano, L., Tagliaferri, R. and Herrera, F. (2020). Deep learning in video multi-object tracking: A survey, Neurocomputing 381: 61–88, DOI: 10.1016/j.neucom.2019.11.023.10.1016/j.neucom.2019.11.023 Search in Google Scholar

Dalal, N. and Triggs, B. (2005). Histograms of oriented gradients for human detection, IEEE Conference on Computer Vision and Pattern Recognition, San Diego, USA, Vol. 1, pp. 886–893, DOI: 10.1109/CVPR.2005.177.10.1109/CVPR.2005.177 Search in Google Scholar

Dehghan, A., Modiri Assari, S. and Shah, M. (2015). GMMCP tracker: Globally optimal generalized maximum multi clique problem for multiple object tracking, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4091–4099, DOI: 10.1109/CVPR.2015.7299036.10.1109/CVPR.2015.7299036 Search in Google Scholar

Dollár, P., Appel, R., Belongie, S. and Perona, P. (2014). Fast feature pyramids for object detection, IEEE Transactions on Pattern Analysis and Machine Intelligence 36(8): 1532–1545, DOI: 10.1109/TPAMI.2014.2300479.10.1109/TPAMI.2014.230047926353336 Search in Google Scholar

Edman, V., Andersson, M., Granström, K. and Gustafsson, F. (2013). Pedestrian group tracking using the GM-PHD filter, European Signal Processing Conference (EUSIPCO), Marrakech, Morocco, pp. 1–5. Search in Google Scholar

Ferryman, J. and Shahrokni, A. (2009). PETS2009: Dataset and challenge, 12th IEEE International Workshop on Performance Evaluation of Tracking and Surveillance, Snowbird, USA, DOI: 10.1109/PETS-WINTER.2009.5399556.10.1109/PETS-WINTER.2009.5399556 Search in Google Scholar

Garcia-Martin, A., Sanchez-Matilla, R. and Martinez, J.M. (2017). Hierarchical detection of persons in groups, Signal, Image and Video Processing 11(7): 1181–1188, DOI: 10.1007/s11760-017-1073-z.10.1007/s11760-017-1073-z Search in Google Scholar

Ge, W., Collins, R.T. and Ruback, R.B. (2012). Vision-based analysis of small groups in pedestrian crowds, IEEE Transactions on Pattern Analysis and Machine Intelligence 34(5): 1003–1016, DOI: 10.1109/TPAMI.2011.176.10.1109/TPAMI.2011.17621844622 Search in Google Scholar

Gong, S., Han, H., Shan, S. and Chen, X. (2016). Actions recognition in crowd based on coarse-to-fine multi-object tracking, Asian Conference on Computer Vision, Taipei, Taiwan, pp. 478–490, DOI: 10.1007/978-3-319-54526-4_35.10.1007/978-3-319-54526-4_35 Search in Google Scholar

Heili, A. and Odobez, J.-M. (2013). Parameter estimation and contextual adaptation for a multi-object tracking CRF model, IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS), Clearwater, USA, pp. 14–21, DOI: 10.1109/PETS.2013.6523790.10.1109/PETS.2013.6523790 Search in Google Scholar

Hofmann, M., Haag, M. and Rigoll, G. (2013). Unified hierarchical multi-object tracking using global data association, IEEE International Workshop on Performance Evaluation of Tracking and Surveillance (PETS), Clearwater, USA, pp. 22–28, DOI: 10.1109/PETS.2013.6523791.10.1109/PETS.2013.6523791 Search in Google Scholar

Jacques, J.C.S., Braun, A., Soldera, J., Musse, S.R. and Jung, C.R. (2007). Understanding people motion in video sequences using Voronoi diagrams, Pattern Analysis and Applications 10(4): 321–332, DOI: 10.1007/s10044-007-0070-1.10.1007/s10044-007-0070-1 Search in Google Scholar

Kasprzak, W., Wilkowski, A. and Czapnik, K. (2012). Hand gesture recognition based on free-form contours and probabilistic inference, International Journal of Applied Mathematics and Computer Science 22(2): 437–448, DOI: 10.2478/v10006-012-0033-6.10.2478/v10006-012-0033-6 Search in Google Scholar

Li, D., Zhu, J., Xu, B., Lu, M. and Li, M. (2018). An ant-based filtering random-finite-set approach to simultaneous localization and mapping, International Journal of Applied Mathematics and Computer Science 28(3): 505–519, DOI: 10.2478/amcs-2018-0039.10.2478/amcs-2018-0039 Search in Google Scholar

Mazzon, R., Poiesi, F. and Cavallaro, A. (2013). Detection and tracking of groups in crowd, IEEE International Conference on Advanced Video and Signal Based Surveil-lance (AVSS), Krakow, Poland, pp. 202–207, DOI: 10.1109/AVSS.2013.6636640.10.1109/AVSS.2013.6636640 Search in Google Scholar

Milan, A., Roth, S. and Schindler, K. (2014). Continuous energy minimization for multitarget tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 36(1): 58–72, DOI: 10.1109/TPAMI.2013.103.10.1109/TPAMI.2013.10324231866 Search in Google Scholar

Park, M.-W. and Brilakis, I. (2016). Continuous localization of construction workers via integration of detection and tracking, Automation in Construction 72(Part 2): 129–142, DOI: 10.1016/j.autcon.2016.08.039.10.1016/j.autcon.2016.08.039 Search in Google Scholar

Raj, K.S. and Poovendran, R. (2014). Pedestrian detection and tracking through hierarchical clustering, International Conference on Information Communication and Embedded Systems, Chennai, India, pp. 1–4, DOI: 10.1109/ICICES.2014.7033991.10.1109/ICICES.2014.7033991 Search in Google Scholar

Ren, W., Kang, D., Tang, Y. and Chan, A.B. (2018). Fusing crowd density maps and visual object trackers for people tracking in crowd scenes, IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, pp. 5353–5362, DOI: 10.1109/CVPR.2018.00561.10.1109/CVPR.2018.00561 Search in Google Scholar

Rezatofighi, S.H., Milan, A., Zhang, Z., Shi, Q., Dick, A. and Reid, I. (2015). Joint probabilistic data association revisited, IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, pp. 3047–3055, DOI: 10.1109/ICCV.2015.349.10.1109/ICCV.2015.349 Search in Google Scholar

Rodriguez, M., Sivic, J., Laptev, I. and Audibert, J.-Y. (2011). Data-driven crowd analysis in videos, 2011 International Conference on Computer Vision, Barcelona, Spain, pp. 1235–1242, DOI: 10.1109/ICCV.2011.6126374.10.1109/ICCV.2011.6126374 Search in Google Scholar

Shao, J., Change Loy, C. and Wang, X. (2014). Scene-independent group profiling in crowd, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, USA, pp. 2219–2226, DOI: 10.1109/CVPR.2014.285.10.1109/CVPR.2014.285 Search in Google Scholar

Tang, S., Andriluka, M., Milan, A., Schindler, K., Roth, S. and Schiele, B. (2013). Learning people detectors for tracking in crowded scenes, Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia, pp. 1049–1056, DOI: 10.1109/ICCV.2013.134.10.1109/ICCV.2013.134 Search in Google Scholar

Wang, Q., Chen, M. and Li, X. (2017). Quantifying and detecting collective motion by manifold learning, AAAI Conference on Artificial Intelligence, San Francisco, USA, pp. 4292–4298, DOI: 10.5555/3298023.3298190. Search in Google Scholar

Wang, Q., Chen, M., Nie, F. and Li, X. (2020a). Detecting coherent groups in crowd scenes by multiview clustering, IEEE Transactions on Pattern Analysis and Machine Intelligence 42(1): 46–58, DOI: 10.1109/TPAMI.2018.2875002.10.1109/TPAMI.2018.287500230307858 Search in Google Scholar

Wang, Q., Gao, J., Lin, W. and Li, X. (2020b). NWPU-crowd: A large-scale benchmark for crowd counting and localization, IEEE Transactions on Pattern Analysis and Machine Intelligence 43(6): 2141–2149, DOI: 10.1109/TPAMI.2020.3013269.10.1109/TPAMI.2020.301326932750840 Search in Google Scholar

Wang, Q., Gao, J., Lin, W. and Yuan, Y. (2020c). Pixel-wise crowd understanding via synthetic data, International Journal of Computer Vision 129(1): 225–245, DOI: 10.1007/s11263-020-01365-4.10.1007/s11263-020-01365-4 Search in Google Scholar

Wen, L., Lei, Z., Lyu, S., Li, S. Z. and Yang, M.-H. (2016). Exploiting hierarchical dense structures on hypergraphs for multi-object tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence 38(10): 1983–1996, DOI: 10.1109/TPAMI.2015.2509979.10.1109/TPAMI.2015.250997926700969 Search in Google Scholar

Yang, B. and Nevatia, R. (2014). Multi-target tracking by online learning a CRF model of appearance and motion patterns, International Journal of Computer Vision 107(2): 203–217, DOI: 10.1007/s11263-013-0666-4.10.1007/s11263-013-0666-4 Search in Google Scholar

Yu, H., Zhou, Y., Simmons, J., Przybyla, C.P., Lin, Y., Fan, X., Mi, Y. and Wang, S. (2016). Groupwise tracking of crowded similar-appearance targets from low-continuity image sequences, IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, pp. 952–960, DOI: 10.1109/CVPR.2016.109.10.1109/CVPR.2016.109 Search in Google Scholar

Zhang, L., He, Z., Gu, M. and Yu, H. (2018). Crowd segmentation method based on trajectory tracking and prior knowledge learning, Arabian Journal for Science and Engineering 43(12): 7143–7152, DOI: 10.1007/s13369-017-2995-z.10.1007/s13369-017-2995-z Search in Google Scholar

Zhang, S., Wang, J., Wang, Z., Gong, Y. and Liu, Y. (2015). Multi-target tracking by learning local-to-global trajectory models, Pattern Recognition 48(2): 580–590, DOI: 10.1016/j.patcog.2014.08.013.10.1016/j.patcog.2014.08.013 Search in Google Scholar

Zhou, X., Zhuo, J. and Krahenbuhl, P. (2019). Bottom-up object detection by grouping extreme and center points, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, pp. 850–859, DOI: 10.1109/CVPR.2019.00094.10.1109/CVPR.2019.00094 Search in Google Scholar

Zhu, F., Wang, X. and Yu, N. (2014). Crowd tracking with dynamic evolution of group structures, European Conference on Computer Vision, Zurich, Switzerland, pp. 139–154, DOI: 10.1007/978-3-319-10599-4_10.10.1007/978-3-319-10599-4_10 Search in Google Scholar

Zhu, F., Wang, X. and Yu, N. (2018). Crowd tracking by group structure evolution, IEEE Transactions on Circuits and Systems for Video Technology 28(3): 772–786, DOI: 10.1109/TCSVT.2016.2615460.10.1109/TCSVT.2016.2615460 Search in Google Scholar

Zitouni, M.S., Bhaskar, H. and Al-Mualla, M.E. (2016). Robust background modeling and foreground detection using dynamic textures, International Conference on Computer Vision Theory and Applications (VISIGRAPP’16), Rome, Italy, pp. 403–410, DOI: 10.5220/0005724204030410.10.5220/0005724204030410 Search in Google Scholar

Zitouni, M.S., Sluzek, A. and Bhaskar, H. (2019a). CNN-based analysis of crowd structure using automatically annotated training data, IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Taipei, Taiwan, pp. 1–8, DOI: 10.1109/AVSS.2019.8909846.10.1109/AVSS.2019.8909846 Search in Google Scholar

Zitouni, M.S., Sluzek, A. and Bhaskar, H. (2019b). Visual analysis of socio-cognitive crowd behaviors for surveillance: A survey and categorization of trends and methods, Engineering Applications of Artificial Intelligence 82: 294–312, DOI: 10.1016/j.engappai.2019.04.012.10.1016/j.engappai.2019.04.012 Search in Google Scholar

Zitouni, M.S., Sluzek, A. and Bhaskar, H. (2020). Towards understanding socio-cognitive behaviors of crowds from visual surveillance data, Multimedia Tools and Applications 79(3): 1781–1799, DOI: 10.1007/s11042-019-08201-z.10.1007/s11042-019-08201-z Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo