Please use this identifier to cite or link to this item:
https://doi.org/10.3390/s20113126
Title: | Spatiotemporal interaction residual networks with pseudo3d for video action recognition | Authors: | Chen, J. Kong, J. Sun, H. Xu, H. Liu, X. Lu, Y. Zheng, C. |
Keywords: | Pseudo3D architecture Spatiotemporal representation learning Two-branches network Video action recognition |
Issue Date: | 2020 | Publisher: | MDPI AG | Citation: | Chen, J., Kong, J., Sun, H., Xu, H., Liu, X., Lu, Y., Zheng, C. (2020). Spatiotemporal interaction residual networks with pseudo3d for video action recognition. Sensors (Switzerland) 20 (11) : 3126. ScholarBank@NUS Repository. https://doi.org/10.3390/s20113126 | Rights: | Attribution 4.0 International | Abstract: | Action recognition is a significant and challenging topic in the field of sensor and computer vision. Two-stream convolutional neural networks (CNNs) and 3D CNNs are two mainstream deep learning architectures for video action recognition. To combine them into one framework to further improve performance, we proposed a novel deep network, named the spatiotemporal interaction residual network with pseudo3D (STINP). The STINP possesses three advantages. First, the STINP consists of two branches constructed based on residual networks (ResNets) to simultaneously learn the spatial and temporal information of the video. Second, the STINP integrates the pseudo3D block into residual units for building the spatial branch, which ensures that the spatial branch can not only learn the appearance feature of the objects and scene in the video, but also capture the potential interaction information among the consecutive frames. Finally, the STINP adopts a simple but effective multiplication operation to fuse the spatial branch and temporal branch, which guarantees that the learned spatial and temporal representation can interact with each other during the entire process of training the STINP. Experiments were implemented on two classic action recognition datasets, UCF101 and HMDB51. The experimental results show that our proposed STINP can provide better performance for video recognition than other state-of-the-art algorithms. © 2020 by the authors. | Source Title: | Sensors (Switzerland) | URI: | https://scholarbank.nus.edu.sg/handle/10635/196139 | ISSN: | 1424-8220 | DOI: | 10.3390/s20113126 | Rights: | Attribution 4.0 International |
Appears in Collections: | Elements Staff Publications |
Show full item record
Files in This Item:
File | Description | Size | Format | Access Settings | Version | |
---|---|---|---|---|---|---|
10_3390_s20113126.pdf | 2.5 MB | Adobe PDF | OPEN | None | View/Download |
This item is licensed under a Creative Commons License