Suspense scene detection using recurrent neural network

• Main Author: Lim, Sin Hui
• Format: Final Year Project / Dissertation / Thesis
• Published: 2021
• Subjects: QA75 Electronic computers. Computer science; T Technology (General)
• Online Access: http://eprints.utar.edu.my/4264/1/17ACB04528_FYP.pdf; http://eprints.utar.edu.my/4264/

Detecting the onset of suspenseful scenes is helpful for optimal ad placement. Some previous works that use movie scripts to imply suspense are somewhat deprived of contextual cues found in audio and video data. Meanwhile, the lack of a public video dataset for suspense scenes adds to the challenges to train ML-based suspense scene detection (SSD). In this project, an expert-annotated suspense scenes dataset containing videos from 3 classes (football, cooking, and room escape) is collected from YouTube. The dataset collection method follows the framework outlined by VSD2014 for dataset integrity. An SSD model is trained using custom RNN-LSTM using features extracted on ResNet50 for suspense scene detection in selected short YouTube videos. First, the minority classes are 'oversampled using a custom data balancing method to preserve these extrapolated frames' temporal sequence. Then, the AX library is used to brute-force the most optimal neural network configurations and hyperparameter tuning. The experimental results showed that the SSD model is highly accurate in detecting suspense scenes on unseen videos and generalized well, scoring a 0.7642 testing accuracy.