A data engineering solution for micrometeorology forecasting using cloud recognition

• Main Author: Chai, Ai Jia
• Format: Final Year Project / Dissertation / Thesis
• Published: 2024
• Subjects: T Technology (General)
• Online Access: http://eprints.utar.edu.my/6629/1/fyp_CS_2024_CAJ.pdf; http://eprints.utar.edu.my/6629/

Micrometeorology forecasting stands at the forefront of accurate localized weather predictions, providing invaluable insights for individuals, communities and industries alike. Traditional weather forecast systems predominantly operate on mesoscale and synoptic meteorology, providing generalized information for large geographic regions. However, the micrometeorological conditions within smaller areas are often overlooked, leading to inaccuracies in local weather predictions. Apart from that, most of the existing methods rely on costly data collection and complex modelling. Thus, this has limited their accessibility and usability. Therefore, this project aims to address these pivotal challenges in weather forecasting. This project falls within the domain of micrometeorology and weather forecasting, focusing on developing a data engineering solution for the micrometeorology forecasting system and the recognition of cloud patterns to predict localized rainfall events. The proposed system employs a multi-faceted approach. It begins with the automatic collection of cloud images by capturing it with a camera module. After capturing, the images are automatically uploaded to AWS cloud storage platform. By leveraging machine learning techniques, the images will undergo a labelling process to categorize them as either rainy or not rainy. Manual labelling method will then be used to validate the accuracy of the previous annotation to create a ground truth dataset. Next, classification method will be utilized to classify the dataset into different classes. This approach empowers the system to label the new images automatically in future. Lastly, a predictive model will be trained to recognize the cloud formation patterns and predict the probability of the occurrence of the rainfall events. The novelties of this project encompass several groundbreaking aspects including the automated cloud recognition, localized and realtime predictions and the integration of data engineering methodologies to ensure the adaptability of the system to various geographical areas by facilitating the efficient collection, storage and labelling of cloud images. This streamlined process enables the generation of labeled datasets essential for training accurate predictive models, thus enhancing the system’s effectiveness across diverse environments. Moreover, this project also aims to offer a costeffective solution by using only a camera module to predict the rainfall events, making weather predictions more accessible to a broader audience.