Leveraging multi-head attention transformer deep neural network architecture for improved wind speed forecasting

Abstract

Wind energy has great potential for electricity generation, but its variability makes accurate wind speed forecasting essential for efficient integration. This study explores the application of a transformer-based deep learning model for wind speed forecasting. The model features an encoder-decoder architecture with multi-head attention, feed-forward layers, and normalization functions. By leveraging a self-attention mechanism, the transformer model effectively captures temporal dependencies in time series data through weighted relationships among input sequences, leading to improved forecasting accuracy. To evaluate its effectiveness, we collected and pre-processed wind speed data from the Hong Phong 1 wind power plant, cleaned the data by removing outliers and addressed missing values. The processed data was then embedded and added positional encoding to prepare for model input. The model was trained, and its performance was benchmarked against other models, including Long Short-Term Memory, Convolutional Neural Networks, and Artificial Neural Networks. The obtained RMSE is quite low, with 0,26 m/s for single-step forecast, 0,73 m/s for 4-step forecast and 1,70 m/s for 16-step forecast. These results demonstrated that the transformer model achieved superior predictive performance, suggesting it as a powerful alternative to traditional forecasting methods, with significant potential for enhancing the accuracy of wind speed predictions.