Transformer-Based Quality Prediction Methods for Complex Industrial Processes: A Review

Deep learning based quality prediction has become one of the mainstream methods in the field of soft sensor, as it can more effectively address nonlinearity and high-dimensionality problems compared to traditional statistical analysis and shallow machine learning. Transformer, a special deep learning model based on the self-attention mechanism, offers a new research avenue for industrial quality prediction due to its unique advantages in sequence modeling. This paper systematically reviews the technological evolution and applications of Transformer in the industrial process quality prediction. Firstly, the limitations of classic deep learning models are discussed. By analyzing the ability of the Transformer model's self-attention mechanism to model dynamic global features of industrial processes, its fundamental principles in soft sensor problems are highlighted. Subsequently, we explore the main improvement directions of the classic Transformer developed in literatures, including the Crossformer, iTransformer, Informer, PatchTST, and Autoformer, and so forth, along with their applications in real-world processes. Finally, current challenges in Transformer-based quality prediction are discussed, and future research directions are prospected.