LSTM Ensemble Learning Model with Additive Gaussian Process Priors

Abstract

Deep learning technology is one of the key research directions in the field of machine learning, especially when dealing with high-dimensional and non-linear data prediction tasks, but it can not avoid the problem of overfitting and underfitting of prediction models. Taking LSTM neural network as an example, this paper proposes an ensemble learning model based on additive Gaussian process priors. On the one hand, the proposed method uses bootstrap technology to realize the randomization of neural network model, so that the network can capture effective information of predictor variables from multiple perspectives. On the other hand, this method takes the set of neural network models after randomization as a new input variable, and uses Gaussian process additive model to integrate the results of different models. By designing orthogonal additive kernel, the marginal effect and interaction effect of LSTM neural network are measured. In addition, the proposed method can quantitatively estimate the uncertainty of the forecast results. Simulation experiment and actual data analysis show that the proposed method is more competitive than some classical regression models.