Abstract:

Training neural networks can be viewed as an identification problem for a nonlinear dynamic system. The generic particle filter has been applied with success to neural network training, but the proposal distribution chosen by the generic particle filter does not incorporate the latest observations which can deteriorate the performance of the algorithm. In this paper, we propose to use the extended Kalman filter to generate proposal distribution in the particle filtering framework. The extended Kalman filter can make efficient use of the latest observations, and the generated proposal distribution can approximate the posterior distribution of neural network weights much better, which consequently improve the performance of the particle filter. The experimental results show that the proposed particle filter outperforms the generic particle filter.

Key words: multilayer perceptron, neural network training, extended Kalman particle filter