Application of recurrent network model on dynamic ...



Application of Recurrent Neural Network in Dynamic Modeling of Sensors

SHI MENG,TIAN SHEPING, JIANG PINGPING, YAN GUOZHENG

Department of Information Measurement Technology and Instruments

Shanghai Jiaotong University

Room 2201,Building 5,1915 Zhen Guang Rd.Shanghai,200333

CHINA

Abstract: - Dynamic modeling of sensors is an important aspect in the field of instrument technique. The recurrent neural network is proposed for nonlinear dynamic modeling of sensors,as its architecture is determined only by the number of nodes in the input, hidden and output layers. With the feedback behavior, the recurrent neural network can catch up with the dynamic response of the system. A recursive prediction error algorithm, which converges fast, is applied to training the recurrent neural network. Experimental results show that the the performance of the recurrent neural network model conforms to the sensor to be modeled, proving the method is not only effective but of high precision.

Key-Words: - Recurrent neural network; Sensor; Dynamic modeling; Recursive prediction error algorithm; Dynamic response; Mechanical sensor

1 Introduction

It is an important aspect to construct the model of a sensor with the dynamic calibrating data in the field of dynamic measurement[1]. Several methods have been applied to dynamic modeling of sensors. A common approach is to find the difference equation of a sensor using the discrete domain calibrating data. The transfer function of the sensor can be gotten from corresponding difference equation through bilinear transform. Above method is suitable for linear dynamic modeling of sensors and is useless when the sensors display nonlinear characteristics.

Several artificial neural network paradigms and neural learning schemes have been used in many dynamic system identification problems, and many promising results are reported. Most people made use of the feedforward neural network, combined with tapped delays, and the back propagation training algorithm to solve the dynamical problems[2]; however, the feedforward network is a static mapping and without the aid of tapped delays it does not represent a dynamic system mapping. On the other hand, the recurrent neural networks have important capabilities that are not found in feedforward networks, such as the ability to store information for later use and higher predicting precision. Thus the recurrent neural network is a dynamic mapping and is better suited for dynamical systems than the feedforward network.

This paper discusses the application of the recurrent neural networks in nonlinear dynamic modeling of sensors. With the feedback behavior, the recurrent neural network can capture the dynamic response of the system. A recursive prediction error algorithm, which converges fast, is applied to training the recurrent neural network. Experimental results show that the dynamic modeling method is effective.

2 Dynamic modeling of sensors based on recurrent neural network model

1. Recurrent neural network model

The fully connected recurrent neural network, however, where all neurons are coupled to one another, is difficult to train and to make it converge in a short time. With the requirement of fewer weights and a shorter training time for the neural network model, a simplified recurrent neural network is proposed. The architecture of the simplified version is a modified model of the fully connected recurrent neural network. It normally has an input layer, an output layer, and one hidden layer. The hidden layer is comprised of self-recurrent neurons, each feeding its output only into itself and not to other neurons in the hidden layer. On the other hand, the neurons in the output layer are linear type neurons, and do not have feedback weights. It was shown that with a proper choice of input and output weights, the simplified recurrent neural network paradigm can be made equivalent to a fully connected recurrent neural network. As shown in Fig.1, the recurrent neural network’s characteristics are determined by the connected weights of neighboring layers.

[pic]

Fig. 1 A recurrent neural network model

Each neuron in the hidden layer is a recurrent one with a nonlinear activation function.[pic] is the ith input; Sj (t), Xj (t) are the sum of input and output of the jth neuron respectively; O (t) represents the output of the whole network. The weighting vectors of the input layer, hidden layer and output layer compose the vector W={wI, wD, wO}. So the relation between the input and output can be described as:

[pic]

Where p and q refer to the numbers of network inputs and nodes in hidden layer respectively. For a recurrent network with single output, it has (p+2) q weights. g(.) is the activation function of neurons which is usually chosen as the sigmoid function:

[pic]

If the input of the recurrent network is denoted as {u (t-1), y’ (t-1)}, output is y’ (t), the input –output mapping of the recurrent neural network can be represented by:

[pic]

Where f (.) is a nonlinear function. So f (.) is a nonlinear dynamic mapping.

Fig. 2 shows the framework of dynamic modeling of sensors based on recurrent neural network model, in which y’ (t) is the output of the recurrent neural network model. The determination of weights can be achieved by applying certain training algorithm.

[pic]

Fig. 2 Dynamic modeling of sensors based on recurrent neural network model

2. Training algorithm of the recurrent neural network

Several methods such as back propagation algorithm can be used to estimate the weights of a recurrent network model. In this paper we utilize the recursive prediction error algorithm (RPE) to train the recurrent network. Prediction error method achieves the parameters’ estimates by minimizing the prediction error criterion. First, let’s define the prediction error as:

After N data have been recorded, a criterion function can be expressed by the following sum of squared prediction errors:

[pic] (6)

The unknown weighing vectors are updated along the Gauss-Newton search direction of [pic] to make J→min . The basic equation is:

Where s (t) is the step size, µ (w) represents the Gauss-Newton search direction.

Here the gradient of J(w) towards w is denoted as ▽J(w), H(w) is the second order derivative of J(w), namely the Hessian matrix of J(w).

It can be easily derived out that:

Where

H (w) can be calculated from the expression as follows:

The RPE algorithm based on above rules is described by following equations[3,4]:

p (t) is called the middle matrix, representing the covariance matrix of parameters when t→∞, whose initial value p (0) is usually chosen from the range of 104I to 105I, where I is the identity matrix.λ(t)is called the forgetting factor. It’s desirable to setλ(t) ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download