Nonlinear prediction and the Wiener process

A Wiener estimator structure is defined for the nonlinear prediction of scalar time series. This leads to a three stage strategy for constructing the predictor which involves: (1) estimation of the probability density function (pdf) associated with the embedding vector; (2) construction of a set of orthonormal polynomials defined on that pdf; (3) calculation of a set of coefficients to define a linear combiner which forms the prediction. The practical difficulties associated with (1) and the theoretical problems inherent in (2) lead to an approach based on a Volterra-type expansion. A set of orthonormal polynomials is constructed through eigenvalue analysis of the `correlation' matrix associated with the Volterra expansion. The orthonormal polynomials are defined by the eigenvalues and eigenvectors of this matrix -- the rank indicating the degree of the orthonormal approximation. Unlike the Wiener approach, the orthonormal polynomials are defined by the higher order moments present in the 'correlation' matrix rather than the pdf. The radial basis function (RBF) network is then reexamined in light of this interpretation. The pseudoinverse, which is often used to calculate the coefficients of the RBF network, is in fact a method for deriving a set of orthonormal functions. Since each RBF can itself be expanded as power series, the eigenvalue analysis inherent in the formation of the pseudo-inverse defines a set of infinite orthonormal functions. In theory at least, the RBF network exploits the entire pdf as demanded by the Wiener approach. Some simulation results are presented concerning the Volterra analysis, albeit using least squares rather than this revised approach.