Stochastic neural networks for modelling random processes from observed data

Abstract

Most Artificial Neural Networks that are widely used today focus on approximating deterministic input-output mapping of nonlinear phenomena, and therefore, they can be well trained to represent the average behaviour of a nonlinear system. However, most natural phenomena are not only nonlinear but also highly variable. Deterministic neural networks do not adequately represent the variability observed in the natural settings of a system and therefore cannot capture the complexity of the whole system behaviour that is characterised by noise. This chapter implements a class of neural networks named Stochastic Neural Networks (SNNs) to simulate internal stochastic properties of natural and biological systems. Developing a suitable mathematical model for SNNs is based on the canonical representation of stochastic processes by means of Karhunen-Loève Theorem. In the implementation of this mathematical formulation for modelling nonlinear random processes from observed data, SNN is represented as a network of embedded deterministic neural networks, each representing a significant eigenfunction characterised by data, juxtaposed with random noise represented by White noise characterised by the corresponding eigenvalues. Two successful examples, including one from biology, are presented in the chapter to confirm the validity of the proposed SNN. Furthermore, analysis of internal working of SNNs provides an in-depth view of how SNNs work giving meaningful insights.