Conclusion

Evaluation of the network performance relative to the linear approaches should be with some combination of in-sample and out-of-sample criteria, as well as by common sense criteria. We should never be afraid to ask how much these models add to our insight and understanding. Of course, we may use a neural network simply to forecast or simply to evaluate particular properties of the data, such as the significance of one or more input variables for explaining the behavior of the output variable....

Matlab Program Notes

Optimization software is quite common. The MATLAB function fminunc.m, for unconstrained minimization, part of the Optimization Toolbox, is the one used for the quasi-Newton gradient-based methods. It has lots of options, such as the specification of tolerance criteria and the maximum number of iterations. This function, like most software, is a minimization function. For maximizing a likelihood function, we minimize the negative of the likelihood function. The genetic algorithm used above is...

Matlab Example

To give the preceding regression diagnostics clearer focus, the following MATLAB code randomly generates a time series y sin(x)2 + exp( x) as a nonlinear function of a random variable x, then uses a linear regression model to approximate the model, and computes the in-sample diagnostic statistics. This program makes use of functions olsl.m, wnnestl.m, and bds.m, available on the webpage of the author. Create random regressors, constant term, Compute ols coefficients and diagnostics beta, tstat,...

Approximation with Polynomials and Neural Networks

We can see how efficient neural networks are relative to linear and polynomial approximations with a very simple example. We first generate a standard normal random variable x of sample size 1000, and then generate a variable y sin x 2 e-x. We can then do a series of regressions with polynomial approximators and a simple neural network with two neurons, and compare the multiple correlation coefficients. We do this with the following set of MATLAB commands, which access the following functions...

The Black Box Criticism and Data Mining

Like polynomial approximation, neural network estimation is often criticized as a black box. How do we justify the number of parameters, neurons, or hidden layers we use in a network How does the design of the network relate to priors based on underlying economic or financial theory Thomas Sargent 1997 , quoting Lucas's advice to researchers, reminds us to beware of economists bearing free parameters. By free, we mean parameters that cannot be justified or restricted on theoretical grounds....

Contents

1.1 Forecasting, Classification, and Dimensionality 1.3 The Interface 1.4 Plan of the Book 1 Econometric Foundations 11 2 What Are Neural Networks 13 2.1 Linear Regression 2.2 GARCH Nonlinear 2.2.1 Polynomial 2.2.2 Orthogonal 2.3 Model 2.4 What Is A Neural 2.4.1 Feedforward 2.4.2 Squasher 2.4.3 Radial Basis 2.4.4 Ridgelet 2.4.5 Jump 2.4.6 Multilayered Feedforward Networks 32 2.4.7 Recurrent 2.4.8 Networks with Multiple Outputs 36 2.5 Neural Network Smooth-Transition Regime Switching 2.5.1...