IEEE Spectrum Review

Copyright @ 1997 Institute of Electrical and Electronics Engineers. Reprinted, with permission, from IEEE Spectrum (Volume 34, Number 12, December 1997).

This material is posted here with permission of the IEEE. Such permission of the IEEE does not in any way imply IEEE endorsement of any of NeuroDimension's products or services. Internal or personal use of this material is permitted. However, permission to reprint/publish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by sending a blank email message to info.pub.permission@ieee.org.

By choosing to view this document, you agree to all provisions of the copyright laws protecting it.

Modeling Your Own Neural Net
DAN ELLIS

NeuroSolutions is one of the slickest and most complete packages for neural network simulation that anyone could wish for-and one of the most flexible as well. It is a visual-programming environment for designing, training, and investigating artificial neural networks, a family of algorithms for organizing data and for pattern classification. (The "neural" memorializes their original inspiration-the brain's ability to process intricate information through large complexes of simple elements.)

Interest in the use of neural networks for applications extending from speech recognition to stock market forecasts has already spawned a number of dedicated simulation packages. But with its third release in as many years, NeuroDimension Inc. has pushed the envelope by combining an extensive palette of processing blocks with an intuitive, icon-based design environment, lending credence to its claim of "virtually unlimited" range of possibilities.

The flexibility springs from the fine granularity of the model components. Rather than having a single unit for, say, a standard three-layer perceptron network, NeuroSolutions has separate components for different kinds of axons (weight matrices) and synapses (layers of nonlinear neuron units), which can be interconnected at will, then "stamped" with different training behaviors and schedules. The network structures available include generalized feed-forward, radial-basis function (where each unit is active over a limited Gaussian window), time-lagged recurrent (in which past inputs can contribute to subsequent patterns), and self-organizing (Kohonen) maps.

To escort the user through all this variety, there is the Neural Wizard, a tool that may be used to automate the assembly of a model through a sequence of simple dialogs. Running a simulation of the newly designed network is all the more instructive because of the superb range of so-called probes that can be attached anywhere in the network. The probes give real-time displays of neuron unit activations, error criteria, and other parameters, presented in such formats as tables, bar charts, or scatter plots, among others.

Performance and usability appear excellent: It was simple to set up a network similar to the pretty much standard speech phoneme classifier used in our group. I started with an 18-element cepstral feature vector, which is a transformation of the speech spectrum used as the basis for most speech recognition systems. I fed the network nine adjacent time frames for a total of 162 (9 by 16) input neuron units, followed by a 200-unit hidden layer, ending up with an estimate of the probability that the signal at that instant corresponds to each of 40 phonemes.

The Neural Wizard accepted almost without modification my ASCII-format file (an interface for exchanging data with the popular Excel spreadsheet comes with the package, but I did not test it). On a 133-MHz Pentium PC, a complete training epoch, in which the network learns to approximate from 16000 examples, took 4 minutes; our special-purpose in-house code, running on a SparcStation 5, is only four times as fast-and of course supports only a single network style and geometry.

The top-of-the-range developers' versions include the ability to generate source code, either for stand-alone operation, or to be included in larger applications. Thus I was able to design a network with the graphical tools on a PC, then compile code with identical geometry, nonlinearities, and training properties on a Unix workstation. The stand-alone code actually trained a little more slowly than the PC prototype, perhaps because it was less optimized; but the neat correspondence between the icons in my design and the C++ objects in the generated code make it conceivable that you could fiddle with the network, even at this level.

The description of the package on the NeuroDimension Web site includes a good, brief introduction to each of the basic network algorithms The full package comes with manuals detailing each available component as well as some more general material introducing neural network theory and giving advice on how to develop models.

But most of the information necessary can be got directly from the program; on being downloaded, the evaluation package starts with a demonstration document-written in a powerful macro language-that leads you through the basic elements in a clear and engaging manner, bringing you straight to a level where you're ready to start experimenting.

If you have access to a good PC and Internet link and want to spend an afternoon getting a feel for an assortment of neural net technologies, this would be an excellent place to start-and it's just a download away

Dan Ellis is a postdoctoral researcher at the International Computer Science Institute in Berkeley Calif., working on speech recognition and computational auditory scene analysis. His e-mail address is dpwe@icsi.berkeley.edu.

(c)IEEE reprinted from IEEE Spectrum (Volume 34, Number 12, December 1997)