1. A neural network acquires knowledge through learning.
2. A neural network's knowledge is stored within inter-neuron connection strengths known as synaptic weights.

The true power and advantage of neural networks lies in their ability to represent both linear and non-linear relationships and in their ability to learn these relationships directly from the data being modeled. Traditional linear models are simply inadequate when it comes to modeling data that contains non-linear characteristics.

 Return to Topic List

A.  Genetic algorithms are general-purpose search algorithms based upon the principles of evolution observed in nature. Even with today's high-powered computers, using an exhaustive search to find the optimal solution for even relatively small problems can be prohibitively expensive.

For many problems, genetic algorithms can often find good solutions (near-optimal) in around 100 generations. This can be many times faster than an exhaustive search. For example, a chromosome containing 32 binary genes would have 4,294,967,296 possible combinations (solutions) to evaluate when using an exhaustive search. If the same problem were to be solved with a genetic algorithm of population size 50, requiring 100 generations of evolution, the genetic algorithm would only need to evaluate 5000 possible solutions.

As mentioned previously, genetic algorithms are able to find optimal or near optimal solutions by using many of the principles of evolution that can be observed in nature. These include selection, crossover, and mutation. Below is an outline of how a genetic algorithm uses these principals to search for optimal solutions:

1. Create an initial population of chromosomes. Each chromosome is made up of a collection of genes (the parameters to be optimized) and represents a complete solution to the problem at hand. The gene values are usually initialized to random values within user-specified boundaries.
2. Evaluate each of the chromosomes in the initial population. A chromosome is evaluated by a fitness function to determine the quality of the solution. This fitness function is problem-specific and defines the genetic algorithm's objective for the current problem.
3. Select chromosomes that will have their information passed on to the next generation. The most common selection operator is “roulette selection”. This selection operator is based upon the evolutionary principle know as “survival of the fittest”, whereby a chromosome's probability of getting selected is proportional to its fitness. Thus, the good solutions are more likely to survive.
4. Crossover the selected chromosomes to produce new offspring chromosomes. This crossover occurs according to some user-defined probability (usually high) and results in new chromosomes having characteristics taken from both of the parent chromosomes. The most common crossover operator is “one point crossover”. This crossover operator picks a random point within the chromosomes, then switches the genes of the two chromosomes at this point to produce two new offspring. If an offspring takes the best parts from each of its parents, the result will likely be a better solution.
5. Mutate the genes of the offspring chromosomes. This mutation occurs according to some user-defined probability (usually low) and serves to introduce some variability into the gene pool. Without mutation, offspring chromosomes would be limited to only the genes available within the initial population. For real genes, one of the most common mutation operators simply adds a Gaussian distributed random value to the gene being mutated.
6. Repeat steps 3 through 5 until a new population of chromosomes has been created. Two chromosomes are selected, crossed-over, and mutated to produce two offspring until there are enough offspring to fill a population the same size as the last population.
7. Evaluate each of the chromosomes in the new population. Each of the chromosomes in the current population is evaluated by the fitness function, just as the initial population was evaluated in step 2.
8. Repeat steps 3 through 7 until some termination condition has been met. This termination condition can be based simply on the number of generations or it can be based upon more complex criteria such as the population convergence, fitness convergence, etc.

After the evolution of the initial population through many generations, the chromosomes (or solutions) within the final population will generally be much better as a whole than the chromosomes within the initial population. Also, the best chromosome in the final population will generally be near optimal if the genetic algorithm was run for enough generations.

 Return to Topic List

A.  Genetic algorithms can be combined with neural networks to enhance their performance by taking some of the guesswork out of optimally choosing neural network parameters, inputs etc. In general, genetic algorithms can be used in conjunction with neural networks in the following four ways. They can be used to choose the best inputs to the neural network, optimize the neural network parameters (such as the learning rates, number of hidden layer processing elements, etc.), train the actual network weights (rather than using backpropagation), or to choose/modify the neural network architecture. In each of these cases, the neural network error is the fitness used to rank the potential solutions.

 Return to Topic List

A.  NeuroSolutions includes two wizards to create neural networks for you based on your data and specifications. One of these wizards, the NeuralBuilder, centers the design specifications around the specific neural network architecture you wish to have built. Some of the most common neural network architectures include:

• Multilayer Perceptron (MLP)
• Generalized Feedforward
• Modular
• Jordan/Elman
• Principal Component Analysis (PCA)
• Radial Basis Function (RBF)
• General Regression Neural Network (GRNN)
• Probabilistic Neural Network (PNN)
• Self-Organizing Map (SOM)
• Time-Lag Recurrent Network (TLRN)
• Recurrent Network
• CANFIS Network (Fuzzy Logic)
• Support Vector Machine (SVM)

 Return to Topic List

A.  The second of the two network construction wizards is called the NeuralExpert. The NeuralExpert centers the design specifications around the type of problem you wish the neural network to solve (Classification, Prediction, Function Approximation or Clustering). Given this problem type and the size of your data set, the NeuralExpert intelligently selects the neural network size and architecture that will most likely produce a good solution. There is an optional beginner level that hides some of the more advanced operations such as cross validation and genetic optimization. In other words, you don't have to be a neural network expert to develop successful models with NeuroSolutions!

 Return to Topic List

A.  NeuroSolutions was designed with flexibility in mind. It is based on the concept that neural networks can be broken down into a fundamental set of neural components. The network construction wizards will connect these components for you based on your specifications. However, once the network is built you can arbitrarily change interconnections and/or add in new components. In other words, a virtually infinite number of neural models are possible!

In addition, the Developers and Developers Lite levels allow you to integrate your own algorithms into NeuroSolutions through dynamic link libraries (DLLs). Every NeuroSolutions component implements a function conforming to a simple protocol in C. To add a new component you simply modify the template function for the base component and compile the code into a DLL -- all directly from NeuroSolutions!

 Return to Topic List

A.  The best way to develop a neural network model from your Excel data is to use to use NeuroSolutions for Excel. This Microsoft Excel add-in simplifies and enhances the process of getting data into and out of a NeuroSolutions neural network. It benefits both the novice and the advanced neural network developer by allowing them to do everything from preprocessing the data, to designing the neural network, to generating reports of the results -- all directly from Excel!

If you are not interested in this add-in product you can still use your Excel data within NeuroSolutions. All you need to do is save a copy of your Excel spreadsheet as a text file (e.g., *.txt, *.csv).

 Return to Topic List

A.  Once you've created your neural network and trained and tested it on the historical data, you are now ready to put the neural network to use. What you want to do is give the neural network current input data as it becomes available, and have it generate an output that you can utilize. There are several ways you can interface with your NeuroSolutions neural network to put it into production:

• Within the Graphical User Interface of NeuroSolutions
  The TestingWizard automates the process of putting input data into a NeuroSolutions neural network and getting data out. You simply select the file that contains your new data and it will generate the network output, either to a file or to the screen.