Morphochem

The Neural Networks Template Library NNTL

Contents

Description
Motivation
Design
Template Parameters
Training class
Data class
Header Files
Examples
Acknowledgements

Description

The Neural Networks Template Library provides a generic framework to write algorithms for the construction, configuration, and training of Artificial Neural Networks.
The NNTL was designed to be the backbone that different Neural Networks configurations will share (ie. full connected, partial connected, feedforward, feedback, kohonen, etc.) The NNTL was written using generic programming paradigm; the idea was to have a generic network and neuron classes, that multiple neural networks configurations share. In this context, if NNTL doesn't provide a NN configuration you are looking for, it will be easy to extend the present functionality to the one you need.

Motivation

The fundamental motivation is to provide the generic framework that will give the necessary mantainability, extensibility, and flexibility to authors of different NN configurations. The goal is that NNTL will evolve to include many NN configurations, and training algorithms, that the final user can easily implement.

Design

NNTL was structured in the following way:


  neural_network class <---------- neuron class
        ^                             ^
        |                             |
        |                             |
        |                             |
        |                             |
  network_detail class           neuron_detail class



Being the neural_network<> class the principal object to be instantiated by the final user.
Depending upon the selector parameters given to the class, different helper classes are selected on compile-time, meaning that the NN class will have different functionalities depending on which configurations for the NN is selected.

Template Parameters


neural_network<Data,LayerContainerS,NodesContainerS,ConnectionS,
                TypeS, InContainerS, OutContainerS, WeightsContainerS,
                InputsContainerS, TFType, PoinType, RandFuncType>
Parameter Description Default Models
Data Fundamental type on which calculations are performed double Floating point type
LayerContainerS Selector for the type of the layers container vecS Container Selector
NodesContainerS Selector for the type of the neurons container vecS Container Selector
ConnectionS Selector that indicates the type of connectivity for the Neural Network full_connectedS Connectivity Selector
TypeS Selector that indicates the type of the Neural Network feedforwardS NN Type Selector
InContainerS Selector for the type of the neuron's input connections container vecS Container Selector
OutContainerS Selector for the type of the neuron's output connections container vecS Container Selector
WeightsContainerS Selector for the type of the neuron's weights container vecS Container Selector
InputsContainerS Selector for the type of the neuron's inputs container vecS Container Selector
TFType Selector that specifies the transfer function for the Neural Network sigmoidS Transfer function Selector
PoinType Selector that specifies the type of the pointer to be used BoostSmrtptrS Pointer Selector
RandFuncType Selector that specifies the type of randomization function std_randS Randomization function Selector

Training class

No documentation available yet :( . Check out this page again in few months

Data class

The data<> class is an utility class that helps with the handling of vectors and matrices. It provides functions for the normalization of data sets.
Frequently, normalization of the training data is needed to guarantee a propper performance of the NN. The recommended normalization bounds is [0.1,0.9]. After the training is performed, usually we want to use the trained NN to carry out some predictions on an arbitrary data set. This data set must also be normalized to and from the same bounds the training sets were normalized. After prediction, the outputs of the NN must be de-normalized to the original bounds of the training set outputs; hence, the original bounds of the training sets must be preserved for future use of the NN.

Header Files

morpho/nntl/neural_network.hpp
morpho/nntl/data.hpp
morpho/nntl/training.hpp

Examples

Consider we have a data sets for training, and one to test the performance of the NN, and you want to create a NN to prognose the 'real' values:

  1. Include the headers :
    #include <neural_network.hpp>
    #include <data.hpp>
    #include <training.hpp>
    
  2. Typedef the nn, the training algorithm, and data classes if needed:
    typedef network::neural_network<> net_test;
    typedef data_ns::data<double>   data_class_type;
    typedef training::back_propagation<vector<vector<double> >::iterator, net_test>  bp;
    
  3. Create the variables:
    net_test          network_test;
    data_class_type   all_data;  // this matrix will be separated into different data-sets
    
  4. Create a container specifying the structure of the NN :
    std::vector<unsigned int>   ns;
    ns.push_back(7);   // 6 inputs (+ one bias)
    ns.push_back(15);  // 14 inputs (+ one bias)
    ns.push_back(1);   // one output
    
  5. Initialize the network :
    network_test.initialize_nn(ns.begin(), ns.end(), -1.0, network_test);
                                                       ^- bias value 
    
  6. If needed, normalize data and separate the data into training, control and test sets. The function select_data is overloaded, so you can exclude selecting the test set. First normalize the data vector. new_bounds and old_bounds are containers of data type std::pair first: min, second: max :
    std::vector<std::pair<double,double> > new_bounds(7,std::make_pair(0.1,0.9)); // bounds for the inputs + outputs
    std::vector<std::pair<double,double> > old_bounds;
    
    all_data.arr_linear_norm(all_data.sets.begin(), all_data.sets.end(),
          new_bounds.begin(), old_bounds.begin());
    all_data.select_data(3, 4, all_data.sets.begin(), all_data.sets.end(),
         train_data.sets, control_data.sets, test_data.sets);
    
    all_data.select_data() parameters :

  7. Create training class, and initialize containers for the learning algorithm:
    bp  backprop(network_test);
    backprop.initialize_containers(backprop);
    
  8. Call the training algorithm for one epoch:
    backprop.train(train_data.sets.begin(), train_data.sets.end(),
        control_data.sets.begin(), control_data.sets.end(), backprop,
        network_test, 0.0833, 0.5);
                      ^        ^- momentum value
                      |
                      - learning rate value
    
  9. If wanted, make predictions with the nn:
    vector<vector<double> > predicted_values(test_data.sets.size() );
    vector<vector<double> >::iterator pi;
    for (int i=0; i<predicted_values.size(); ++i) {
      predicted_values[i].resize(ns[2]);
    }
    pi=predicted_values.begin();
    for (int i=0; i<test_data.sets.size(); ++i) {
      // If the inputs and outputs are merged :
    	network_test.predict(test_data.sets[i].begin(), (test_data.sets[i].end() -1), pi->begin(), network_test);
      // If there's only inputs :
    	network_test.predict(test_data.sets[i].begin(), test_data.sets[i].end(), pi->begin(), network_test);
      ++pi;
    }
    
    1. Denormalize the predicted values:
      all_data.arr_linear_norm(predicted_values.begin(), predicted_values.end(),
            (new_bounds.begin()+(ns[0]-1)), (old_bounds.begin()+(ns[0]-1) ), 0);
      
    Neat!
    Now you can compare the predicted values in the predicted_values container, and the real values you had at the beginning.

Acknowledgements

I would like to thank Henrik Kuhn for the support and useful discussions while writing NNTL.
Copyright (c) Vladimir Josef Sykora & Morphochem AG 2001