mlpack_nn_SparseAutoencoder (3)

A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network.

  1. Carta.tech
  2. Man Pages
  3. Library calls
  4. mlpack_nn_SparseAutoencoder: A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network.
  1. Carta.tech
  2. Packages
  3. mlpack-doc
  4. mlpack_nn_SparseAutoencoder: A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network.

SYNOPSIS

Public Member Functions

SparseAutoencoder (const arma::mat &data, const size_t visibleSize, const size_t hiddenSize, const double lambda=0.0001, const double beta=3, const double rho=0.01)

Construct the sparse autoencoder model with the given training data. SparseAutoencoder (OptimizerType< SparseAutoencoderFunction > &optimizer)

Construct the sparse autoencoder model with the given training data. void Beta (const double b)

Sets the KL divergence parameter. double Beta () const

Gets the KL divergence parameter. void GetNewFeatures (arma::mat &data, arma::mat &features)

Transforms the provided data into the representation learned by the sparse autoencoder. void HiddenSize (const size_t hidden)

Sets size of the hidden layer. size_t HiddenSize () const

Gets the size of the hidden layer. void Lambda (const double l)

Sets the L2-regularization parameter. double Lambda () const

Gets the L2-regularization parameter. void Rho (const double r)

Sets the sparsity parameter. double Rho () const

Gets the sparsity parameter. void Sigmoid (const arma::mat &x, arma::mat &output) const

Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))]. void VisibleSize (const size_t visible)

Sets size of the visible layer. size_t VisibleSize () const

Gets size of the visible layer.

Private Attributes

double beta

KL divergence parameter. size_t hiddenSize

Size of the hidden layer. double lambda

L2-regularization parameter. arma::mat parameters

Parameters after optimization. double rho

Sparsity parameter. size_t visibleSize

Size of the visible layer.

Detailed Description

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS>class mlpack::nn::SparseAutoencoder< OptimizerType >

A sparse autoencoder is a neural network whose aim to learn compressed representations of the data, typically for dimensionality reduction, with a constraint on the activity of the neurons in the network.

Sparse autoencoders can be stacked together to learn a hierarchy of features, which provide a better representation of the data for classification. This is a method used in the recently developed field of deep learning. More technical details about the model can be found on the following webpage:

http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial

An example of how to use the interface is shown below:

arma::mat data; // Data matrix.
const size_t vSize = 64; // Size of visible layer, depends on the data.
const size_t hSize = 25; // Size of hidden layer, depends on requirements.

// Train the model using default options.
SparseAutoencoder encoder1(data, vSize, hSize);

const size_t numBasis = 5; // Parameter required for L-BFGS algorithm.
const size_t numIterations = 100; // Maximum number of iterations.

// Use an instantiated optimizer for the training.
SparseAutoencoderFunction saf(data, vSize, hSize);
L_BFGS<SparseAutoencoderFunction> optimizer(saf, numBasis, numIterations);
SparseAutoencoder<L_BFGS> encoder2(optimizer);

arma::mat features1, features2; // Matrices for storing new representations.

// Get new representations from the trained models.
encoder1.GetNewFeatures(data, features1);
encoder2.GetNewFeatures(data, features2);

This implementation allows the use of arbitrary mlpack optimizers via the OptimizerType template parameter.

Template Parameters:

OptimizerType The optimizer to use; by default this is L-BFGS. Any mlpack optimizer can be used here.

Definition at line 78 of file sparse_autoencoder.hpp.

Constructor & Destructor Documentation

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::\fBSparseAutoencoder\fP (const arma::mat &data, const size_tvisibleSize, const size_thiddenSize, const doublelambda = \fC0.0001\fP, const doublebeta = \fC3\fP, const doublerho = \fC0.01\fP)

Construct the sparse autoencoder model with the given training data. This will train the model. The parameters 'lambda', 'beta' and 'rho' can be set optionally. Changing these parameters will have an effect on regularization and sparsity of the model.

Parameters:

data Input data with each column as one example.

visibleSize Size of input vector expected at the visible layer.

hiddenSize Size of input vector expected at the hidden layer.

lambda L2-regularization parameter.

beta KL divergence parameter.

rho Sparsity parameter.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::\fBSparseAutoencoder\fP (OptimizerType< \fBSparseAutoencoderFunction\fP > &optimizer)

Construct the sparse autoencoder model with the given training data. This will train the model. This overload takes an already instantiated optimizer and uses it to train the model. The optimizer should hold an instantiated SparseAutoencoderFunction object for the function to operate upon. This option should be preferred when the optimizer options are to be changed.

Parameters:

optimizer Instantiated optimizer with instantiated error function.

Member Function Documentation

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> void \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::Beta (const doubleb)\fC [inline]\fP

Sets the KL divergence parameter.

Definition at line 170 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::beta.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> double \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::Beta () const\fC [inline]\fP

Gets the KL divergence parameter.

Definition at line 176 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::beta.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> void \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::GetNewFeatures (arma::mat &data, arma::mat &features)

Transforms the provided data into the representation learned by the sparse autoencoder. The function basically performs a feedforward computation using the learned weights, and returns the hidden layer activations.

Parameters:

data Matrix of the provided data.

features The hidden layer representation of the provided data.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> void \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::HiddenSize (const size_thidden)\fC [inline]\fP

Sets size of the hidden layer.

Definition at line 146 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> size_t \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::HiddenSize () const\fC [inline]\fP

Gets the size of the hidden layer.

Definition at line 152 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::hiddenSize.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> void \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::Lambda (const doublel)\fC [inline]\fP

Sets the L2-regularization parameter.

Definition at line 158 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::lambda.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> double \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::Lambda () const\fC [inline]\fP

Gets the L2-regularization parameter.

Definition at line 164 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::lambda.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> void \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::Rho (const doubler)\fC [inline]\fP

Sets the sparsity parameter.

Definition at line 182 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::rho.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> double \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::Rho () const\fC [inline]\fP

Gets the sparsity parameter.

Definition at line 188 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::rho.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> void \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::Sigmoid (const arma::mat &x, arma::mat &output) const\fC [inline]\fP

Returns the elementwise sigmoid of the passed matrix, where the sigmoid function of a real number 'x' is [1 / (1 + exp(-x))].

Parameters:

x Matrix of real values for which we require the sigmoid activation.

Definition at line 128 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> void \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::VisibleSize (const size_tvisible)\fC [inline]\fP

Sets size of the visible layer.

Definition at line 134 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> size_t \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::VisibleSize () const\fC [inline]\fP

Gets size of the visible layer.

Definition at line 140 of file sparse_autoencoder.hpp.

References mlpack::nn::SparseAutoencoder< OptimizerType >::visibleSize.

Member Data Documentation

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> double \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::beta\fC [private]\fP

KL divergence parameter.

Definition at line 203 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::Beta().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> size_t \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::hiddenSize\fC [private]\fP

Size of the hidden layer.

Definition at line 199 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::HiddenSize().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> double \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::lambda\fC [private]\fP

L2-regularization parameter.

Definition at line 201 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::Lambda().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> arma::mat \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::parameters\fC [private]\fP

Parameters after optimization.

Definition at line 195 of file sparse_autoencoder.hpp.

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> double \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::rho\fC [private]\fP

Sparsity parameter.

Definition at line 205 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::Rho().

template<template< typename > class OptimizerType = mlpack::optimization::L_BFGS> size_t \fBmlpack::nn::SparseAutoencoder\fP< OptimizerType >::visibleSize\fC [private]\fP

Size of the visible layer.

Definition at line 197 of file sparse_autoencoder.hpp.

Referenced by mlpack::nn::SparseAutoencoder< OptimizerType >::VisibleSize().

Author

Generated automatically by Doxygen for MLPACK from the source code.