DeepLearning: Deep Learning Toolbox

In xrobin/DeepLearning: Deep Learning of neural networks

Description

DeepLearning provides tools to train deep neural networks (or deep belief nets, DBN, represented by DeepBeliefNet objects) as a stack of restricted bolzman machines (RBM, represented by RestrictedBolzmannMachine objects) that provide coupling between two Layers. The bolzman machines and deep networks can be pre-trained (with pretrain methods provided by the package) and fine-tuned (see train) with the method proposed by Hinton and Salakhutdinov (Science, 2006) and refined by Bengio et al.

Classes

The DeepLearning package defines three classes: Layer, RestrictedBolzmannMachine and DeepBeliefNet. A RestrictedBolzmannMachine object represents one layer of the network that can be pre-trained independently. It stores the size of the input and output in two Layers, that represent the input and output size and type well as the type of Bolzmann machine (“binary”, “continuous” or “gaussian”). A deep belief network is an object of class DeepBeliefNet that compiles a set of one or more RestrictedBolzmannMachines with compatible Layers.

Functions and Methods

Initialization and Training

Topic	Function name(s)	RBM	DBN	Layer
Initialization	RestrictedBolzmannMachine, DeepBeliefNet, Layer	+	+	+
Combine	c	+	+	+
Pre-training	pretrain	+	+	-
Fine-tuning	train	-	+	-
Unrolling	unroll	-	+	-

Modeling functions

Topic	Function name(s)	RBM	DBN
Prediction	predict	+	+
Reconstruction	reconstruct	+	+
Error	error	+	+
Energy	energy	+	+
Resample	resample	+	+

Methods

The following methods extend standard R functions to perform various manipulations on the network:

Topic	Function name(s)	RBM	DBN	Layer
Drop	drop	-	+	-
Extract	see Extract	-	+	-
Length	length	-	+	+
Print	print	+	+	+
Reverse	rev	+	+	-
Comparison	see Comparison	-	-	+

References

1. Hinton GE, Salakhutdinov RR. Reducing the Dimensionality of Data with Neural Networks. Science. 2006 Jul 28;313(5786):504–7. http://dx.doi.org/10.1126/science.1127647.

2. Bengio Y, Lamblin P, Popovici D, Larochelle H. Greedy layer-wise training of deep networks. Advances in neural information processing systems. 2007;19:153–60.

Examples

library(mnist)
data(mnist)

# Initialize a 784-1000-500-250-30 layers DBN to process the MNIST data set
dbn.mnist <- DeepBeliefNet(
    Layers(c(784, 1000, 500, 250, 30),
           input="continuous",
           output="gaussian"))
dbn.mnist <- DeepBeliefNet(
    Layers(c(784, 1000, 500, 250, 30),
           input="continuous",
           output="gaussian"))
print(dbn.mnist)

# Pre-train this DBN
## Not run: 
pretrained.mnist <- pretrain(dbn.mnist, mnist$train$x, 
                       penalization = "l2", lambda=0.0002, epsilon=c(.1, .1, .1, .001), 
                       batchsize = 100, maxiters=100000)

## End(Not run)
data(pretrained.mnist) # Load an already pre-trained network

# Make predictions to 2 dimensions
predictions <- predict(pretrained.mnist, mnist$test$x)
# See how the data is reconstructed
reconstructions <- reconstruct(pretrained.mnist, mnist$test$x)
dim(predictions)
# And test the RMS error
error <- rmse(pretrained.mnist, mnist$test$x)
head(error)

# Plot predictions
plot.mnist(predictions = predictions, reconstructions = reconstructions)
par(family="mono")
legend("bottomleft", legend = sprintf("Mean error = %.3f", mean(error)), bty="n")

# Unrolling the network is the same as c(pretrained.mnist, rev(pretrained.mnist))
unrolled.mnist <- unroll(pretrained.mnist)
print(unrolled.mnist)

# Fine-tune the DBN with backpropagation
## Not run: 
trained.mnist <- train(unrolled.mnist, mnist$train$x, maxiters = 2000, batchsize = 1000,
                optim.control = list(maxit = 10))

## End(Not run)
data(trained.mnist) # Load an already trained network

# Make predictions to 2 dimensions
predictions <- predict(trained.mnist, mnist$test$x)
dim(predictions)
# Use reconstruct to pass through the whole unrolled network
reconstructions <- reconstruct(trained.mnist, mnist$test$x)
dim(reconstructions)

# test the RMS error
error <- rmse(trained.mnist, mnist$test$x)
head(error)

# Plot predictions
plot.mnist(predictions = predictions, reconstructions = reconstructions)
par(family="mono")
legend("bottomleft", legend = sprintf("Mean error = %.3f", mean(error)), bty="n")

# Perform various operations on it (not all of them make sense in this context)
combined <- c(trained.mnist, rev(trained.mnist[7:8]))
combined[[10]] <- RestrictedBolzmannMachine(Layer(1000, "binary"), Layer(784, "gaussian"))
print(combined)
length(combined)
drop(combined[4])

xrobin/DeepLearning documentation built on Sept. 18, 2020, 5:23 a.m.