Multilayer perceptron network construction

As I informed you in the preceding chapter, DL4J-based neural networks are made of multiple layers. Everything starts with a MultiLayerConfiguration, which organizes those layers and their hyperparameters.

Hyperparameters are a set of variables that determine how a neural network would learn. There are many parameters, for example, how many times and how often to update the weights of the model (called an epoch), how to initialize network weights, which activation function to be used, which updater and optimization algorithms to be used, the learning rate (that is, how fast the model should learn), how many hidden layers are there, how many neurons are there in each layer, and so on.

We now create the network. First, let us create the layers. Similar to the MLP we created in Chapter 1, Getting Started with Deep Learning, our MLP will have four layers:

• Layer 0: Input layer
• Lauer 1: Hidden layer 1
• Layer 2: Hidden layer 2
• Layer 3: Output layer

More technically, the first layer is the input layer, and then two layers are placed as hidden layers. For the first three layers, we initialized the weights using Xavier and the activation function is ReLU. Finally, the output layer is placed. This setting is shown in the following figure:

Multilayer perceptron for Titanic survival prediction input layer

We have specified the neurons (that is, nodes), which are an equal number of inputs, and an arbitrary number of neurons as output. We set a smaller value considering very few inputs and features:

DenseLayer input_layer = new DenseLayer.Builder()
                .weightInit(WeightInit.XAVIER)
                .activation(Activation.RELU)
                .nIn(numInputs)
                .nOut(16)
                .build();