A layer takes as input a tensor and returns a tensor through the forward function. Layers can hold variables, these are created and registered using a Var_store.t when creating the layer.

A layer of type t_with_training is similar to a layer of type t except that it is also given a boolean argument when applying it to a tensor that specifies whether the layer is currently used in training or in testing mode. This is typically the case for batch normalization or dropout.

with_training t returns a layer using the is_training argument from a standard layer. The is_training argument is discarded. This is useful when sequencing multiple layers via fold.

The identity layer. forward id tensor returns tensor.

The identity layer with an is_training argument.

of_fn f creates a layer based on a function from tensors to tensors.

of_fn_ f creates a layer based on a function from tensors to tensors. f also has access to the is_training flag.

sequential ts applies sequentially a list of layers ts.

sequential_ ts applies sequentially a list of layers ts.

forward t tensor applies layer t to tensor.

forward_ t tensor ~is_training applies layer t to tensor with the specified is_training flag.

The different kind of activations supported by the various layers below.

linear vs ~input_dim output_dim returns a linear layer. When using forward, the input tensor has to use a shape of batch_size * input_dim. The returned tensor has a shape batch_size * output_dim.

conv2d vs ~ksize ~stride ~input_dim output_dim returns a 2 dimension convolution layer. ksize specifies the kernel size and stride the stride. When using forward, the input tensor should have a shape batch_size * input_dim * h * w and the returned tensor will have a shape batch_size * output_dim * h' * w'.

conv2d_ is similar to conv2d but uses the same kernel size, stride, and padding on both the height and width dimensions, so a single integer needs to be specified for these parameters.

conv_transpose2d creates a 2D transposed convolution layer, this is sometimes also called 'deconvolution'.

conv_transpose2d_ is similar to conv_transpose2d but uses a single value for the height and width dimension for the kernel size, stride, padding and output padding.

batch_norm2d vs dim creates a batch norm 2D layer. This layer applies Batch Normalization over a 4D input batch_size * dim * h * w. The returned tensor has the same shape.

A Long Short Term Memory (LSTM) recurrent neural network.