Convolutional Layer

Specialization of the Layer for Convolution in Convolutional Neural Networks. The input image of size \(IH\times IW\) with \(IC\) channels is padded using \(PT\) zeros from the top, \(PL\) left, \(PB\) bottom and \(PR\) rights, respectively. The trainable convolution has size \(KH\times KW\) and \(OC\) channels and performs the operation using a stride of \(SV\) in vertical and \(SH\) in horizontal direction respectively.

The dimension \(OC\times OH\times OW\) of the resulting convoluted image can be computed as

\[OH = \lfloor \frac{IH - (KH - (PR + PL))}{SH}\rfloor + 1 OW = \lfloor \frac{IW - (KW - (PT + PB))}{SV}\rfloor + 1 OC = \frac{\text{_outputChannels}}{OH \cdot OW}\]

We note that the _outputChannels must be specified such that \(OC\) takes the wished value.

Usage

eLayer/Convolution"

Configuration

These are settings required by this module.

Image Height

  • Usage: e[“Image Height”] = s*unsigned integer*

  • Description: Height of the incoming 2D image.

Image Width

  • Usage: e[“Image Width”] = s*unsigned integer*

  • Description: Width of the incoming 2D image.

Kernel Height

  • Usage: e[“Kernel Height”] = s*unsigned integer*

  • Description: Height of the incoming 2D image.

Kernel Width

  • Usage: e[“Kernel Width”] = s*unsigned integer*

  • Description: Width of the incoming 2D image.

Vertical Stride

  • Usage: e[“Vertical Stride”] = s*unsigned integer*

  • Description: Strides for the image on the vertical dimension.

Horizontal Stride

  • Usage: e[“Horizontal Stride”] = s*unsigned integer*

  • Description: Strides for the image on the horizontal dimension.

Padding Left

  • Usage: e[“Padding Left”] = s*unsigned integer*

  • Description: Paddings for the image left side.

Padding Right

  • Usage: e[“Padding Right”] = s*unsigned integer*

  • Description: Paddings for the image right side.

Padding Top

  • Usage: e[“Padding Top”] = s*unsigned integer*

  • Description: Paddings for the image top side.

Padding Bottom

  • Usage: e[“Padding Bottom”] = s*unsigned integer*

  • Description: Paddings for the image Bottom side.

Output Channels

  • Usage: e[“Output Channels”] = unsigned integer

  • Description: Indicates the size of the output vector produced by the layer.

Weight Scaling

  • Usage: e[“Weight Scaling”] = float

  • Description: Factor that is mutliplied by the layers’ weights.

Engine

  • Usage: e[“Engine”] = string

  • Description: Specifies which Neural Network backend engine to use.

  • Options:

    • “Korali”: Uses Korali’s lightweight NN support. (CPU Sequential - Does not require installing third party software other than Eigen)

    • “OneDNN”: Uses oneDNN as NN support. (CPU Sequential/Parallel - Requires installing oneDNN)

    • “CuDNN”: Uses cuDNN as NN support. (GPU - Requires installing cuDNN)

Mode

  • Usage: e[“Mode”] = string

  • Description: Specifies the execution mode of the Neural Network.

  • Options:

    • “Training”: Use for training. Stores data during forward propagation and allows backward propagation.

    • “Inference”: Use for inference only. Only runs forward propagation. Faster for inference.

Layers

  • Usage: e[“Layers”] = knlohmann::json

  • Description: Complete description of the NN’s layers.

Timestep Count

  • Usage: e[“Timestep Count”] = unsigned integer

  • Description: Provides the sequence length for the input/output data.

Batch Sizes

  • Usage: e[“Batch Sizes”] = List of unsigned integer

  • Description: Specifies the batch sizes.

Default Configuration

These following configuration will be assigned by default. Any settings defined by the user will override the given settings specified in these defaults.

{
"Batch Sizes": [],
"Engine": "Korali",
"Input Values": [],
"Output Channels": 0,
"Uniform Generator": {
    "Maximum": 1.0,
    "Minimum": -1.0,
    "Name": "Neural Network / Uniform Generator",
    "Type": "Univariate/Uniform"
    },
"Weight Scaling": 1.0
}