Linear Layer


Specialization of the Layer for Linear Maps. The entries of the previous layer \(\mathbf{z}^{l-1}\in\mathbb{R}^{n_{l-1}}\) are multiplied by a weight matrix \(W\in\mathbb{R}^{n_{l}\times n_{l-1}}\) and a bias \(\mathbf{b}^l\in\mathbb{R}^{n_l}\).

\[\mathbf{z}^l = W^l \mathbf{z}^{l-1}+\mathbf{b}^l\]

If we denote the components of the vectors by \(z_i^{l-1}\) and \(z_j^{l}, b_j^l\) and for the matrix by \(W_{ij}^{l}\) for \(i=1,\dots,n_{l-1}\) and \(j=1,\dots,n_{l}\), this operation can be written as

\[z_j^l = W_{ij}^l z_i^{l-1}+b_j^l\]




These are settings required by this module.

Output Channels
  • Usage: e[“Output Channels”] = unsigned integer

  • Description: Indicates the size of the output vector produced by the layer.

Weight Scaling
  • Usage: e[“Weight Scaling”] = float

  • Description: Factor that is mutliplied by the layers’ weights.

  • Usage: e[“Engine”] = string

  • Description: Specifies which Neural Network backend engine to use.

  • Options:

    • Korali”: Uses Korali’s lightweight NN support. (CPU Sequential - Does not require installing third party software other than Eigen)

    • OneDNN”: Uses oneDNN as NN support. (CPU Sequential/Parallel - Requires installing oneDNN)

    • CuDNN”: Uses cuDNN as NN support. (GPU - Requires installing cuDNN)

  • Usage: e[“Mode”] = string

  • Description: Specifies the execution mode of the Neural Network.

  • Options:

    • Training”: Use for training. Stores data during forward propagation and allows backward propagation.

    • Inference”: Use for inference only. Only runs forward propagation. Faster for inference.

  • Usage: e[“Layers”] = knlohmann::json

  • Description: Complete description of the NN’s layers.

Timestep Count
  • Usage: e[“Timestep Count”] = unsigned integer

  • Description: Provides the sequence length for the input/output data.

Batch Sizes
  • Usage: e[“Batch Sizes”] = List of unsigned integer

  • Description: Specifies the batch sizes.

Default Configuration

These following configuration will be assigned by default. Any settings defined by the user will override the given settings specified in these defaults.

"Batch Sizes": [],
"Engine": "Korali",
"Input Values": [],
"Output Channels": 0,
"Uniform Generator": {
    "Maximum": 1.0,
    "Minimum": -1.0,
    "Type": "Univariate/Uniform"
"Weight Scaling": 1.0