Linear Layer¶
=
Specialization of the Layer for Linear Maps. The entries of the previous layer \(\mathbf{z}^{l-1}\in\mathbb{R}^{n_{l-1}}\) are multiplied by a weight matrix \(W\in\mathbb{R}^{n_{l}\times n_{l-1}}\) and a bias \(\mathbf{b}^l\in\mathbb{R}^{n_l}\).
If we denote the components of the vectors by \(z_i^{l-1}\) and \(z_j^{l}, b_j^l\) and for the matrix by \(W_{ij}^{l}\) for \(i=1,\dots,n_{l-1}\) and \(j=1,\dots,n_{l}\), this operation can be written as
Usage¶
eLayer/Linear"
Configuration¶
These are settings required by this module.
- Output Channels
Usage: e[“Output Channels”] = unsigned integer
Description: Indicates the size of the output vector produced by the layer.
- Weight Scaling
Usage: e[“Weight Scaling”] = float
Description: Factor that is mutliplied by the layers’ weights.
- Engine
Usage: e[“Engine”] = string
Description: Specifies which Neural Network backend engine to use.
Options:
“Korali”: Uses Korali’s lightweight NN support. (CPU Sequential - Does not require installing third party software other than Eigen)
“OneDNN”: Uses oneDNN as NN support. (CPU Sequential/Parallel - Requires installing oneDNN)
“CuDNN”: Uses cuDNN as NN support. (GPU - Requires installing cuDNN)
- Mode
Usage: e[“Mode”] = string
Description: Specifies the execution mode of the Neural Network.
Options:
“Training”: Use for training. Stores data during forward propagation and allows backward propagation.
“Inference”: Use for inference only. Only runs forward propagation. Faster for inference.
- Layers
Usage: e[“Layers”] = knlohmann::json
Description: Complete description of the NN’s layers.
- Timestep Count
Usage: e[“Timestep Count”] = unsigned integer
Description: Provides the sequence length for the input/output data.
- Batch Sizes
Usage: e[“Batch Sizes”] = List of unsigned integer
Description: Specifies the batch sizes.
Default Configuration¶
These following configuration will be assigned by default. Any settings defined by the user will override the given settings specified in these defaults.
{ "Batch Sizes": [], "Engine": "Korali", "Input Values": [], "Output Channels": 0, "Uniform Generator": { "Maximum": 1.0, "Minimum": -1.0, "Type": "Univariate/Uniform" }, "Weight Scaling": 1.0 }