Code Documentation

class korali::neuralNetwork::layer::Activation : public korali::neuralNetwork::Layer
#include <activation.hpp>

Class declaration for module: Activation.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void createForwardPipeline() override

Initializes the layer’s internal memory structures for the forward pipeline.

virtual void createBackwardPipeline() override

Initializes the internal memory structures for the backward pipeline.

virtual void forwardData(const size_t t) override

Performs the forward propagation of the Wx+b operations.

Parameters

t – Indicates the current timestep

virtual void backwardData(const size_t t) override

Performs the backward propagation of the data.

Parameters

t – Indicates the current timestep

Public Members

std::string _function

Indicates the activation function for the weighted inputs to the current layer.

float _alpha

First (alpha) argument to the activation function, as detailed in https://oneapi-src.github.io/oneDNN/dev_guide_eltwise.html.

float _beta

Second (beta) argument to the activation function, as detailed in https://oneapi-src.github.io/oneDNN/dev_guide_eltwise.html.

class korali::solver::optimizer::AdaBelief : public korali::solver::Optimizer
#include <AdaBelief.hpp>

Class declaration for module: AdaBelief.

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void processResult(double evaluation, std::vector<double> &gradient)

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters
  • evaluation – The value of the objective function at the current set of parameters

  • gradient – The gradient of the objective function at the current set of parameters

virtual void finalize() override

Finalizes Module. Deallocates memory and produces outputs.

virtual void setInitialConfiguration() override

Initializes the solver with starting values for the first generation.

virtual void runGeneration() override

Runs the current generation.

virtual void printGenerationBefore() override

Prints solver information before the execution of the current generation.

virtual void printGenerationAfter() override

Prints solver information after the execution of the current generation.

Public Members

double _beta1

Smoothing factor for momentum update.

double _beta2

Smoothing for gradient update.

double _eta

Learning Rate (Step Size)

double _epsilon

Term to facilitate numerical stability.

std::vector<double> _currentVariable

[Internal Use] Current value of parameters.

std::vector<double> _gradient

[Internal Use] Gradient of Function with respect to Parameters.

std::vector<double> _bestEverGradient

[Internal Use] Gradient of function with respect to Best Ever Variables.

double _gradientNorm

[Internal Use] Norm of gradient of function with respect to Parameters.

std::vector<double> _firstMoment

[Internal Use] Estimate of first moment of Gradient.

std::vector<double> _biasCorrectedFirstMoment

[Internal Use] Bias corrected estimate of first moment of Gradient.

std::vector<double> _secondCentralMoment

[Internal Use] Previous estimate of second moment of Gradient.

std::vector<double> _biasCorrectedSecondCentralMoment

[Internal Use] Bias corrected estimate of second moment of Gradient.

double _minGradientNorm

[Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

double _maxGradientNorm

[Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

class korali::solver::optimizer::Adam : public korali::solver::Optimizer
#include <Adam.hpp>

Class declaration for module: Adam.

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void processResult(double evaluation, std::vector<double> &gradient)

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters
  • evaluation – The value of the objective function at the current set of parameters

  • gradient – The gradient of the objective function at the current set of parameters

virtual void finalize() override

Finalizes Module. Deallocates memory and produces outputs.

virtual void setInitialConfiguration() override

Initializes the solver with starting values for the first generation.

virtual void runGeneration() override

Runs the current generation.

virtual void printGenerationBefore() override

Prints solver information before the execution of the current generation.

virtual void printGenerationAfter() override

Prints solver information after the execution of the current generation.

Public Members

double _beta1

Smoothing factor for momentum update.

double _beta2

Smoothing factor for gradient update.

double _eta

Learning Rate (Step Size)

double _epsilon

Term to facilitate numerical stability.

std::vector<double> _currentVariable

[Internal Use] Current value of parameters.

std::vector<double> _gradient

[Internal Use] Gradient of Function with respect to Parameters.

std::vector<double> _bestEverGradient

[Internal Use] Gradient of function with respect to Best Ever Variables.

std::vector<double> _squaredGradient

[Internal Use] Square of gradient of function with respect to Parameters.

double _gradientNorm

[Internal Use] Norm of gradient of function with respect to Parameters.

std::vector<double> _firstMoment

[Internal Use] Estimate of first moment of Gradient.

std::vector<double> _biasCorrectedFirstMoment

[Internal Use] Bias corrected estimate of first moment of Gradient.

std::vector<double> _secondMoment

[Internal Use] Old estimate of second moment of Gradient.

std::vector<double> _biasCorrectedSecondMoment

[Internal Use] Bias corrected estimate of second moment of Gradient.

double _minGradientNorm

[Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

double _maxGradientNorm

[Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

class korali::solver::Agent : public korali::Solver
#include <agent.hpp>

Class declaration for module: Agent.

Subclassed by korali::solver::agent::Continuous, korali::solver::agent::Discrete

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void normalizeStateActionNeuralNetwork(NeuralNetwork *neuralNetwork, size_t miniBatchSize, size_t normalizationSteps)

Mini-batch based normalization routine for Neural Networks with state and action inputs (typically critics)

Parameters
  • neuralNetwork – Neural Network to normalize

  • miniBatchSize – Number of entries in the normalization minibatch

  • normalizationSteps – How many normalization steps to perform (and grab the average)

void normalizeStateNeuralNetwork(NeuralNetwork *neuralNetwork, size_t miniBatchSize, size_t normalizationSteps)

Mini-batch based normalization routine for Neural Networks with state inputs only (typically policy)

Parameters
  • neuralNetwork – Neural Network to normalize

  • miniBatchSize – Number of entries in the normalization minibatch

  • normalizationSteps – How many normalization steps to perform (and grab the average)

void processEpisode(knlohmann::json &episode)

Additional post-processing of episode after episode terminated.

Parameters

episode – A vector of experiences pertaining to the episode.

std::vector<size_t> generateMiniBatch(size_t miniBatchSize)

Generates an experience mini batch from the replay memory.

Parameters

miniBatchSize – Size of the mini batch to create

Returns

A vector with the indexes to the experiences in the mini batch

void updateExperienceMetadata(const std::vector<size_t> &miniBatch, const std::vector<policy_t> &policyData)

Updates the state value, retrace, importance weight and other metadata for a given minibatch of experiences.

Parameters
  • miniBatch – The mini batch of experience ids to update

  • policyData – The policy to use to evaluate the experiences

void resetTimeSequence()

Resets time sequence within the agent, to forget past actions from other episodes.

virtual std::vector<policy_t> runPolicy(const std::vector<std::vector<std::vector<float>>> &stateBatch) = 0

Function to pass a state time series through the NN and calculates the action probabilities, along with any additional information.

Parameters

stateBatch – The batch of state time series (Format: BxTxS, B is batch size, T is the time series lenght, and S is the state size)

Returns

A JSON object containing the information produced by the policies given the current state series

size_t getTimeSequenceStartExpId(size_t expId)

Calculates the starting experience index of the time sequence for the selected experience.

Parameters

expId – The index of the latest experience in the sequence

Returns

The starting time sequence index

std::vector<std::vector<std::vector<float>>> getMiniBatchStateSequence(const std::vector<size_t> &miniBatch, const bool includeAction = false)

Gets a vector of states corresponding of time sequence corresponding to the provided last experience index.

Parameters
  • miniBatch – Indexes to the latest experiences in a batch of sequences

  • includeAction – Specifies whether to include the experience’s action in the sequence

Returns

The time step vector of states

std::vector<std::vector<float>> getTruncatedStateSequence(size_t expId)

Gets a vector of states corresponding of time sequence corresponding to the provided second-to-last experience index for which a truncated state exists.

Parameters

expId – The index of the second-to-latest experience in the sequence

Returns

The time step vector of states, including the truncated state

virtual float calculateImportanceWeight(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy) = 0

Calculates importance weight of current action from old and new policies.

Parameters
  • action – The action taken

  • curPolicy – The current policy

  • oldPolicy – The old policy, the one used for take the action in the first place

Returns

The importance weight

void attendAgent(const size_t agentId)

Listens to incoming experience from the given agent, sends back policy or terminates the episode depending on what’s needed.

Parameters

agentId – The Agent’s ID

void serializeExperienceReplay()

Serializes the experience replay into a JSON compatible format.

void deserializeExperienceReplay()

Deserializes a JSON object into the experience replay.

void trainingGeneration()

Runs a generation when running in training mode.

void testingGeneration()

Runs a generation when running in testing mode.

void rescaleStates()

Rescales states to have a zero mean and unit variance.

inline float getScaledReward(const size_t environmentId, const float reward)

Rescales a given reward by the square root of the sum of squarred rewards.

Parameters
  • environmentId – The id of the environment to which this reward belongs

  • reward – the input reward to rescale

Returns

The normalized reward

virtual void trainPolicy() = 0

Trains the Agent’s policy, based on the new experiences.

virtual knlohmann::json getAgentPolicy() = 0

Obtains the policy hyperaparamters from the learner for the agent to generate new actions.

Returns

The current policy hyperparameters

virtual void setAgentPolicy(const knlohmann::json &hyperparameters) = 0

Updates the agent’s hyperparameters.

Parameters

hyperparameters – The hyperparameters to update the agent.

virtual void initializeAgent() = 0

Initializes the internal state of the policy.

virtual void printAgentInformation() = 0

Prints information about the training policy.

virtual void getAction(korali::Sample &sample) = 0

Gathers the next action either from the policy or randomly.

Parameters

sampleSample on which the action and metadata will be stored

virtual void runGeneration() override

Runs the current generation.

virtual void printGenerationAfter() override

Prints solver information after the execution of the current generation.

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void finalize() override

Finalizes Module. Deallocates memory and produces outputs.

Public Members

std::string _mode

Specifies the operation mode for the agent.

std::vector<size_t> _testingSampleIds

A vector with the identifiers for the samples to test the hyperparameters with.

knlohmann::json _testingCurrentPolicy

The current hyperparameters of the policy to test.

size_t _trainingAverageDepth

Specifies the depth of the running training average to report.

size_t _concurrentEnvironments

Indicates the number of concurrent environments to use to collect experiences.

size_t _episodesPerGeneration

Indicates how many episodes to complete in a generation (checkpoints are generated between generations).

size_t _miniBatchSize

The number of experiences to randomly select to train the neural network(s) with.

std::string _miniBatchStrategy

Determines how to select experiences from the replay memory for mini batch creation.

size_t _timeSequenceLength

Indicates the number of contiguous experiences to pass to the NN for learning. This is only useful when using recurrent NNs.

float _learningRate

The initial learning rate to use for the NN hyperparameter optimization.

int _l2RegularizationEnabled

Boolean to determine if l2 regularization will be applied to the neural networks.

float _l2RegularizationImportance

Coefficient for l2 regularization.

knlohmann::json _neuralNetworkHiddenLayers

Indicates the configuration of the hidden neural network layers.

std::string _neuralNetworkOptimizer

Indicates the optimizer algorithm to update the NN hyperparameters.

std::string _neuralNetworkEngine

Specifies which Neural Network backend to use.

float _discountFactor

Represents the discount factor to weight future experiences.

float _importanceWeightTruncationLevel

Represents the discount factor to weight future experiences.

int _experienceReplaySerialize

Indicates whether to serialize and store the experience replay after each generation. Disabling will reduce I/O overheads but will disable the checkpoint/resume function.

size_t _experienceReplayStartSize

The minimum number of experiences before learning starts.

size_t _experienceReplayMaximumSize

The size of the replay memory. If this number is exceeded, experiences are deleted.

float _experienceReplayOffPolicyCutoffScale

Initial Cut-Off to classify experiences as on- or off-policy. (c_max in https://arxiv.org/abs/1807.05827)

float _experienceReplayOffPolicyTarget

Target fraction of off-policy experiences in the replay memory. (D in https://arxiv.org/abs/1807.05827)

float _experienceReplayOffPolicyAnnealingRate

Annealing rate for Off Policy Cutoff Scale and Learning Rate. (A in https://arxiv.org/abs/1807.05827)

float _experienceReplayOffPolicyREFERBeta

Initial value for the penalisation coefficient for off-policiness. (beta in https://arxiv.org/abs/1807.05827)

float _experiencesBetweenPolicyUpdates

The number of experiences to receive before training/updating (real number, may be less than < 1.0, for more than one update per experience).

int _stateRescalingEnabled

Determines whether to normalize the states, such that they have mean 0 and standard deviation 1 (done only once after the initial exploration phase).

int _rewardRescalingEnabled

Determines whether to normalize the rewards, such that they have mean 0 and standard deviation 1.

int _rewardOutboundPenalizationEnabled

If enabled, it penalizes the rewards for experiences with out of bound actions. This is useful for problems with truncated actions (e.g., openAI gym Mujoco) where out of bounds actions are clipped in the environment. This prevents policy means to extend too much outside the bounds.

float _rewardOutboundPenalizationFactor

The factor (f) by which te reward is scaled down. R = f * R.

size_t _policyParameterCount

[Internal Use] Stores the number of parameters that determine the probability distribution for the current state sequence.

std::vector<float> _actionLowerBounds

[Internal Use] Lower bounds for actions.

std::vector<float> _actionUpperBounds

[Internal Use] Upper bounds for actions.

size_t _currentEpisode

[Internal Use] Indicates the current episode being processed.

std::vector<float> _trainingRewardHistory

[Internal Use] Keeps a history of all training episode rewards.

std::vector<size_t> _trainingEnvironmentIdHistory

[Internal Use] Keeps a history of all training environment ids.

std::vector<size_t> _trainingExperienceHistory

[Internal Use] Keeps a history of all training episode experience counts.

float _trainingAverageReward

[Internal Use] Contains a running average of the training episode rewards.

float _trainingLastReward

[Internal Use] Remembers the cumulative sum of rewards for the last training episode.

float _trainingBestReward

[Internal Use] Remembers the best cumulative sum of rewards found so far in any episodes.

size_t _trainingBestEpisodeId

[Internal Use] Remembers the episode that obtained the maximum cumulative sum of rewards found so far.

knlohmann::json _trainingCurrentPolicy

[Internal Use] Stores the current training policy configuration.

knlohmann::json _trainingBestPolicy

[Internal Use] Stores the best training policy configuration found so far.

std::vector<float> _testingReward

[Internal Use] The cumulative sum of rewards obtained when evaluating the testing samples.

size_t _experienceReplayOffPolicyCount

[Internal Use] Number of off-policy experiences in the experience replay.

float _experienceReplayOffPolicyRatio

[Internal Use] Current off policy ratio in the experience replay.

float _experienceReplayOffPolicyCurrentCutoff

[Internal Use] Indicates the current cutoff to classify experiences as on- or off-policy

float _currentLearningRate

[Internal Use] The current learning rate to use for the NN hyperparameter optimization.

size_t _policyUpdateCount

[Internal Use] Keeps track of the number of policy updates that have been performed.

size_t _currentSampleID

[Internal Use] Keeps track of the current Sample ID, and makes sure no two equal sample IDs are produced such that this value can be used as random seed.

korali::distribution::univariate::Uniform *_uniformGenerator

[Internal Use] Uniform random number generator.

size_t _experienceCount

[Internal Use] Count of the number of experiences produced so far.

std::vector<size_t> _experienceCountPerEnvironment

[Internal Use] Count of the number of experiences in the replay memory per environment.

std::vector<float> _rewardRescalingSigma

[Internal Use] Contains the standard deviation of the rewards. They will be scaled by this value in order to normalize the reward distribution in the RM.

std::vector<float> _rewardRescalingSumSquaredRewards

[Internal Use] Sum of squared rewards in experience replay.

size_t _rewardOutboundPenalizationCount

[Internal Use] Keeps track of the number of out of bound actions taken.

std::vector<float> _stateRescalingMeans

[Internal Use] Contains the mean of the states. They will be shifted by this value in order to normalize the state distribution in the RM.

std::vector<float> _stateRescalingSigmas

[Internal Use] Contains the standard deviations of the states. They will be scaled by this value in order to normalize the state distribution in the RM.

size_t _maxEpisodes

[Termination Criteria] The solver will stop when the given number of episodes have been run.

size_t _maxExperiences

[Termination Criteria] The solver will stop when the given number of experiences have been gathered.

size_t _maxPolicyUpdates

[Termination Criteria] The solver will stop when the given number of optimization steps have been performed.

std::vector<Sample> _agents

Array of agents collecting new experiences.

std::vector<bool> _isAgentRunning

Keeps track of the age.

size_t _sessionExperienceCount

Session-specific experience count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionEpisodeCount

Session-specific episode count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionGeneration

Session-specific generation count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionPolicyUpdateCount

Session-specific policy update count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionExperiencesUntilStartSize

Session-specific counter that keeps track of how many experiences need to be obtained this session to reach the start training threshold.

cBuffer<std::vector<float>> _stateVector

Stores the state of the experience.

cBuffer<std::vector<float>> _actionVector

Stores the action taken by the agent at the given state.

cBuffer<std::vector<float>> _stateTimeSequence

Stores the current sequence of states observed by the agent (limited to time sequence length defined by the user)

cBuffer<size_t> _episodeIdVector

Episode that experience belongs to.

cBuffer<size_t> _episodePosVector

Position within the episode of this experience.

cBuffer<float> _importanceWeightVector

Contains the latest calculation of the experience’s importance weight.

cBuffer<float> _truncatedImportanceWeightVector

Contains the latest calculation of the experience’s truncated importance weight.

cBuffer<float> _priorityVector

For prioritized experience replay, this stores the experience’s priority.

cBuffer<float> _probabilityVector

For prioritized experience replay, this stores the experience’s probability.

cBuffer<policy_t> _curPolicyVector

Contains the most current policy information given the experience state.

cBuffer<policy_t> _expPolicyVector

Contains the policy information produced at the moment of the action was taken.

cBuffer<bool> _isOnPolicyVector

Indicates whether the experience is on policy, given the specified off-policiness criteria.

cBuffer<termination_t> _terminationVector

Specifies whether the experience is terminal (truncated or normal) or not.

cBuffer<float> _retraceValueVector

Contains the result of the retrace (Vtbc) function for the currrent experience.

cBuffer<float> _truncatedStateValueVector

If this is a truncated terminal experience, this contains the state value for that state.

cBuffer<std::vector<float>> _truncatedStateVector

If this is a truncated terminal experience, the truncated state is also saved here.

cBuffer<size_t> _environmentIdVector

Contains the environment id of every experience.

cBuffer<float> _rewardVector

Contains the rewards of every experience.

cBuffer<float> _stateValueVector

Contains the state value evaluation for every experience.

float _priorityAnnealingRate

Stores the priority annealing rate.

float _importanceWeightAnnealingRate

Stores the importance weight annealing factor.

problem::ReinforcementLearning *_problem

Storage for the pointer to the learning problem.

std::random_device rd

Random device for the generation of shuffling numbers.

std::mt19937 *mt

Mersenne twister for the generation of shuffling numbers.

double _sessionRunningTime

[Profiling] Measures the amount of time taken by the generation

double _sessionSerializationTime

[Profiling] Measures the amount of time taken by ER serialization

double _sessionAgentComputationTime

[Profiling] Stores the computation time per episode taken by Agents

double _sessionAgentCommunicationTime

[Profiling] Measures the average communication time per episode taken by Agents

double _sessionAgentPolicyEvaluationTime

[Profiling] Measures the average policy evaluation time per episode taken by Agents

double _sessionPolicyUpdateTime

[Profiling] Measures the time taken to update the policy in the current generation

double _sessionAgentAttendingTime

[Profiling] Measures the time taken to update the attend the agent’s state

double _generationRunningTime

[Profiling] Measures the amount of time taken by the generation

double _generationSerializationTime

[Profiling] Measures the amount of time taken by ER serialization

double _generationAgentComputationTime

[Profiling] Stores the computation time per episode taken by Agents

double _generationAgentCommunicationTime

[Profiling] Measures the average communication time per episode taken by Agents

double _generationAgentPolicyEvaluationTime

[Profiling] Measures the average policy evaluation time per episode taken by Agents

double _generationPolicyUpdateTime

[Profiling] Measures the time taken to update the policy in the current generation

double _generationAgentAttendingTime

[Profiling] Measures the time taken to update the attend the agent’s state

class korali::problem::Bayesian : public korali::Problem
#include <bayesian.hpp>

Class declaration for module: Bayesian.

Subclassed by korali::problem::bayesian::Custom, korali::problem::bayesian::Reference

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool runOperation(std::string operation, korali::Sample &sample) override

Runs the operation specified on the given sample. It checks recursively whether the function was found by the current module or its parents.

Parameters
  • sampleSample to operate on. Should contain in the ‘Operation’ field an operation accepted by this module or its parents.

  • operation – Should specify an operation type accepted by this module or its parents.

Returns

True, if operation found and executed; false, otherwise.

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void evaluate(korali::Sample &sample)

Produces a generic evaluation from the Posterior distribution of the sample, for optimization with CMAES, DEA, storing it in and stores it in sample[“F(x)”].

Parameters

sample – A Korali Sample

void evaluateLogPrior(korali::Sample &sample)

Evaluates the log prior of the given sample, and stores it in sample[“Log Prior”].

Parameters

sample – A Korali Sample

virtual void evaluateLoglikelihood(korali::Sample &sample) = 0

Evaluates the log likelihood of the given sample, and stores it in sample[“Log Likelihood”].

Parameters

sample – A Korali Sample

void evaluateLogPosterior(korali::Sample &sample)

Evaluates the log posterior of the given sample, and stores it in sample[“Log Posterior”].

Parameters

sample – A Korali Sample

virtual void evaluateGradient(korali::Sample &sample)

Evaluates the gradient of the objective w.r.t. to the variables, and stores it in sample[“Gradient”].

Parameters

sample – A Korali Sample

void evaluateLogPriorGradient(korali::Sample &sample)

Evaluates the gradient of the logPrior w.r.t. to the variables, and stores it in sample[“logPrior Gradient”].

Parameters

sample – A Korali Sample

inline virtual void evaluateLoglikelihoodGradient(korali::Sample &sample)

Evaluates the gradient of the logLikelikood w.r.t. to the variables, and stores it in sample[“logLikelihood Gradient”].

Parameters

sample – A Korali Sample

virtual void evaluateHessian(korali::Sample &sample)

Evaluates the hessian of the objective w.r.t. to the variables, and stores it in sample[“Hessian”].

Parameters

sample – A Korali Sample

void evaluateLogPriorHessian(korali::Sample &sample)

Evaluates the gradient of the logPrior w.r.t. to the variables, and stores it in sample[“logPrior Hessian”].

Parameters

sample – A Korali Sample

inline virtual void evaluateLogLikelihoodHessian(korali::Sample &sample)

Evaluates the gradient of the logLikelikood w.r.t. to the variables, and stores it in sample[“logLikelihood Hessian”].

Parameters

sample – A Korali Sample

inline virtual void evaluateFisherInformation(korali::Sample &sample)

Evaluates the empirical Fisher information.

Parameters

sample – A Korali Sample

class korali::distribution::univariate::Beta : public korali::distribution::Univariate
#include <beta.hpp>

Class declaration for module: Beta.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual double *getPropertyPointer(const std::string &property) override

Retrieves the pointer of a conditional value of a distribution property.

Parameters

property – Name of the property to find.

Returns

The pointer to the property..

virtual void updateDistribution() override

Updates the parameters of the distribution based on conditional variables.

virtual double getDensity(const double x) const override

Gets the probability density of the distribution at point x.

Parameters

x – point to evaluate P(x)

Returns

Value of the probability density.

virtual double getLogDensity(double x) const override

Gets the Log probability density of the distribution at point x.

Parameters

x – point to evaluate log(P(x))

Returns

Log of probability density.

virtual double getLogDensityGradient(double x) const override

Gets the Gradient of the log probability density of the distribution wrt. to x.

Parameters

x – point to evaluate grad(log(P(x)))

Returns

Gradient of log of probability density.

virtual double getLogDensityHessian(double x) const override

Gets the second derivative of the log probability density of the distribution wrt. to x.

Parameters

x – point to evaluate H(log(P(x)))

Returns

Hessian of log of probability density.

virtual double getRandomNumber() override

Draws and returns a random number from the distribution.

Returns

Random real number.

Public Members

double _alpha

[Conditional Variable Value]

std::string _alphaConditional

[Conditional Variable Reference]

double _beta

[Conditional Variable Value]

std::string _betaConditional

[Conditional Variable Reference]

template<typename valType, typename timerType>
struct korali::cacheElement_t
#include <kcache.hpp>

Struct that defines an element present in Korali’s cache structure.

Public Members

valType value

Value of the element.

timerType time

Time when the element was last updated.

class korali::distribution::univariate::Cauchy : public korali::distribution::Univariate
#include <cauchy.hpp>

Class declaration for module: Cauchy.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual double *getPropertyPointer(const std::string &property) override

Retrieves the pointer of a conditional value of a distribution property.

Parameters

property – Name of the property to find.

Returns

The pointer to the property..

virtual void updateDistribution() override

Updates the parameters of the distribution based on conditional variables.

virtual double getDensity(const double x) const override

Gets the probability density of the distribution at point x.

Parameters

x – point to evaluate P(x)

Returns

Value of the probability density.

virtual double getLogDensity(const double x) const override

Gets the Log probability density of the distribution at point x.

Parameters

x – point to evaluate log(P(x))

Returns

Log of probability density.

virtual double getLogDensityGradient(double x) const override

Gets the Gradient of the log probability density of the distribution wrt. to x.

Parameters

x – point to evaluate grad(log(P(x)))

Returns

Gradient of log of probability density.

virtual double getLogDensityHessian(double x) const override

Gets the second derivative of the log probability density of the distribution wrt. to x.

Parameters

x – point to evaluate H(log(P(x)))

Returns

Hessian of log of probability density.

virtual double getRandomNumber() override

Draws and returns a random number from the distribution.

Returns

Random real number.

Public Members

double _location

[Conditional Variable Value] Specifies the location of the peak of the distribution.

std::string _locationConditional

[Conditional Variable Reference] Specifies the location of the peak of the distribution.

double _scale

[Conditional Variable Value] Specifies the half-width at half-maximum (HWHM)

std::string _scaleConditional

[Conditional Variable Reference] Specifies the half-width at half-maximum (HWHM)

template<typename T>
class korali::cBuffer
#include <cbuffer.hpp>

This class defines a circular buffer with overwrite policy on add.

Public Functions

inline cBuffer()

Default constructor.

inline cBuffer(size_t size)

Constructor with a specific size.

Parameters

size – The buffer size

inline size_t size()

Returns the current number of elements in the buffer.

Returns

The number of elements

inline void resize(size_t maxSize)

Returns the current number of elements in the buffer.

Parameters

maxSize – The buffer size

inline void add(const T &v)

Adds an element to the buffer.

Parameters

v – The element to add

inline std::vector<T> getVector()

Returns the elements of the buffer in a vector format.

Returns

The vector with the circular buffer elements

inline void clear()

Eliminates all contents of the buffer.

inline T &operator[](size_t pos)

Accesses an element at the required position.

Parameters

pos – The access position

Returns

The element corresponding to the position

Private Members

size_t _maxSize

Size of buffer container.

size_t _size

Number of elements already added.

std::unique_ptr<T[]> _data

Container for data.

size_t _start

Position of the start of the buffer.

size_t _end

Position of the end of the buffer.

class korali::solver::optimizer::CMAES : public korali::solver::Optimizer
#include <CMAES.hpp>

Class declaration for module: CMAES.

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void prepareGeneration()

Prepares generation for the next set of evaluations.

void sampleSingle(size_t sampleIdx, const std::vector<double> &randomNumbers)

Evaluates a single sample.

Parameters
  • sampleIdx – Index of the sample to evaluate

  • randomNumbers – Random numbers to generate sample

void adaptC(int hsig)

Adapts the covariance matrix.

Parameters

hsig – Sign

void updateSigma()

Updates scaling factor of covariance matrix.

void updateDistribution()

Updates mean and covariance of Gaussian proposal distribution.

void updateEigensystem(const std::vector<double> &M)

Updates the system of eigenvalues and eigenvectors.

Parameters

M – Input matrix

void numericalErrorTreatment()

Method that checks potential numerical issues and does correction. Not yet implemented.

void eigen(size_t N, const std::vector<double> &C, std::vector<double> &diag, std::vector<double> &Q) const

Function for eigenvalue decomposition.

Parameters
  • N – Matrix size

  • C – Input matrix

  • diag – Sorted eigenvalues

  • Q – eingenvectors of C

void sort_index(const std::vector<double> &vec, std::vector<size_t> &_sortingIndex, size_t N) const

Descending sort of vector elements, stores ordering in _sortingIndex.

Parameters
  • _sortingIndex – Ordering of elements in vector

  • vec – Vector to sort

  • N – Number of current samples.

void initMuWeights(size_t numsamples)

Initializes the weights of the mu vector.

Parameters

numsamples – Length of mu vector

void initCovariance()

Initialize Covariance Matrix and Cholesky Decomposition.

void checkMeanAndSetRegime()

Check if mean of proposal distribution is inside of valid domain (does not violate constraints), if yes, re-initialize internal vars. Method for CCMA-ES.

void updateConstraints()

Update constraint evaluationsa. Method for CCMA-ES.

void updateViabilityBoundaries()

Update viability boundaries. Method for CCMA-ES.

void handleConstraints()

Process samples that violate constraints. Method for CCMA-ES.

void reEvaluateConstraints()

Reevaluate constraint evaluations. Called in handleConstraints. Method for CCMA-ES.

void updateDiscreteMutationMatrix()

Update mutation matrix for discrete variables. Method for discrete/integer optimization.

void discretize(std::vector<double> &sample)

Discretize variables to given granularity using arithmetic rounding.

Parameters

sampleSample to discretize

virtual void setInitialConfiguration() override

Configures CMA-ES.

virtual void runGeneration() override

Executes sampling & evaluation generation.

virtual void printGenerationBefore() override

Console Output before generation runs.

virtual void printGenerationAfter() override

Console output after generation.

virtual void finalize() override

Final console output at termination.

Public Members

size_t _populationSize

Specifies the number of samples to evaluate per generation (preferably $4+3*log(N)$, where $N$ is the number of variables).

size_t _muValue

Number of best samples (offspring samples) used to update the covariance matrix and the mean (by default it is half the Sample Count).

std::string _muType

Weights given to the Mu best values to update the covariance matrix and the mean.

double _initialSigmaCumulationFactor

Controls the learning rate of the conjugate evolution path (by default this variable is internally calibrated).

double _initialDampFactor

Controls the updates of the covariance matrix scaling factor (by default this variable is internally calibrated).

int _useGradientInformation

Include gradient information for proposal distribution update.

float _gradientStepSize

Scaling factor for gradient step, only relevant if gradient information used.

int _isSigmaBounded

Sets an upper bound for the covariance matrix scaling factor. The upper bound is given by the average of the initial standard deviation of the variables.

double _initialCumulativeCovariance

Controls the learning rate of the evolution path for the covariance update (must be in (0,1], by default this variable is internally calibrated).

int _diagonalCovariance

Covariance matrix updates will be optimized for diagonal matrices.

int _mirroredSampling

Generate the negative counterpart of each random number during sampling.

size_t _viabilityPopulationSize

Specifies the number of samples per generation during the viability regime, i.e. during the search for a parameter vector not violating the constraints.

size_t _viabilityMuValue

Number of best samples used to update the covariance matrix and the mean during the viability regime (by default this variable is half the Viability Sample Count).

size_t _maxCovarianceMatrixCorrections

Max number of covairance matrix adaptions per generation during the constraint handling loop.

double _targetSuccessRate

Controls the updates of the covariance matrix scaling factor during the viability regime.

double _covarianceMatrixAdaptionStrength

Controls the covariane matrix adaption strength if samples violate constraints.

double _normalVectorLearningRate

Learning rate of constraint normal vectors (must be in (0, 1], by default this variable is internally calibrated).

double _globalSuccessLearningRate

Learning rate of success probability of objective function improvements.

korali::distribution::univariate::Normal *_normalGenerator

[Internal Use] Normal random number generator.

korali::distribution::univariate::Uniform *_uniformGenerator

[Internal Use] Uniform random number generator.

int _isViabilityRegime

[Internal Use] True if mean is outside feasible domain. During viability regime CMA-ES is working with relaxed constraint boundaries that contract towards the true constraint boundaries.

std::vector<double> _valueVector

[Internal Use] Objective function values.

std::vector<std::vector<double>> _gradients

[Internal Use] Gradients of objective function evaluations.

size_t _currentPopulationSize

[Internal Use] Actual number of samples used per generation (Population Size or Viability Population Size).

size_t _currentMuValue

[Internal Use] Actual value of mu (Mu Value or Viability Mu Value).

std::vector<double> _muWeights

[Internal Use] Calibrated Weights for each of the Mu offspring samples.

double _effectiveMu

[Internal Use] Variance effective selection mass.

double _sigmaCumulationFactor

[Internal Use] Increment for sigma, calculated from muEffective and dimension.

double _dampFactor

[Internal Use] Dampening parameter controls step size adaption.

double _cumulativeCovariance

[Internal Use] Controls the step size adaption.

double _chiSquareNumber

[Internal Use] Expectation of $||N(0,I)||^2$.

size_t _covarianceEigenvalueEvaluationFrequency

[Internal Use] Establishes how frequently the eigenvalues are updated.

double _sigma

[Internal Use] Determines the step size.

double _trace

[Internal Use] The trace of the initial covariance matrix.

std::vector<std::vector<double>> _samplePopulation

[Internal Use] Sample coordinate information.

size_t _finishedSampleCount

[Internal Use] Counter of evaluated samples to terminate evaluation.

std::vector<double> _currentBestVariables

[Internal Use] Best variables of current generation.

double _previousBestValue

[Internal Use] Best model evaluation from previous generation.

double _previousBestEverValue

[Internal Use] Best ever model evaluation as of previous generation.

std::vector<size_t> _sortingIndex

[Internal Use] Sorted indeces of samples according to their model evaluation.

std::vector<double> _covarianceMatrix

[Internal Use] (Unscaled) covariance Matrix of proposal distribution.

std::vector<double> _auxiliarCovarianceMatrix

[Internal Use] Temporary Storage for Covariance Matrix.

std::vector<double> _covarianceEigenvectorMatrix

[Internal Use] Matrix with eigenvectors in columns.

std::vector<double> _auxiliarCovarianceEigenvectorMatrix

[Internal Use] Temporary Storage for Matrix with eigenvectors in columns.

std::vector<double> _axisLengths

[Internal Use] Axis lengths (sqrt(Evals))

std::vector<double> _auxiliarAxisLengths

[Internal Use] Temporary storage for Axis lengths.

std::vector<double> _bDZMatrix

[Internal Use] Temporary storage.

std::vector<double> _auxiliarBDZMatrix

[Internal Use] Temporary storage.

std::vector<double> _currentMean

[Internal Use] Current mean of proposal distribution.

std::vector<double> _previousMean

[Internal Use] Previous mean of proposal distribution.

std::vector<double> _meanUpdate

[Internal Use] Update differential from previous to current mean.

std::vector<double> _evolutionPath

[Internal Use] Evolution path for Covariance Matrix update.

std::vector<double> _conjugateEvolutionPath

[Internal Use] Conjugate evolution path for sigma update.

double _conjugateEvolutionPathL2Norm

[Internal Use] L2 Norm of the conjugate evolution path.

size_t _infeasibleSampleCount

[Internal Use] Keeps count of the number of infeasible samples.

double _maximumDiagonalCovarianceMatrixElement

[Internal Use] Maximum diagonal element of the Covariance Matrix.

double _minimumDiagonalCovarianceMatrixElement

[Internal Use] Minimum diagonal element of the Covariance Matrix.

double _maximumCovarianceEigenvalue

[Internal Use] Maximum Covariance Matrix Eigenvalue.

double _minimumCovarianceEigenvalue

[Internal Use] Minimum Covariance Matrix Eigenvalue.

int _isEigensystemUpdated

[Internal Use] Flag determining if the covariance eigensystem is up to date.

std::vector<std::vector<int>> _viabilityIndicator

[Internal Use] Evaluation of each constraint for each sample.

int _hasConstraints

[Internal Use] True if the number of constraints is higher than zero.

double _covarianceMatrixAdaptionFactor

[Internal Use] This is the beta factor that indicates how fast the covariance matrix is adapted.

int _bestValidSample

[Internal Use] Index of best sample without constraint violations (otherwise -1).

double _globalSuccessRate

[Internal Use] Estimated Global Success Rate, required for calibration of covariance matrix scaling factor updates.

double _viabilityFunctionValue

[Internal Use] Viability Function Value.

size_t _resampledParameterCount

[Internal Use] Number of resampled parameters due constraint violation.

size_t _covarianceMatrixAdaptationCount

[Internal Use] Number of Covariance Matrix Adaptations.

std::vector<double> _viabilityBoundaries

[Internal Use] Viability Boundaries.

std::vector<int> _viabilityImprovement

[Internal Use] Sample evaluations larger than fviability.

size_t _maxConstraintViolationCount

[Internal Use] Temporary counter of maximal amount of constraint violations attained by a sample (must be 0).

std::vector<size_t> _sampleConstraintViolationCounts

[Internal Use] Maximal amount of constraint violations.

std::vector<std::vector<double>> _constraintEvaluations

[Internal Use] Functions to be evaluated as constraint evaluations, if the return from any of them is > 0, then the constraint is met.

std::vector<std::vector<double>> _normalConstraintApproximation

[Internal Use] Normal approximation of constraints.

std::vector<double> _bestConstraintEvaluations

[Internal Use] Constraint evaluations for best ever.

int _hasDiscreteVariables

[Internal Use] Flag indicating if at least one of the variables is discrete.

std::vector<double> _discreteMutations

[Internal Use] Vector storing discrete mutations, required for covariance matrix update.

size_t _numberOfDiscreteMutations

[Internal Use] Number of discrete mutations in current generation.

size_t _numberMaskingMatrixEntries

[Internal Use] Number of nonzero entries on diagonal in Masking Matrix.

std::vector<double> _maskingMatrix

[Internal Use] Diagonal Matrix signifying where an integer mutation may be conducted.

std::vector<double> _maskingMatrixSigma

[Internal Use] Sigma of the Masking Matrix.

double _chiSquareNumberDiscreteMutations

[Internal Use] Expectation of $||N(0,I^S)||^2$ for discrete mutations.

double _currentMinStandardDeviation

[Internal Use] Current minimum standard deviation of any variable.

double _currentMaxStandardDeviation

[Internal Use] Current maximum standard deviation of any variable.

size_t _constraintEvaluationCount

[Internal Use] Number of Constraint Evaluations.

size_t _maxInfeasibleResamplings

[Termination Criteria] Maximum number of resamplings per candidate per generation if sample is outside of Lower and Upper Bound.

double _maxConditionCovarianceMatrix

[Termination Criteria] Specifies the maximum condition of the covariance matrix.

double _minStandardDeviation

[Termination Criteria] Specifies the minimal standard deviation for any variable in any proposed sample.

double _maxStandardDeviation

[Termination Criteria] Specifies the maximal standard deviation for any variable in any proposed sample.

class korali::conduit::Concurrent : public korali::Conduit
#include <concurrent.hpp>

Class declaration for module: Concurrent.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool isRoot() override

Determines whether the caller rank/thread/process is root.

Returns

True, if it is root; false, otherwise.

virtual void initServer() override

Initializes the worker/server bifurcation in the conduit.

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void terminateServer() override

Finalizes the workers.

virtual void stackEngine(Engine *engine) override

Stacks a new Engine into the engine stack.

Parameters

engine – A Korali Engine

virtual void popEngine() override

Pops the current Engine from the engine stack.

virtual void listenWorkers() override

(Engine <- Worker) Receives all pending incoming messages and stores them into the corresponding sample’s message queue.

virtual void broadcastMessageToWorkers(knlohmann::json &message) override

(Engine -> Worker) Broadcasts a message to all workers

Parameters

message – JSON object with information to broadcast

virtual void sendMessageToEngine(knlohmann::json &message) override

(Sample -> Engine) Sends an update to the engine to provide partial information while the sample is still active

Parameters

message – Message to send to engine

virtual knlohmann::json recvMessageFromEngine() override

(Sample <- Engine) Blocking call that waits until any message incoming from the engine.

Returns

message from the engine.

virtual void sendMessageToSample(Sample &sample, knlohmann::json &message) override

(Engine -> Sample) Sends an update to a still active sample

Parameters
  • sample – The sample from which to receive an update

  • message – Message to send to the sample.

virtual size_t getProcessId() override

Returns the identifier corresponding to the executing process (to differentiate their random seeds)

Returns

The executing process id

Public Members

size_t _concurrentJobs

Specifies the number of worker processes (jobs) running concurrently.

std::vector<pid_t> _workerPids

PID of worker processes.

int _workerId

Worker Id for current workers - 0 for the master process.

std::vector<std::vector<int>> _resultContentPipe

OS Pipe to handle result contents communication coming from worker processes.

std::vector<std::vector<int>> _resultSizePipe

OS Pipe to handle result size communication coming from worker processes.

std::vector<std::vector<int>> _inputsPipe

OS Pipe to handle sample parameter communication to worker processes.

struct korali::problem::hierarchical::Psi::conditionalPriorInfo

Stores the pre-computed positions (pointers) of the conditional priors to evaluate for performance.

Public Members

std::vector<size_t> _samplePositions

Stores the position of the conditional prior.

std::vector<double*> _samplePointers

Stores the pointer of the conditional prior.

class korali::Conduit : public korali::Module
#include <conduit.hpp>

Class declaration for module: Conduit.

Subclassed by korali::conduit::Concurrent, korali::conduit::Distributed, korali::conduit::Sequential

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void worker()

Lifetime function for korali workers.

inline virtual bool isRoot()

Determines whether the caller rank/thread/process is root.

Returns

True, if it is root; false, otherwise.

inline virtual bool isWorkerLeadRank()

Determines whether the caller rank is the leader of its worker root.

Returns

True, if it is the worker leader rank; false, otherwise.

void workerProcessSample(const knlohmann::json &js)

(Worker Side) Starts the processing of a sample at the worker side

Parameters

js – Contains sample’s input data and metadata

void workerStackEngine(const knlohmann::json &js)

(Worker Side) Accepts and stacks an incoming Korali engine from the main process

Parameters

js – Contains Engine’s input data and metadata

void workerPopEngine()

(Worker Side) Pops the top of the engine stack

void start(Sample &sample)

Starts the execution of the sample.

Parameters

sample – A Korali sample

void wait(Sample &sample)

Waits for a given sample to finish. The experiment will not continue until the sample has been evaluated.

Parameters

sample – A Korali sample

void waitAll(std::vector<Sample> &samples)

Waits for a set of sample to finish. The experiment will not continue until all samples have been evaluated.

Parameters

samples – A list of Korali samples

size_t waitAny(std::vector<Sample> &samples)

Waits for a set of sample to finish. The experiment will not continue until at least one of the samples have been evaluated.

Parameters

samples – A list of Korali samples

Returns

Position in the vector of the sample that has finished.

virtual void stackEngine(Engine *engine) = 0

Stacks a new Engine into the engine stack.

Parameters

engine – A Korali Engine

virtual void popEngine() = 0

Pops the current Engine from the engine stack.

void runSample(Sample *sample, Engine *engine)

Starts the execution of a sample, given an Engine.

Parameters
  • sample – the sample to execute

  • engine – The Korali engine to use for its execution

virtual void initServer() = 0

Initializes the worker/server bifurcation in the conduit.

virtual void terminateServer() = 0

Finalizes the workers.

virtual void broadcastMessageToWorkers(knlohmann::json &message) = 0

(Engine -> Worker) Broadcasts a message to all workers

Parameters

message – JSON object with information to broadcast

virtual void listenWorkers() = 0

(Engine <- Worker) Receives all pending incoming messages and stores them into the corresponding sample’s message queue.

void listen(std::vector<Sample> &samples)

Start pending samples and retrieve any pending messages for them.

Parameters

samples – The set of samples

virtual void sendMessageToEngine(knlohmann::json &message) = 0

(Sample -> Engine) Sends an update to the engine to provide partial information while the sample is still active

Parameters

message – Message to send to engine

virtual knlohmann::json recvMessageFromEngine() = 0

(Sample <- Engine) Blocking call that waits until any message incoming from the engine.

Returns

message from the engine.

virtual void sendMessageToSample(Sample &sample, knlohmann::json &message) = 0

(Engine -> Sample) Sends an update to a still active sample

Parameters
  • sample – The sample from which to receive an update

  • message – Message to send to the sample.

virtual size_t getProcessId() = 0

Returns the identifier corresponding to the executing process (to differentiate their random seeds)

Returns

The executing process id

Public Members

std::queue<size_t> _workerQueue

Queue to store idle workers to assign samples to.

std::map<size_t, Sample*> _workerToSampleMap

Map that links workers to their currently-executing sample.

Public Static Functions

static void coroutineWrapper()

Wrapper function for the sample coroutine.

class korali::problem::reinforcementLearning::Continuous : public korali::problem::ReinforcementLearning
#include <continuous.hpp>

Class declaration for module: Continuous.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool runOperation(std::string operation, korali::Sample &sample) override

Runs the operation specified on the given sample. It checks recursively whether the function was found by the current module or its parents.

Parameters
  • sampleSample to operate on. Should contain in the ‘Operation’ field an operation accepted by this module or its parents.

  • operation – Should specify an operation type accepted by this module or its parents.

Returns

True, if operation found and executed; false, otherwise.

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

class korali::solver::agent::Continuous : public korali::solver::Agent
#include <continuous.hpp>

Class declaration for module: Continuous.

Subclassed by korali::solver::agent::continuous::VRACER

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

std::vector<float> calculateImportanceWeightGradient(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy)

Calculates the gradient of teh importance weight wrt to the parameter of the 2nd (current) distribution evaluated at old action.

Parameters
  • action – The action taken by the agent in the given experience

  • oldPolicy – The policy for the given state used at the time the action was performed

  • curPolicy – The current policy for the given state

Returns

gradient of policy wrt curParamsOne and curParamsTwo

std::vector<float> calculateKLDivergenceGradient(const policy_t &oldPolicy, const policy_t &curPolicy)

Calculates the gradient of KL(p_old, p_cur) wrt to the parameter of the 2nd (current) distribution.

Parameters
  • oldPolicy – The policy for the given state used at the time the action was performed

  • curPolicy – The current policy for the given state

Returns

std::vector<float> generateTrainingAction(policy_t &curPolicy)

Function to generate randomized actions from neural network output.

Parameters

curPolicy – The current policy for the given state

Returns

An action vector

std::vector<float> generateTestingAction(const policy_t &curPolicy)

Function to generate deterministic actions from neural network output required for policy evaluation, respectively testing.

Parameters

curPolicy – The current policy for the given state

Returns

An action vector

virtual float calculateImportanceWeight(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy) override

Calculates importance weight of current action from old and new policies.

Parameters
  • action – The action taken

  • curPolicy – The current policy

  • oldPolicy – The old policy, the one used for take the action in the first place

Returns

The importance weight

virtual void getAction(korali::Sample &sample) override

Gathers the next action either from the policy or randomly.

Parameters

sampleSample on which the action and metadata will be stored

virtual void initializeAgent() override

Initializes the internal state of the policy.

Public Members

std::string _policyDistribution

Specifies which probability distribution to use for the policy.

korali::distribution::univariate::Normal *_normalGenerator

[Internal Use] Gaussian random number generator to generate the agent’s action.

std::vector<float> _actionShifts

[Internal Use] Shifts required for bounded actions.

std::vector<float> _actionScales

[Internal Use] Scales required for bounded actions (half the action domain width).

std::vector<std::string> _policyParameterTransformationMasks

[Internal Use] Stores the transformations required for each parameter.

std::vector<float> _policyParameterScaling

[Internal Use] Stores the scaling required for the parameter after the transformation is applied.

std::vector<float> _policyParameterShifting

[Internal Use] Stores the shifting required for the parameter after the scaling is applied.

problem::reinforcementLearning::Continuous *_problem

Storage for the pointer to the (continuous) learning problem.

class korali::neuralNetwork::layer::Convolution : public korali::neuralNetwork::Layer
#include <convolution.hpp>

Class declaration for module: Convolution.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void copyHyperparameterPointers(Layer *dstLayer) override

Replicates the pointers for the current layer onto a destination layer.

Parameters

dstLayer – The destination layer onto which to copy the pointers

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual std::vector<float> generateInitialHyperparameters() override

Generates the initial weight/bias hyperparameters for the layer.

Returns

The initial hyperparameters

virtual void createHyperparameterMemory() override

Initializes the layer’s internal memory structures for hyperparameter storage.

virtual void createForwardPipeline() override

Initializes the layer’s internal memory structures for the forward pipeline.

virtual void createBackwardPipeline() override

Initializes the internal memory structures for the backward pipeline.

virtual void forwardData(const size_t t) override

Performs the forward propagation of the Wx+b operations.

Parameters

t – Indicates the current timestep

virtual void setHyperparameters(float *hyperparameters) override

Updates layer’s hyperparameters (e.g., weights and biases)

Parameters

hyperparameters – (Input) Pointer to read the hyperparameters from.

virtual void getHyperparameters(float *hyperparameters) override

Gets layer’s hyperparameters (e.g., weights and biases)

Parameters

hyperparameters – (Output) Pointer to write the hyperparameters to.

virtual void getHyperparameterGradients(float *gradient) override

Gets the gradients of the layer’s output wrt to is hyperparameters (e.g., weights and biases)

Parameters

gradient – (Output) Pointer to write the hyperparameter gradients to.

virtual void backwardData(const size_t t) override

Performs the backward propagation of the data.

Parameters

t – Indicates the current timestep

virtual void backwardHyperparameters(const size_t t) override

Calculates the gradients of layer hyperparameters.

Parameters

t – Indicates the current timestep

Public Members

ssize_t _imageHeight

Height of the incoming 2D image.

ssize_t _imageWidth

Width of the incoming 2D image.

ssize_t _kernelHeight

Height of the incoming 2D image.

ssize_t _kernelWidth

Width of the incoming 2D image.

ssize_t _verticalStride

Strides for the image on the vertical dimension.

ssize_t _horizontalStride

Strides for the image on the horizontal dimension.

ssize_t _paddingLeft

Paddings for the image left side.

ssize_t _paddingRight

Paddings for the image right side.

ssize_t _paddingTop

Paddings for the image top side.

ssize_t _paddingBottom

Paddings for the image Bottom side.

ssize_t N

Pre-calculated value for Mini-Batch Size.

ssize_t IC

Pre-calculated value for Input Channels.

ssize_t IH

Pre-calculated value for Input Image Height.

ssize_t IW

Pre-calculated value for Input Image Width.

ssize_t OC

Pre-calculated value for Output Channels.

ssize_t OH

Pre-calculated value for Output Image Height.

ssize_t OW

Pre-calculated value for Output Image Width.

ssize_t KH

Pre-calculated value for Kernel Image Height.

ssize_t KW

Pre-calculated value for Kernel Image Width.

ssize_t PL

Pre-calculated values for padding left.

ssize_t PR

Pre-calculated values for padding right.

ssize_t PT

Pre-calculated values for padding top.

ssize_t PB

Pre-calculated values for padding bottom.

ssize_t SH

Pre-calculated values for horizontal stride.

ssize_t SV

Pre-calculated values for vertical stride.

class korali::problem::bayesian::Custom : public korali::problem::Bayesian
#include <custom.hpp>

Class declaration for module: Custom.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void evaluateLoglikelihood(korali::Sample &sample) override

Evaluates the log likelihood of the given sample, and stores it in sample[“Log Likelihood”].

Parameters

sample – A Korali Sample

virtual void evaluateLoglikelihoodGradient(korali::Sample &sample) override

Evaluates the gradient of the logLikelikood w.r.t. to the variables, and stores it in sample[“logLikelihood Gradient”].

Parameters

sample – A Korali Sample

virtual void evaluateFisherInformation(korali::Sample &sample) override

Evaluates the empirical Fisher information.

Parameters

sample – A Korali Sample

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

Public Members

std::uint64_t _likelihoodModel

Stores the user-defined likelihood model. It should return the value of the Log Likelihood of the given sample.

class korali::solver::optimizer::DEA : public korali::solver::Optimizer
#include <DEA.hpp>

Class declaration for module: DEA.

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void setInitialConfiguration() override

Configures Differential Evolution/.

virtual void runGeneration() override

Executes sampling & evaluation generation.

virtual void printGenerationBefore() override

Console Output before generation runs.

virtual void printGenerationAfter() override

Console output after generation.

virtual void finalize() override

Final console output at termination.

Public Members

size_t _populationSize

Specifies the number of samples to evaluate per generation (preferably 5-10x the number of variables).

double _crossoverRate

Controls the rate at which dimensions of the samples are mixed (must be in [0,1]).

double _mutationRate

Controls the scaling of the vector differentials (must be in [0,2], preferably < 1).

std::string _mutationRule

Controls the Mutation Rate.

std::string _parentSelectionRule

Defines the selection rule of the parent vector.

std::string _acceptRule

Sets the accept rule after sample mutation and evaluation.

int _fixInfeasible

If set true, Korali samples a random sample between Parent and the voiolated boundary. If set false, infeasible samples are mutated again until feasible.

korali::distribution::univariate::Normal *_normalGenerator

[Internal Use] Normal random number generator.

korali::distribution::univariate::Uniform *_uniformGenerator

[Internal Use] Uniform random number generator.

std::vector<double> _valueVector

[Internal Use] Objective Function Values.

std::vector<double> _previousValueVector

[Internal Use] Objective Function Values from previous evaluations.

std::vector<std::vector<double>> _samplePopulation

[Internal Use] Sample variable information.

std::vector<std::vector<double>> _candidatePopulation

[Internal Use] Sample candidates variable information.

size_t _bestSampleIndex

[Internal Use] Index of the best sample in current generation.

double _previousBestEverValue

[Internal Use] Best ever model evaluation as of previous generation.

std::vector<double> _currentMean

[Internal Use] Current mean of population.

std::vector<double> _previousMean

[Internal Use] Previous mean of population.

std::vector<double> _currentBestVariables

[Internal Use] Best variables of current generation.

std::vector<double> _maxDistances

[Internal Use] Max distance between samples per dimension.

size_t _infeasibleSampleCount

[Internal Use] Keeps count of infeasible samples so far.

double _currentMinimumStepSize

[Internal Use] Minimum step size of any variable in the current generation.

size_t _maxInfeasibleResamplings

[Termination Criteria] Max number of mutations per sample per generation if infeasible (only relevant if Fix Infeasible is set False).

double _minValue

[Termination Criteria] Specifies the target fitness to stop minimization.

double _minStepSize

[Termination Criteria] Specifies the minimal step size of the population mean from one gneration to another.

Private Functions

void mutateSingle(size_t sampleIdx)

Mutate a sample.

Parameters

sampleIdx – Index of sample to be mutated.

void fixInfeasible(size_t sampleIdx)

Fix sample params that are outside of domain.

Parameters

sampleIdx – Index of sample that is outside of domain.

void updateSolver(std::vector<Sample> &samples)

Update the state of Differential Evolution.

Parameters

samplesSample evaluations.

void initSamples()

Create new set of candidates.

void prepareGeneration()

Mutate samples and distribute them.

class korali::neuralNetwork::layer::Deconvolution : public korali::neuralNetwork::Layer
#include <deconvolution.hpp>

Class declaration for module: Deconvolution.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void copyHyperparameterPointers(Layer *dstLayer) override

Replicates the pointers for the current layer onto a destination layer.

Parameters

dstLayer – The destination layer onto which to copy the pointers

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual std::vector<float> generateInitialHyperparameters() override

Generates the initial weight/bias hyperparameters for the layer.

Returns

The initial hyperparameters

virtual void createHyperparameterMemory() override

Initializes the layer’s internal memory structures for hyperparameter storage.

virtual void createForwardPipeline() override

Initializes the layer’s internal memory structures for the forward pipeline.

virtual void createBackwardPipeline() override

Initializes the internal memory structures for the backward pipeline.

virtual void forwardData(const size_t t) override

Performs the forward propagation of the Wx+b operations.

Parameters

t – Indicates the current timestep

virtual void setHyperparameters(float *hyperparameters) override

Updates layer’s hyperparameters (e.g., weights and biases)

Parameters

hyperparameters – (Input) Pointer to read the hyperparameters from.

virtual void getHyperparameters(float *hyperparameters) override

Gets layer’s hyperparameters (e.g., weights and biases)

Parameters

hyperparameters – (Output) Pointer to write the hyperparameters to.

virtual void getHyperparameterGradients(float *gradient) override

Gets the gradients of the layer’s output wrt to is hyperparameters (e.g., weights and biases)

Parameters

gradient – (Output) Pointer to write the hyperparameter gradients to.

virtual void backwardData(const size_t t) override

Performs the backward propagation of the data.

Parameters

t – Indicates the current timestep

virtual void backwardHyperparameters(const size_t t) override

Calculates the gradients of layer hyperparameters.

Parameters

t – Indicates the current timestep

Public Members

ssize_t _imageHeight

Height of the incoming 2D image.

ssize_t _imageWidth

Width of the incoming 2D image.

ssize_t _kernelHeight

Height of the incoming 2D image.

ssize_t _kernelWidth

Width of the incoming 2D image.

ssize_t _verticalStride

Strides for the image on the vertical dimension.

ssize_t _horizontalStride

Strides for the image on the horizontal dimension.

ssize_t _paddingLeft

Paddings for the image left side.

ssize_t _paddingRight

Paddings for the image right side.

ssize_t _paddingTop

Paddings for the image top side.

ssize_t _paddingBottom

Paddings for the image Bottom side.

ssize_t N

Pre-calculated value for Mini-Batch Size.

ssize_t IC

Pre-calculated value for Input Channels.

ssize_t IH

Pre-calculated value for Input Image Height.

ssize_t IW

Pre-calculated value for Input Image Width.

ssize_t OC

Pre-calculated value for Output Channels.

ssize_t OH

Pre-calculated value for Output Image Height.

ssize_t OW

Pre-calculated value for Output Image Width.

ssize_t KH

Pre-calculated value for Kernel Image Height.

ssize_t KW

Pre-calculated value for Kernel Image Width.

ssize_t PL

Pre-calculated values for padding left.

ssize_t PR

Pre-calculated values for padding right.

ssize_t PT

Pre-calculated values for padding top.

ssize_t PB

Pre-calculated values for padding bottom.

ssize_t SH

Pre-calculated values for horizontal stride.

ssize_t SV

Pre-calculated values for vertical stride.

class korali::solver::learner::DeepSupervisor : public korali::solver::Learner
#include <deepSupervisor.hpp>

Class declaration for module: DeepSupervisor.

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual std::vector<std::vector<float>> &getEvaluation(const std::vector<std::vector<std::vector<float>>> &input) override

For learner modules which have been trained, test returns an inferred output batch, from a batch of inputs to process.

Parameters

input – The inputs from which to infer outputs. Format: BxTxIC (B: Batch Size, T: Time steps, IC: Input channels)

Returns

The inferred batch outputs for the last given timestep. Format: BxOC (B: Batch Size, OC: Output channels)

virtual std::vector<float> getHyperparameters() override

Returns the hyperparameters required to continue training in the future.

Returns

The hyperparameters

virtual void setHyperparameters(const std::vector<float> &hyperparameters) override

Sets the hyperparameters required to continue training from a previous state.

Parameters

hyperparameters – The hyperparameters to use

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void runGeneration() override

Runs the current generation.

virtual void printGenerationAfter() override

Prints solver information after the execution of the current generation.

Public Members

knlohmann::json _neuralNetworkHiddenLayers

Sets the configuration of the hidden layers for the neural network.

knlohmann::json _neuralNetworkOutputActivation

Allows setting an aditional activation for the output layer.

knlohmann::json _neuralNetworkOutputLayer

Sets any additional configuration (e.g., masks) for the output NN layer.

std::string _neuralNetworkEngine

Specifies which Neural Network backend engine to use.

std::string _neuralNetworkOptimizer

Determines which optimizer algorithm to use to apply the gradients on the neural network’s hyperparameters.

std::vector<float> _hyperparameters

Stores the training neural network hyperparameters (weights and biases).

std::string _lossFunction

Function to calculate the difference (loss) between the NN inference and the exact solution and its gradients for optimization.

size_t _stepsPerGeneration

Represents the number of opitmization steps to run per each generation.

float _learningRate

Learning rate for the underlying ADAM optimizer.

int _l2RegularizationEnabled

Regulates if l2 regularization will be applied to the neural network.

int _l2RegularizationImportance

Importance weight of l2 regularization.

float _outputWeightsScaling

Specified by how much will the weights of the last linear transformation of the NN be scaled. A value of < 1.0 is useful for a more deterministic start.

float _currentLoss

[Internal Use] Current value of the loss function.

std::vector<float> _normalizationMeans

[Internal Use] Stores the current neural network normalization mean parameters.

std::vector<float> _normalizationVariances

[Internal Use] Stores the current neural network normalization variance parameters.

float _targetLoss

[Termination Criteria] Specifies the maximum number of suboptimal generations.

problem::SupervisedLearning *_problem

Korali Problem for optimizing NN weights and biases.

korali::Experiment _optExperiment

Korali Experiment for optimizing the NN’s weights and biases.

korali::fGradientBasedOptimizer *_optimizer

Gradient-based solver pointer to access directly (for performance)

NeuralNetwork *_neuralNetwork

A neural network to be trained based on inputs and solutions.

class korali::solver::agent::Discrete : public korali::solver::Agent
#include <discrete.hpp>

Class declaration for module: Discrete.

Subclassed by korali::solver::agent::discrete::dVRACER

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual float calculateImportanceWeight(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy) override

Calculates importance weight of current action from old and new policies.

Parameters
  • action – The action taken

  • curPolicy – The current policy

  • oldPolicy – The old policy, the one used for take the action in the first place

Returns

The importance weight

std::vector<float> calculateImportanceWeightGradient(const policy_t &curPolicy, const policy_t &oldPolicy)

Calculates the gradient of importance weight wrt to NN output.

Parameters
  • curPolicy – current policy object

  • oldPolicy – old policy object from RM

Returns

gradient of importance weight wrt NN output (q_i’s and inverse temperature)

std::vector<float> calculateKLDivergenceGradient(const policy_t &oldPolicy, const policy_t &curPolicy)

Calculates the gradient of KL(p_old, p_cur) wrt to the NN output.

Parameters
  • oldPolicy – current policy object

  • curPolicy – old policy object from RM

Returns

gradient of KL wrt curent distribution parameter (q_i’s and inverse temperature)

virtual void getAction(korali::Sample &sample) override

Gathers the next action either from the policy or randomly.

Parameters

sampleSample on which the action and metadata will be stored

virtual void initializeAgent() override

Initializes the internal state of the policy.

Public Members

problem::reinforcementLearning::Discrete *_problem

Storage for the pointer to the (discrete) learning problem.

class korali::problem::reinforcementLearning::Discrete : public korali::problem::ReinforcementLearning
#include <discrete.hpp>

Class declaration for module: Discrete.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool runOperation(std::string operation, korali::Sample &sample) override

Runs the operation specified on the given sample. It checks recursively whether the function was found by the current module or its parents.

Parameters
  • sampleSample to operate on. Should contain in the ‘Operation’ field an operation accepted by this module or its parents.

  • operation – Should specify an operation type accepted by this module or its parents.

Returns

True, if operation found and executed; false, otherwise.

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

Public Members

std::vector<std::vector<float>> _possibleActions

The set of all possible actions.

class korali::conduit::Distributed : public korali::Conduit
#include <distributed.hpp>

Class declaration for module: Distributed.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void checkRankCount()

Checks whether the number of MPI workers satisfies the requirement.

virtual void initServer() override

Initializes the worker/server bifurcation in the conduit.

virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void terminateServer() override

Finalizes the workers.

virtual void stackEngine(Engine *engine) override

Stacks a new Engine into the engine stack.

Parameters

engine – A Korali Engine

virtual void popEngine() override

Pops the current Engine from the engine stack.

virtual void listenWorkers() override

(Engine <- Worker) Receives all pending incoming messages and stores them into the corresponding sample’s message queue.

virtual void broadcastMessageToWorkers(knlohmann::json &message) override

(Engine -> Worker) Broadcasts a message to all workers

Parameters

message – JSON object with information to broadcast

virtual void sendMessageToEngine(knlohmann::json &message) override

(Sample -> Engine) Sends an update to the engine to provide partial information while the sample is still active

Parameters

message – Message to send to engine

virtual knlohmann::json recvMessageFromEngine() override

(Sample <- Engine) Blocking call that waits until any message incoming from the engine.

Returns

message from the engine.

virtual void sendMessageToSample(Sample &sample, knlohmann::json &message) override

(Engine -> Sample) Sends an update to a still active sample

Parameters
  • sample – The sample from which to receive an update

  • message – Message to send to the sample.

virtual size_t getProcessId() override

Returns the identifier corresponding to the executing process (to differentiate their random seeds)

Returns

The executing process id

int getRootRank()

Determines which rank is the root.

Returns

The rank id of the root rank.

virtual bool isRoot() override

Determines whether the caller rank/thread/process is root.

Returns

True, if it is root; false, otherwise.

virtual bool isWorkerLeadRank() override

Determines whether the caller rank is the leader of its worker root.

Returns

True, if it is the worker leader rank; false, otherwise.

Public Members

int _ranksPerWorker

Specifies the number of MPI ranks per Korali worker (k).

int _rankId

ID of the current rank.

int _rankCount

Total number of ranks in execution.

int _workerCount

Number of Korali Teams in execution.

int _workerIdSet

Signals whether the worker has been assigned a team.

int _localRankId

Local ID the rank within its Korali Worker.

std::vector<std::vector<int>> _workerTeams

Storage that contains the rank teams for each worker.

std::vector<int> _rankToWorkerMap

Map that indicates to which worker does the current rank correspond to.

class korali::Distribution : public korali::Module
#include <distribution.hpp>

Class declaration for module: Distribution.

Subclassed by korali::distribution::Multivariate, korali::distribution::Specific, korali::distribution::Univariate

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

gsl_rng *setRange(const std::string rangeString)

Creates and sets the RNG range (state and seed) of the random distribution.

Parameters

rangeString – The range to load, in string of hexadecimal values form

Returns

Pointer to the new range.

std::string getRange(gsl_rng *range) const

Gets a hexadecimal string from a given range’s state and seed.

Parameters

range – Range to read from

Returns

Hexadecimal string produced.

inline virtual void updateDistribution()

Updates the parameters of the distribution based on conditional variables.

inline virtual double *getPropertyPointer(const std::string &property)

Gets the pointer to a distribution property.

Parameters

property – The name of the property to update

Returns

Pointer to the property

Public Members

std::string _name

Defines the name of the distribution.

size_t _randomSeed

Defines the random seed of the distribution.

gsl_rng *_range

Stores the current state of the distribution in hexadecimal notation.

std::map<std::string, double*> _conditionalsMap

Map to store the link between parameter names and their pointers.

double _aux

Auxiliar variable to hold pre-calculated data to avoid re-processing information.

bool _hasConditionalVariables

Indicates whether or not this distribution contains conditional variables.

class korali::solver::agent::discrete::dVRACER : public korali::solver::agent::Discrete
#include <dVRACER.hpp>

Class declaration for module: dVRACER.

Public Functions

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void updateVtbc(size_t expId)

Update the V-target or current and previous experiences in the episode.

Parameters

expId – Current Experience Id

void calculatePolicyGradients(const std::vector<size_t> &miniBatch)

Calculates the gradients for the policy/critic neural network.

Parameters

miniBatch – The indexes of the experience mini batch

virtual std::vector<policy_t> runPolicy(const std::vector<std::vector<std::vector<float>>> &stateBatch) override

Function to pass a state time series through the NN and calculates the action probabilities, along with any additional information.

Parameters

stateBatch – The batch of state time series (Format: BxTxS, B is batch size, T is the time series lenght, and S is the state size)

Returns

A JSON object containing the information produced by the policies given the current state series

virtual knlohmann::json getAgentPolicy() override

Obtains the policy hyperaparamters from the learner for the agent to generate new actions.

Returns

The current policy hyperparameters

virtual void setAgentPolicy(const knlohmann::json &hyperparameters) override

Updates the agent’s hyperparameters.

Parameters

hyperparameters – The hyperparameters to update the agent.

virtual void trainPolicy() override

Trains the Agent’s policy, based on the new experiences.

virtual void printAgentInformation() override

Prints information about the training policy.

virtual void initializeAgent() override

Initializes the internal state of the policy.

Public Members

std::vector<float> _statisticsAverageActionSigmas

[Internal Use] Standard deviation of the actions in the minibatch.

learner::DeepSupervisor *_criticPolicyLearner

Pointer to training the actor network.

korali::Experiment _criticPolicyExperiment

Korali experiment for obtaining the agent’s action.

problem::SupervisedLearning *_criticPolicyProblem

Pointer to actor’s experiment problem.

struct korali::solver::sampler::ellipse_t
#include <Nested.hpp>

Ellipse object to generate bounds.

Public Functions

ellipse_t() = delete

Default c-tor (avoid empty initialization).

inline ellipse_t(size_t dim)

Init d-dimensional ellipse without covariance.

Parameters

dim – Dimension of ellipsoid.

void initSphere()

Init d-dimensional unit sphere.

void scaleVolume(double factor)

Scale volume.

Parameters

factor – Volume multiplicator.

Public Members

size_t dim

Dimension of ellipsoid.

size_t num

Number samples in ellipse.

double det

Determinant of covariance.

std::vector<size_t> sampleIdx

Indices of samples from live data set.

std::vector<double> mean

Mean vector of samples in ellipse.

std::vector<double> cov

Covariance Matrix of samples in ellipse.

std::vector<double> invCov

Inverse of Covariance Matrix.

std::vector<double> axes

Axes of the ellipse.

std::vector<double> evals

Eigenvalues of the ellipse.

std::vector<double> paxes

Principal axes of the ellipse.

double volume

Volume estimated from covariance.

double pointVolume

‘True’ volume from which the subset of samples were sampled from.

class korali::Engine
#include <engine.hpp>

A Korali Engine initializes the conduit and experiments, and guides their execution.

Public Functions

Engine()
void saveProfilingInfo(const bool forceSave = false)

Saves the profiling information to the specified path.

Parameters

forceSave – Saves even if the current generation does not divide _profilingFrequency. Reserved for last generation.

void initialize()

Initialization stage of the Korali Engine.

void run(std::vector<Experiment> &experiments)

Stores a set experiments into the experiment list and runs them to completion.

Parameters

experiments – Set of experiments.

void run(Experiment &experiment)

Stores a single experiment into the experiment list and runs it to completion.

Parameters

experiment – The experiment to run.

void start()

Runs the stored list of experiments.

knlohmann::json &operator[](const std::string &key)

C++ wrapper for the getItem operator.

Parameters

key – A C++ string acting as JSON key.

Returns

The referenced JSON object content.

knlohmann::json &operator[](const unsigned long int &key)

C++ wrapper for the getItem operator.

Parameters

key – A C++ integer acting as JSON key.

Returns

The referenced JSON object content.

pybind11::object getItem(const pybind11::object key)

Gets an item from the JSON object at the current pointer position.

Parameters

key – A pybind11 object acting as JSON key (number or string).

Returns

A pybind11 object

void setItem(const pybind11::object key, const pybind11::object val)

Sets an item on the JSON object at the current pointer position.

Parameters
  • key – A pybind11 object acting as JSON key (number or string).

  • val – The value of the item to set.

void serialize(knlohmann::json &js)

Serializes Engine’s data into a JSON object.

Parameters

js – Json object onto which to store the Engine data.

Public Members

Conduit *_conduit

A pointer to the execution conduit. Shared among all experiments in the engine.

std::string _verbosityLevel

Verbosity level of the Engine (‘Silent’, ‘Minimal’ (default), ‘Normal’ or ‘Detailed’).

std::vector<Experiment*> _experimentVector

Stores the list of experiments to run.

cothread_t _thread

Stores the main execution thread (coroutine).

std::string _profilingPath

Saves the output path for the profiling information file.

std::string _profilingDetail

Specifies how much detail will be saved in the profiling file (None, Full)

double _profilingFrequency

Specifies every how many generation will the profiling file be updated.

std::chrono::time_point<std::chrono::high_resolution_clock> _profilingLastSave

Stores the timepoint of the last time the profiling information was saved.

KoraliJson _js

Stores the JSON based configuration for the engine.

bool _isDryRun

Determines whether this is a dry run (no conduit initialization nor execution)

Experiment *_currentExperiment

(Worker) Stores a pointer to the current Experiment being processed

Public Static Functions

static Engine *deserialize(const knlohmann::json &js)

Deserializes JSON object and returns a Korali Engine.

Parameters

js – Json object onto which to store the Engine data.

Returns

The Korali Engine

class korali::solver::Executor : public korali::Solver
#include <executor.hpp>

Class declaration for module: Executor.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void runGeneration() override

Generate a sample and evaluate it.

virtual void printGenerationBefore() override

Console Output before generation runs.

virtual void printGenerationAfter() override

Console output after generation.

Public Members

size_t _executionsPerGeneration

Specifies the number of model executions per generation. By default this setting is 0, meaning that all executions will be performed in the first generation. For values greater 0, executions will be split into batches and split int generations for intermediate output.

size_t _sampleCount

[Internal Use] Number of samples to execute.

class korali::Experiment : public korali::Module
#include <experiment.hpp>

Class declaration for module: Experiment.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

Experiment()
virtual void initialize() override

Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void finalize() override

Finalizes Module. Deallocates memory and produces outputs.

pybind11::object getItem(const pybind11::object key)

Gets an item from the JSON object at the current pointer position.

Parameters

key – A pybind11 object acting as JSON key (number or string).

Returns

A pybind11 object

void setItem(const pybind11::object key, const pybind11::object val)

Sets an item on the JSON object at the current pointer position.

Parameters
  • key – A pybind11 object acting as JSON key (number or string).

  • val – The value of the item to set.

bool loadState(const std::string &path)

Load the state of an experiment from a Korali result file.

Parameters

path – Path from which to load the experiment state.

Returns

true, if file was found; false, otherwise

void saveState()

Saves the state into the experiment’s result path.

void run()

Start the execution of the current experiment.

knlohmann::json &operator[](const std::string &key)

C++ wrapper for the getItem operator.

Parameters

key – A C++ string acting as JSON key.

Returns

The referenced JSON object content.

std::vector<std::vector<float>> getEvaluation(const std::vector<std::vector<std::vector<float>>> &inputBatch)

For learner modules which have been trained, test returns an inferred output batch, from a batch of inputs to process.

Parameters

inputBatch – The inputs from which to infer outputs. Format: TxBxIC (T: Time steps, B: Batch Size, IC: Input channels)

Returns

The inferred outputs. Format: BxOC (Time steps, B: Batch Size, OC: Output channels)

void setSeed(knlohmann::json &js)

Initializes seed to a random value based on current time if not set by the user (i.e. Random Seed is 0).

Parameters

js – Json object onto which to store the Experiment data.

Public Members

size_t _randomSeed

Specifies the initializing seed for the generation of random numbers. If 0 is specified, Korali will automatically derivate a new seed base on the current time.

int _preserveRandomNumberGeneratorStates

Indicates that the engine must preserve the state of their RNGs for reproducibility purposes.

std::vector<korali::distribution::Univariate*> _distributions

Represents the distributions to use during execution.

std::vector<korali::Variable*> _variables

Sample coordinate information.

korali::Problem *_problem

Represents the configuration of the problem to solve.

korali::Solver *_solver

Represents the state and configuration of the solver algorithm.

std::string _fileOutputPath

Specifies the path of the results directory.

int _fileOutputUseMultipleFiles

If true, Korali stores a different generation file per generation with incremental numbering. If disabled, Korali stores the latest generation files into a single file, overwriting previous results.

int _fileOutputEnabled

Specifies whether the partial results should be saved to the results directory.

size_t _fileOutputFrequency

Specifies how often (in generations) will partial result files be saved on the results directory. The default, 1, indicates that every generation’s results will be saved. 0 indicates that only the latest is saved.

int _storeSampleInformation

Specifies whether the sample information should be saved to samples.json in the results path.

std::string _consoleOutputVerbosity

Specifies how much information will be displayed on console when running Korali.

size_t _consoleOutputFrequency

Specifies how often (in generations) will partial results be printed on console. The default, 1, indicates that every generation’s results will be printed.

size_t _currentGeneration

[Internal Use] Indicates the current generation in execution.

int _isFinished

[Internal Use] Indicates whether execution has reached a termination criterion.

size_t _runID

[Internal Use] Specifies the Korali run’s unique identifier. Used to distinguish run results when two or more use the same output directory.

std::string _timestamp

[Internal Use] Indicates the current time when saving a result file.

KoraliJson _js

JSON object to store the experiment’s configuration.

Logger *_logger

A pointer to the Experiment’s logger object.

Engine *_engine

A pointer to the parent engine.

KoraliJson _sampleInfo

JSON object to details of all the samples that have been executed, if requested by the user.

size_t _experimentId

Experiment Identifier.

cothread_t _thread

Experiment’s coroutine (thread). It is swapped among other experiments, and sample threads.

bool _isInitialized

Flag to indicate that the experiment has been initialized to prevent it from re-initializing upon resuming.

double _resultSavingTime

[Profiling] Measures the amount of time taken by saving results

bool _overrideEngine = false

For testing purposes, this field establishes whether the engine is the one to run samples (default = false) or a custom function (true)

std::function<void(Sample&)> _overrideFunction

For testing purposes, this field establishes which custom function to use to override the engine on sample execution for testing.

class korali::distribution::univariate::Exponential : public korali::distribution::Univariate
#include <exponential.hpp>

Class declaration for module: Exponential.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override

Obtains the entire current state and configuration of the module.

Parameters

js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override

Sets the entire state and configuration of the module, given a JSON object.

Parameters

js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override

Applies the module’s default configuration upon its creation.

Parameters

js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override

Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual double *getPropertyPointer(const std::string &property) override

Retrieves the pointer of a conditional value of a distribution property.

Parameters

property – Name of the property to find.

Returns

The pointer to the property..

virtual void updateDistribution() override

Updates the parameters of the distribution based on conditional variables.

virtual double getDensity(const double x) const override

Gets the probability density of the distribution at point x.

Parameters

x – point to evaluate P(x)

Returns

Value of the probability density.

virtual double getLogDensity(const double x) const override

Gets the Log probability density of the distribution at point x.

Parameters

x – point to evaluate log(P(x))

Returns

Log of probability density.

virtual double getLogDensityGradient(double x) const override

Gets the Gradient of the log probability density of the distribution wrt. to x.

Parameters

x – point to evaluate grad(log(P(x)))

Returns

Gradient of log of probability density.

virtual double getLogDensityHessian(double x) const override

Gets the second derivative of the log probability density of the distribution wrt. to x.

Parameters

x – point to evaluate H(log(P(x)))

Returns

Hessian of log of probability density.

virtual double getRandomNumber() override

Draws and returns a random number from the distribution.

Returns

Random real number.

Public Members

double _location

[Conditional Variable Value]

std::string _locationConditional

[Conditional Variable Reference]

double _mean

[Conditional Variable Value]

std::string _meanConditional

[Conditional Variable Reference]

class korali::fAdaBelief : public korali::fAdam
#include <fAdaBelief.hpp>

Class declaration for module: AdaBelief.

Public Functions

fAdaBelief(size_t nVars)

Default constructor for the optimizer.

Parameters

nVars – Variable-space dimensionality

virtual void processResult(float evaluation, std::vector<float> &gradient) override

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters
  • evaluation – The value of the objective function at the current set of parameters

  • gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override

Restores the optimizer to the initial state.

Public Members

std::vector<float> _secondCentralMoment

[Internal Use] Old estimate of second moment of Gradient.

class korali::fAdagrad : public korali::fAdam
#include <fAdagrad.hpp>

Class declaration for module: MADGRAD.

Public Functions

fAdagrad(size_t nVars)

Default constructor for the optimizer.

Parameters

nVars – Variable-space dimensionality

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) override

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters
  • evaluation – The value of the objective function at the current set of parameters

  • gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override

Restores the optimizer to the initial state.

virtual void printInfo() override

Prints progress information.

Public Members

std::vector<float> _s

Squared Gradient Component.

class korali::fAdam : public korali::fGradientBasedOptimizer
#include <fAdam.hpp>

Class declaration for module: Adam.

Subclassed by korali::fAdaBelief, korali::fAdagrad

Public Functions

fAdam(size_t nVars)

Default constructor for the optimizer.

Parameters

nVars – Variable-space dimensionality

virtual bool checkTermination() override

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) override

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters
  • evaluation – The value of the objective function at the current set of parameters

  • gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override

Restores the optimizer to the initial state.

virtual void printInfo() override

Prints progress information.

Public Members

float _beta1

Beta for momentum update.

float _beta2

Beta for gradient update.

float _beta1Pow

Running powers of _beta2.

float _beta2Pow

Running powers of _beta2.

float _epsilon

Smoothing Term.

std::vector<float> _firstMoment

[Internal Use] Estimate of first moment of Gradient.

std::vector<float> _secondMoment

[Internal Use] Old estimate of second moment of Gradient.

float _minGradientNorm

[Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

float _maxGradientNorm

[Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

class korali::fCMAES
#include <fCMAES.hpp>

Class declaration for module: fCMAES.

Public Functions

fCMAES(size_t nVars, size_t populationSize = 0, size_t muSize = 0)

Default constructor for the optimizer.

Parameters
  • nVars – Variable-space dimensionality

  • populationSize – How many samples per generation to use

  • muSize – How many sample selections for covariance matrix adaptation

~fCMAES()

Default destructor for the optimizer.

void setSeed(size_t seed)

Defines the random number generator seed.

Parameters

seed – Random seed

bool checkTermination()

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

void prepareGeneration()

Prepares generation for the next set of evaluations.

void sampleSingle(size_t sampleIdx)

Evaluates a single sample.

Parameters

sampleIdx – Index of the sample to evaluate

void adaptC(int hsig)

Adapts the covariance matrix.

Parameters

hsig – Sign

void updateSigma()

Updates scaling factor of covariance matrix.

void updateDistribution(const std::vector<float> &evaluations)

Updates mean and covariance of Gaussian proposal distribution.

Parameters

evaluations – Model evaluations for all proposed samples

void updateEigensystem(std::vector<float> &M)

Updates the system of eigenvalues and eigenvectors.

Parameters

M – Input matrix

void eigen(size_t N, std::vector<float> &C, std::vector<float> &diag, std::vector<float> &Q) const

Function for eigenvalue decomposition.

Parameters
  • N – Matrix size

  • C – Input matrix

  • diag – Means

  • Q – Output Matrix

void sort_index(const std::vector<float> &vec, std::vector<size_t> &_sortingIndex, size_t N) const

Descending sort of vector elements, stores ordering in _sortingIndex.

Parameters
  • _sortingIndex – Ordering of elements in vector

  • vec – Vector to sort

  • N – Number of current samples.

void initMuWeights(size_t numsamples)

Initializes the weights of the mu vector.

Parameters

numsamples – Length of mu vector

void initCovariance()

Initialize Covariance Matrix and Cholesky Decomposition.

void printInfo()

Console output after generation.

void reset()

Restores the optimizer to the initial state.

bool isSampleFeasible(const std::vector<float> &sample)

Checks whether a proposed sample is feasible (all variables within acceptable range)

Parameters

sampleSample to check

Returns

Whether the sample is feasible

Public Members

size_t _nVars

Number of problem variables.

size_t _currentGeneration

Counter for the current generation.

std::vector<float> _initialMeans

Indicates the initial gaussian means for all variables.

std::vector<float> _initialStandardDeviations

Indicates the initial gaussian standard deviations for all variables.

std::vector<float> _lowerBounds

Indicates the lower bounds for all variables.

std::vector<float> _upperBounds

Indicates the upper bounds for all variables.

std::default_random_engine _randomGenerator

Random number generator.

std::normal_distribution<float> _normalGenerator

Gaussian number generator.

size_t _populationSize

Specifies the number of samples to evaluate per generation (default: $4+3*log(N)$, where $N$ is the number of variables).

size_t _muValue

Number of best samples used to update the covariance matrix and the mean (by default it is half the Sample Count).

std::string _muType

Weights given to the Mu best values to update the covariance matrix and the mean.

float _initialSigmaCumulationFactor

Controls the learning rate of the conjugate evolution path (by default this variable is internally calibrated).

float _initialDampFactor

Controls the updates of the covariance matrix scaling factor (by default this variable is internally calibrated).

int _isSigmaBounded

Sets an upper bound for the covariance matrix scaling factor. The upper bound is given by the average of the initial standard deviation of the variables.

float _initialCumulativeCovariance

Controls the learning rate of the evolution path for the covariance update (must be in (0,1], by default this variable is internally calibrated).

int _isDiagonal

Covariance matrix updates will be optimized for diagonal matrices.

std::vector<float> _valueVector

[Internal Use] Objective function values.

std::vector<float> _previousValueVector

[Internal Use] Objective function values from previous generation.

std::vector<float> _muWeights

[Internal Use] Weights for each of the Mu samples.

float _effectiveMu

[Internal Use] Variance effective selection mass.

float _sigmaCumulationFactor

[Internal Use] Increment for sigma, calculated from muEffective and dimension.

float _dampFactor

[Internal Use] Dampening parameter controls step size adaption.

float _cumulativeCovariance

[Internal Use] Controls the step size adaption.

float _chiSquareNumber

[Internal Use] Expectation of $||N(0,I)||^2$.

size_t _covarianceEigenvalueEvaluationFrequency

[Internal Use] Establishes how frequently the eigenvalues are updated.

float _sigma

[Internal Use] Determines the step size.

bool _noUpdatePossible

[Internal Use] This flag remembers if it was not possible to produce an update and terminates

float _trace

[Internal Use] The trace of the initial covariance matrix.

std::vector<std::vector<float>> _samplePopulation

[Internal Use] Sample coordinate information.

size_t _finishedSampleCount

[Internal Use] Counter of evaluated samples to terminate evaluation.

float _currentBestValue

[Internal Use] Best model evaluation from current generation.

std::vector<float> _currentBestVariables

[Internal Use] Best variables of current generation.

std::vector<float> _bestEverVariables

[Internal Use] Best ever found variables.

float _previousBestValue

[Internal Use] Best model evaluation from previous generation.

size_t _bestSampleIndex

[Internal Use] Index of the best sample in current generation.

float _bestEverValue

[Internal Use] Best ever model evaluation.

float _previousBestEverValue

[Internal Use] Best ever model evaluation as of previous generation.

std::vector<size_t> _sortingIndex

[Internal Use] Sorted indeces of samples according to their model evaluation.

std::vector<float> _covarianceMatrix

[Internal Use] (Unscaled) covariance Matrix of proposal distribution.

std::vector<float> _auxiliarCovarianceMatrix

[Internal Use] Temporary Storage for Covariance Matrix.

std::vector<float> _covarianceEigenvectorMatrix

[Internal Use] Matrix with eigenvectors in columns.

std::vector<float> _auxiliarCovarianceEigenvectorMatrix

[Internal Use] Temporary Storage for Matrix with eigenvectors in columns.

std::vector<float> _axisLengths

[Internal Use] Axis lengths (sqrt(Evals))

std::vector<float> _auxiliarAxisLengths

[Internal Use] Temporary storage for Axis lengths.

std::vector<float> _bDZMatrix

[Internal Use] Temporary storage.

std::vector<float> _auxiliarBDZMatrix

[Internal Use] Temporary storage.

std::vector<float> _currentMean

[Internal Use] Current mean of proposal distribution.

std::vector<float> _previousMean

[Internal Use] Previous mean of proposal distribution.

std::vector<float> _meanUpdate

[Internal Use] Update differential from previous to current mean.

std::vector<float> _evolutionPath

[Internal Use] Evolution path for Covariance Matrix update.

std::vector<float> _conjugateEvolutionPath

[Internal Use] Conjugate evolution path for sigma update.

float _conjugateEvolutionPathL2Norm

[Internal Use] L2 Norm of the conjugate evolution path.

size_t _infeasibleSampleCount

[Internal Use] Keeps count of the number of infeasible samples.

float _maximumDiagonalCovarianceMatrixElement

[Internal Use] Maximum diagonal element of the Covariance Matrix.

float _minimumDiagonalCovarianceMatrixElement

[Internal Use] Minimum diagonal element of the Covariance Matrix.

float _maximumCovarianceEigenvalue

[Internal Use] Maximum Covariance Matrix Eigenvalue.

float _minimumCovarianceEigenvalue

[Internal Use] Minimum Covariance Matrix Eigenvalue.

int _isEigensystemUpdated

[Internal Use] Flag determining if the covariance eigensystem is up to date.

float _covarianceMatrixAdaptionFactor

[Internal Use] This is the beta factor that indicates how fast the covariance matrix is adapted.

float _globalSuccessRate

[Internal Use] Estimated Global Success Rate, required for calibration of covariance matrix scaling factor updates.

size_t _covarianceMatrixAdaptationCount

[Internal Use] Number of Covariance Matrix Adaptations.

float _currentMinStandardDeviation

[Internal Use] Current minimum standard deviation of any variable.

float _currentMaxStandardDeviation

[Internal Use] Current maximum standard deviation of any variable.

size_t _maxInfeasibleResamplings

[Termination Criteria] Maximum number of resamplings per candidate per generation if sample is outside of Lower and Upper Bound.

float _maxConditionCovarianceMatrix

[Termination Criteria] Specifies the maximum condition of the covariance matrix.

float _minValue

[Termination Criteria] Specifies the minimum target fitness to stop minimization.

float _maxValue

[Termination Criteria] Specifies the maximum target fitness to stop maximization.

float _minValueDifferenceThreshold

[Termination Criteria] Specifies the minimum fitness differential between two consecutive generations before stopping execution.

float _targetMaxStandardDeviation

[Termination Criteria] Finishes execution when the maximum standard deviation falls below this threshold

std::vector<float> _minMeanUpdates

[Termination Criteria] Specifies the mininum update to the variable means before triggering termination.

size_t _maxGenerations

Stores how many generations to run for.

gsl_vector *_gsl_eval

Internal GSL storage.

gsl_matrix *_gsl_evec

Internal GSL storage.

gsl_eigen_symmv_workspace *_gsl_work

Internal GSL storage.

class korali::fGradientBasedOptimizer

Class declaration for module: GRADIENT_BASED_OPTIMIZER.

Subclassed by korali::fAdam, korali::fMadGrad, korali::fRMSProp

Public Functions

virtual ~fGradientBasedOptimizer() = default

Default destructor to avoid warnings.

virtual bool checkTermination() = 0

Determines whether the module can trigger termination of an experiment run.

Returns

True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) = 0

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters
  • evaluation – The value of the objective function at the current set of parameters

  • gradient – The gradient of the objective function at the current set of parameters

virtual void reset() = 0

Restores the optimizer to the initial state.

virtual void printInfo() = 0

Prints progress information.

Public Members

size_t _nVars

Number of problem variables.

float _eta

Learning Rate.

float _decay

Decay for gradient update.

size_t _currentGeneration

Counter for the current generation.

std::vector<float> _initialValues

Initial values for the variables.

size_t