Code Documentation¶

class korali::neuralNetwork::layer::Activation : public korali::neuralNetwork::Layer ¶

#include <activation.hpp>

Class declaration for module: Activation.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void createForwardPipeline() override¶: Initializes the layer’s internal memory structures for the forward pipeline.

virtual void createBackwardPipeline() override¶: Initializes the internal memory structures for the backward pipeline.

virtual void forwardData(const size_t t) override¶

Performs the forward propagation of the Wx+b operations.

Parameters: t – Indicates the current timestep

virtual void backwardData(const size_t t) override¶

Performs the backward propagation of the data.

Parameters: t – Indicates the current timestep

Public Members

std::string _function¶: Indicates the activation function for the weighted inputs to the current layer.

float _alpha¶: First (alpha) argument to the activation function, as detailed in https://oneapi-src.github.io/oneDNN/dev_guide_eltwise.html.

float _beta¶: Second (beta) argument to the activation function, as detailed in https://oneapi-src.github.io/oneDNN/dev_guide_eltwise.html.

class korali::solver::optimizer::AdaBelief : public korali::solver::Optimizer ¶

#include <AdaBelief.hpp>

Class declaration for module: AdaBelief.

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void processResult(double evaluation, std::vector<double> &gradient)¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void finalize() override¶: Finalizes Module. Deallocates memory and produces outputs.

virtual void setInitialConfiguration() override¶: Initializes the solver with starting values for the first generation.

virtual void runGeneration() override¶: Runs the current generation.

virtual void printGenerationBefore() override¶: Prints solver information before the execution of the current generation.

virtual void printGenerationAfter() override¶: Prints solver information after the execution of the current generation.

Public Members

double _beta1¶: Smoothing factor for momentum update.

double _beta2¶: Smoothing for gradient update.

double _eta¶: Learning Rate (Step Size)

double _epsilon¶: Term to facilitate numerical stability.

std::vector<double> _currentVariable¶: [Internal Use] Current value of parameters.

std::vector<double> _gradient¶: [Internal Use] Gradient of Function with respect to Parameters.

std::vector<double> _bestEverGradient¶: [Internal Use] Gradient of function with respect to Best Ever Variables.

double _gradientNorm¶: [Internal Use] Norm of gradient of function with respect to Parameters.

std::vector<double> _firstMoment¶: [Internal Use] Estimate of first moment of Gradient.

std::vector<double> _biasCorrectedFirstMoment¶: [Internal Use] Bias corrected estimate of first moment of Gradient.

std::vector<double> _secondCentralMoment¶: [Internal Use] Previous estimate of second moment of Gradient.

std::vector<double> _biasCorrectedSecondCentralMoment¶: [Internal Use] Bias corrected estimate of second moment of Gradient.

double _minGradientNorm¶: [Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

double _maxGradientNorm¶: [Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

class korali::solver::optimizer::Adam : public korali::solver::Optimizer ¶

#include <Adam.hpp>

Class declaration for module: Adam.

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void processResult(double evaluation, std::vector<double> &gradient)¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void finalize() override¶: Finalizes Module. Deallocates memory and produces outputs.

virtual void setInitialConfiguration() override¶: Initializes the solver with starting values for the first generation.

virtual void runGeneration() override¶: Runs the current generation.

virtual void printGenerationBefore() override¶: Prints solver information before the execution of the current generation.

virtual void printGenerationAfter() override¶: Prints solver information after the execution of the current generation.

Public Members

double _beta1¶: Smoothing factor for momentum update.

double _beta2¶: Smoothing factor for gradient update.

double _eta¶: Learning Rate (Step Size)

double _epsilon¶: Term to facilitate numerical stability.

std::vector<double> _currentVariable¶: [Internal Use] Current value of parameters.

std::vector<double> _gradient¶: [Internal Use] Gradient of Function with respect to Parameters.

std::vector<double> _bestEverGradient¶: [Internal Use] Gradient of function with respect to Best Ever Variables.

std::vector<double> _squaredGradient¶: [Internal Use] Square of gradient of function with respect to Parameters.

double _gradientNorm¶: [Internal Use] Norm of gradient of function with respect to Parameters.

std::vector<double> _firstMoment¶: [Internal Use] Estimate of first moment of Gradient.

std::vector<double> _biasCorrectedFirstMoment¶: [Internal Use] Bias corrected estimate of first moment of Gradient.

std::vector<double> _secondMoment¶: [Internal Use] Old estimate of second moment of Gradient.

std::vector<double> _biasCorrectedSecondMoment¶: [Internal Use] Bias corrected estimate of second moment of Gradient.

double _minGradientNorm¶: [Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

double _maxGradientNorm¶: [Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

class korali::solver::Agent : public korali::Solver ¶

#include <agent.hpp>

Class declaration for module: Agent.

Subclassed by korali::solver::agent::Continuous, korali::solver::agent::Discrete

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void normalizeStateActionNeuralNetwork(NeuralNetwork *neuralNetwork, size_t miniBatchSize, size_t normalizationSteps)¶

Mini-batch based normalization routine for Neural Networks with state and action inputs (typically critics)

Parameters

neuralNetwork – Neural Network to normalize
miniBatchSize – Number of entries in the normalization minibatch
normalizationSteps – How many normalization steps to perform (and grab the average)

void normalizeStateNeuralNetwork(NeuralNetwork *neuralNetwork, size_t miniBatchSize, size_t normalizationSteps)¶

Mini-batch based normalization routine for Neural Networks with state inputs only (typically policy)

Parameters

neuralNetwork – Neural Network to normalize
miniBatchSize – Number of entries in the normalization minibatch
normalizationSteps – How many normalization steps to perform (and grab the average)

void processEpisode(size_t episodeId, knlohmann::json &episode)¶

Additional post-processing of episode after episode terminated.

Parameters

episodeId – The unique identifier of the provided episode
episode – A vector of experiences pertaining to the episode.

std::vector<size_t> generateMiniBatch(size_t miniBatchSize)¶

Generates an experience mini batch from the replay memory.

Parameters: miniBatchSize – Size of the mini batch to create
Returns: A vector with the indexes to the experiences in the mini batch

void updateExperienceMetadata(const std::vector<size_t> &miniBatch, const std::vector<policy_t> &policyData)¶

Updates the state value, retrace, importance weight and other metadata for a given minibatch of experiences.

Parameters

miniBatch – The mini batch of experience ids to update
policyData – The policy to use to evaluate the experiences

void resetTimeSequence()¶: Resets time sequence within the agent, to forget past actions from other episodes.

virtual std::vector<policy_t> runPolicy(const std::vector<std::vector<std::vector<float>>> &stateBatch) = 0¶

Function to pass a state time series through the NN and calculates the action probabilities, along with any additional information.

Parameters: stateBatch – The batch of state time series (Format: BxTxS, B is batch size, T is the time series lenght, and S is the state size)
Returns: A JSON object containing the information produced by the policies given the current state series

size_t getTimeSequenceStartExpId(size_t expId)¶

Calculates the starting experience index of the time sequence for the selected experience.

Parameters: expId – The index of the latest experience in the sequence
Returns: The starting time sequence index

std::vector<std::vector<std::vector<float>>> getMiniBatchStateSequence(const std::vector<size_t> &miniBatch, const bool includeAction = false)¶

Gets a vector of states corresponding of time sequence corresponding to the provided last experience index.

Parameters

miniBatch – Indexes to the latest experiences in a batch of sequences
includeAction – Specifies whether to include the experience’s action in the sequence

Returns

The time step vector of states

std::vector<std::vector<float>> getTruncatedStateSequence(size_t expId)¶

Gets a vector of states corresponding of time sequence corresponding to the provided second-to-last experience index for which a truncated state exists.

Parameters: expId – The index of the second-to-latest experience in the sequence
Returns: The time step vector of states, including the truncated state

virtual float calculateImportanceWeight(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy) = 0¶

Calculates importance weight of current action from old and new policies.

Parameters

action – The action taken
curPolicy – The current policy
oldPolicy – The old policy, the one used for take the action in the first place

Returns

The importance weight

void attendAgent(const size_t agentId)¶

Listens to incoming experience from the given agent, sends back policy or terminates the episode depending on what’s needed.

Parameters: agentId – The Agent’s ID

void serializeExperienceReplay()¶: Serializes the experience replay into a JSON compatible format.

void deserializeExperienceReplay()¶: Deserializes a JSON object into the experience replay.

void trainingGeneration()¶: Runs a generation when running in training mode.

void testingGeneration()¶: Runs a generation when running in testing mode.

void rescaleStates()¶: Rescales states to have a zero mean and unit variance.

inline float getScaledReward(const float reward)¶

Rescales a given reward according to gaussian normalization parameters (mean+sigma)

Parameters: reward – the input reward to rescale
Returns: The normalized reward

void calculateRewardRescalingFactors()¶: Re-calculates reward scaling factors to have a zero mean and unit variance.

virtual void trainPolicy() = 0¶: Trains the Agent’s policy, based on the new experiences.

virtual knlohmann::json getAgentPolicy() = 0¶

Obtains the policy hyperaparamters from the learner for the agent to generate new actions.

Returns: The current policy hyperparameters

virtual void setAgentPolicy(const knlohmann::json &hyperparameters) = 0¶

Updates the agent’s hyperparameters.

Parameters: hyperparameters – The hyperparameters to update the agent.

virtual void resetAgentOptimizers() = 0¶: Resets the states of the optimizers.

virtual void initializeAgent() = 0¶: Initializes the internal state of the policy.

virtual void printAgentInformation() = 0¶: Prints information about the training policy.

virtual void getAction(korali::Sample &sample) = 0¶

Gathers the next action either from the policy or randomly.

Parameters: sample – Sample on which the action and metadata will be stored

virtual void runGeneration() override¶: Runs the current generation.

virtual void printGenerationAfter() override¶: Prints solver information after the execution of the current generation.

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void finalize() override¶: Finalizes Module. Deallocates memory and produces outputs.

Public Members

std::string _mode¶: Specifies the operation mode for the agent.

std::vector<size_t> _testingSampleIds¶: A vector with the identifiers for the samples to test the hyperparameters with.

knlohmann::json _testingPolicy¶: The hyperparameters of the policy to test.

size_t _trainingAverageDepth¶: Specifies the depth of the running training average to report.

size_t _concurrentEnvironments¶: Indicates the number of concurrent environments to use to collect experiences.

size_t _episodesPerGeneration¶: Indicates how many episodes to complete in a generation (checkpoints are generated between generations).

size_t _miniBatchSize¶: The number of experiences to randomly select to train the neural network(s) with.

std::string _miniBatchStrategy¶: Determines how to select experiences from the replay memory for mini batch creation.

size_t _timeSequenceLength¶: Indicates the number of contiguous experiences to pass to the NN for learning. This is only useful when using recurrent NNs.

float _learningRate¶: The initial learning rate to use for the NN hyperparameter optimization.

int _l2RegularizationEnabled¶: Boolean to determine if l2 regularization will be applied to the neural networks.

float _l2RegularizationImportance¶: Coefficient for l2 regularization.

knlohmann::json _neuralNetworkHiddenLayers¶: Indicates the configuration of the hidden neural network layers.

std::string _neuralNetworkOptimizer¶: Indicates the optimizer algorithm to update the NN hyperparameters.

std::string _neuralNetworkEngine¶: Specifies which Neural Network backend to use.

float _discountFactor¶: Represents the discount factor to weight future experiences.

int _experienceReplaySerialize¶: Indicates whether to serialize and store the experience replay after each generation. Disabling will reduce I/O overheads but will disable the checkpoint/resume function.

size_t _experienceReplayStartSize¶: The minimum number of experiences before learning starts.

size_t _experienceReplayMaximumSize¶: The size of the replay memory. If this number is exceeded, experiences are deleted.

float _experienceReplayOffPolicyCutoffScale¶: Initial Cut-Off to classify experiences as on- or off-policy. (c_max in https://arxiv.org/abs/1807.05827)

float _experienceReplayOffPolicyTarget¶: Target fraction of off-policy experiences in the replay memory. (D in https://arxiv.org/abs/1807.05827)

float _experienceReplayOffPolicyAnnealingRate¶: Annealing rate for Off Policy Cutoff Scale and Learning Rate. (A in https://arxiv.org/abs/1807.05827)

float _experienceReplayOffPolicyREFERBeta¶: Initial value for the penalisation coefficient for off-policiness. (beta in https://arxiv.org/abs/1807.05827)

float _experiencesBetweenPolicyUpdates¶: The number of experiences to receive before training/updating (real number, may be less than < 1.0, for more than one update per experience).

int _stateRescalingEnabled¶: Determines whether to normalize the states, such that they have mean 0 and standard deviation 1 (done only once after the initial exploration phase).

int _rewardRescalingEnabled¶: Determines whether to normalize the rewards, such that they have mean 0 and standard deviation 1.

size_t _rewardRescalingFrequency¶: The number of policy updates between consecutive reward rescalings.

int _rewardOutboundPenalizationEnabled¶: If enabled, it penalizes the rewards for experiences with out of bound actions. This is useful for problems with truncated actions (e.g., openAI gym Mujoco) where out of bounds actions are clipped in the environment. This prevents policy means to extend too much outside the bounds.

float _rewardOutboundPenalizationFactor¶: The factor (f) by which te reward is scaled down. R = f * R.

std::vector<float> _actionLowerBounds¶: [Internal Use] Lower bounds for actions.

std::vector<float> _actionUpperBounds¶: [Internal Use] Upper bounds for actions.

size_t _currentEpisode¶: [Internal Use] Indicates the current episode being processed.

std::vector<float> _trainingRewardHistory¶: [Internal Use] Keeps a history of all training episode rewards.

std::vector<size_t> _trainingExperienceHistory¶: [Internal Use] Keeps a history of all training episode experience counts.

float _trainingAverageReward¶: [Internal Use] Contains a running average of the training episode rewards.

float _trainingLastReward¶: [Internal Use] Remembers the cumulative sum of rewards for the last training episode.

float _trainingBestReward¶: [Internal Use] Remembers the best cumulative sum of rewards found so far in any episodes.

size_t _trainingBestEpisodeId¶: [Internal Use] Remembers the episode that obtained the maximum cumulative sum of rewards found so far.

std::vector<float> _testingReward¶: [Internal Use] The cumulative sum of rewards obtained when evaluating the testing samples.

float _testingBestReward¶: [Internal Use] Remembers the best cumulative sum of rewards from latest testing episodes, if any.

float _testingWorstReward¶: [Internal Use] Remembers the worst cumulative sum of rewards from latest testing episodes, if any.

size_t _testingBestEpisodeId¶: [Internal Use] Remembers the episode Id that obtained the maximum cumulative sum of rewards found so far during testing.

size_t _testingCandidateCount¶: [Internal Use] Remembers the number of candidate policies tested so far.

float _testingAverageReward¶: [Internal Use] Remembers the average cumulative sum of rewards from latest testing episodes, if any.

float _testingStdevReward¶: [Internal Use] Remembers the standard deviation of the cumulative sum of rewards from latest testing episodes, if any.

float _testingPreviousAverageReward¶: [Internal Use] Remembers the average cumulative sum of rewards from previous testing episodes, if any.

float _testingBestAverageReward¶: [Internal Use] Remembers the best cumulative sum of rewards found so far from testing episodes.

size_t _experienceReplayOffPolicyCount¶: [Internal Use] Number of off-policy experiences in the experience replay.

float _experienceReplayOffPolicyRatio¶: [Internal Use] Current off policy ratio in the experience replay.

float _experienceReplayOffPolicyCurrentCutoff¶: [Internal Use] Indicates the current cutoff to classify experiences as on- or off-policy

float _currentLearningRate¶: [Internal Use] The current learning rate to use for the NN hyperparameter optimization.

size_t _policyUpdateCount¶: [Internal Use] Keeps track of the number of policy updates that have been performed.

size_t _currentSampleID¶: [Internal Use] Keeps track of the current Sample ID, and makes sure no two equal sample IDs are produced such that this value can be used as random seed.

korali::distribution::univariate::Uniform *_uniformGenerator¶: [Internal Use] Uniform random number generator.

size_t _experienceCount¶: [Internal Use] Count of the number of experiences produced so far.

float _rewardRescalingMean¶: [Internal Use] Contains the mean of the rewards. They will be shifted by this value in order to normalize the reward distribution in the RM.

float _rewardRescalingSigma¶: [Internal Use] Contains the standard deviation of the rewards. They will be scaled by this value in order to normalize the reward distribution in the RM.

size_t _rewardRescalingCount¶: [Internal Use] Indicates how many times have the rewards been rescaled

size_t _rewardOutboundPenalizationCount¶: [Internal Use] Keeps track of the number of out of bound actions taken.

std::vector<float> _stateRescalingMeans¶: [Internal Use] Contains the mean of the states. They will be shifted by this value in order to normalize the state distribution in the RM.

std::vector<float> _stateRescalingSigmas¶: [Internal Use] Contains the standard deviations of the states. They will be scaled by this value in order to normalize the state distribution in the RM.

size_t _maxEpisodes¶: [Termination Criteria] The solver will stop when the given number of episodes have been run.

size_t _maxExperiences¶: [Termination Criteria] The solver will stop when the given number of experiences have been gathered.

float _testingTargetAverageReward¶: [Termination Criteria] The solver will stop when the given best average per-episode reward has been reached among the experiences between two learner updates.

float _testingAverageRewardIncrement¶: [Termination Criteria] The solver will stop when the average testing reward is below the previous testing average by more than a threshold given by this factor multiplied with the testing standard deviation.

size_t _maxPolicyUpdates¶: [Termination Criteria] The solver will stop when the given number of optimization steps have been performed.

std::vector<Sample> _agents¶: Array of agents collecting new experiences.

std::vector<bool> _isAgentRunning¶: Keeps track of the age.

size_t _sessionExperienceCount¶: Session-specific experience count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionEpisodeCount¶: Session-specific episode count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionGeneration¶: Session-specific generation count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionPolicyUpdateCount¶: Session-specific policy update count. This is useful in case of restart: counters from the old session won’t count.

size_t _sessionExperiencesUntilStartSize¶: Session-specific counter that keeps track of how many experiences need to be obtained this session to reach the start training threshold.

cBuffer<std::vector<float>> _stateVector¶: Stores the state of the experience.

cBuffer<std::vector<float>> _actionVector¶: Stores the action taken by the agent at the given state.

cBuffer<std::vector<float>> _stateTimeSequence¶: Stores the current sequence of states observed by the agent (limited to time sequence length defined by the user)

cBuffer<size_t> _episodeIdVector¶: Episode that experience belongs to.

cBuffer<size_t> _episodePosVector¶: Position within the episode of this experience.

cBuffer<float> _importanceWeightVector¶: Contains the latest calculation of the experience’s importance weight.

cBuffer<float> _truncatedImportanceWeightVector¶: Contains the latest calculation of the experience’s truncated importance weight.

cBuffer<float> _priorityVector¶: For prioritized experience replay, this stores the experience’s priority.

cBuffer<float> _probabilityVector¶: For prioritized experience replay, this stores the experience’s probability.

cBuffer<policy_t> _curPolicyVector¶: Contains the most current policy information given the experience state.

cBuffer<policy_t> _expPolicyVector¶: Contains the policy information produced at the moment of the action was taken.

cBuffer<bool> _isOnPolicyVector¶: Indicates whether the experience is on policy, given the specified off-policiness criteria.

cBuffer<termination_t> _terminationVector¶: Specifies whether the experience is terminal (truncated or normal) or not.

cBuffer<float> _retraceValueVector¶: Contains the result of the retrace (Vtbc) function for the currrent experience.

cBuffer<float> _truncatedStateValueVector¶: If this is a truncated terminal experience, this contains the state value for that state.

cBuffer<std::vector<float>> _truncatedStateVector¶: If this is a truncated terminal experience, the truncated state is also saved here.

cBuffer<float> _rewardVector¶: Contains the rewards for every experience.

cBuffer<float> _stateValueVector¶: Contains the state value evaluation for every experience.

float _priorityAnnealingRate¶: Stores the priority annealing rate.

float _importanceWeightAnnealingRate¶: Stores the importance weight annealing factor.

knlohmann::json _trainingCurrentPolicy¶: Stores the current policy configuration.

knlohmann::json _trainingBestPolicy¶: Stores the training policy configuration that has produced the best results.

knlohmann::json _testingBestPolicy¶: Stores the candidate policy configuration that has produced the best results.

problem::ReinforcementLearning *_problem¶: Storage for the pointer to the learning problem.

std::random_device rd¶: Random device for the generation of shuffling numbers.

std::mt19937 *mt¶: Mersenne twister for the generation of shuffling numbers.

double _sessionRunningTime¶: [Profiling] Measures the amount of time taken by the generation

double _sessionSerializationTime¶: [Profiling] Measures the amount of time taken by ER serialization

double _sessionAgentComputationTime¶: [Profiling] Stores the computation time per episode taken by Agents

double _sessionAgentCommunicationTime¶: [Profiling] Measures the average communication time per episode taken by Agents

double _sessionAgentPolicyEvaluationTime¶: [Profiling] Measures the average policy evaluation time per episode taken by Agents

double _sessionPolicyUpdateTime¶: [Profiling] Measures the time taken to update the policy in the current generation

double _sessionAgentAttendingTime¶: [Profiling] Measures the time taken to update the attend the agent’s state

double _generationRunningTime¶: [Profiling] Measures the amount of time taken by the generation

double _generationSerializationTime¶: [Profiling] Measures the amount of time taken by ER serialization

double _generationAgentComputationTime¶: [Profiling] Stores the computation time per episode taken by Agents

double _generationAgentCommunicationTime¶: [Profiling] Measures the average communication time per episode taken by Agents

double _generationAgentPolicyEvaluationTime¶: [Profiling] Measures the average policy evaluation time per episode taken by Agents

double _generationPolicyUpdateTime¶: [Profiling] Measures the time taken to update the policy in the current generation

double _generationAgentAttendingTime¶: [Profiling] Measures the time taken to update the attend the agent’s state

class korali::problem::Bayesian : public korali::Problem ¶

#include <bayesian.hpp>

Class declaration for module: Bayesian.

Subclassed by korali::problem::bayesian::Custom, korali::problem::bayesian::Reference

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool runOperation(std::string operation, korali::Sample &sample) override¶

Runs the operation specified on the given sample. It checks recursively whether the function was found by the current module or its parents.

Parameters

sample – Sample to operate on. Should contain in the ‘Operation’ field an operation accepted by this module or its parents.
operation – Should specify an operation type accepted by this module or its parents.

Returns

True, if operation found and executed; false, otherwise.

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void evaluate(korali::Sample &sample)¶

Produces a generic evaluation from the Posterior distribution of the sample, for optimization with CMAES, DEA, storing it in and stores it in sample[“F(x)”].

Parameters: sample – A Korali Sample

void evaluateLogPrior(korali::Sample &sample)¶

Evaluates the log prior of the given sample, and stores it in sample[“Log Prior”].

Parameters: sample – A Korali Sample

virtual void evaluateLoglikelihood(korali::Sample &sample) = 0¶

Evaluates the log likelihood of the given sample, and stores it in sample[“Log Likelihood”].

Parameters: sample – A Korali Sample

void evaluateLogPosterior(korali::Sample &sample)¶

Evaluates the log posterior of the given sample, and stores it in sample[“Log Posterior”].

Parameters: sample – A Korali Sample

virtual void evaluateGradient(korali::Sample &sample)¶

Evaluates the gradient of the objective w.r.t. to the variables, and stores it in sample[“Gradient”].

Parameters: sample – A Korali Sample

void evaluateLogPriorGradient(korali::Sample &sample)¶

Evaluates the gradient of the logPrior w.r.t. to the variables, and stores it in sample[“logPrior Gradient”].

Parameters: sample – A Korali Sample

inline virtual void evaluateLoglikelihoodGradient(korali::Sample &sample)¶

Evaluates the gradient of the logLikelikood w.r.t. to the variables, and stores it in sample[“logLikelihood Gradient”].

Parameters: sample – A Korali Sample

virtual void evaluateHessian(korali::Sample &sample)¶

Evaluates the hessian of the objective w.r.t. to the variables, and stores it in sample[“Hessian”].

Parameters: sample – A Korali Sample

void evaluateLogPriorHessian(korali::Sample &sample)¶

Evaluates the gradient of the logPrior w.r.t. to the variables, and stores it in sample[“logPrior Hessian”].

Parameters: sample – A Korali Sample

inline virtual void evaluateLogLikelihoodHessian(korali::Sample &sample)¶

Evaluates the gradient of the logLikelikood w.r.t. to the variables, and stores it in sample[“logLikelihood Hessian”].

Parameters: sample – A Korali Sample

inline virtual void evaluateFisherInformation(korali::Sample &sample)¶

Evaluates the empirical Fisher information.

Parameters: sample – A Korali Sample

class korali::distribution::univariate::Beta : public korali::distribution::Univariate ¶

#include <beta.hpp>

Class declaration for module: Beta.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual double *getPropertyPointer(const std::string &property) override¶

Retrieves the pointer of a conditional value of a distribution property.

Parameters: property – Name of the property to find.
Returns: The pointer to the property..

virtual void updateDistribution() override¶: Updates the parameters of the distribution based on conditional variables.

virtual double getDensity(const double x) const override¶

Gets the probability density of the distribution at point x.

Parameters: x – point to evaluate P(x)
Returns: Value of the probability density.

virtual double getLogDensity(double x) const override¶

Gets the Log probability density of the distribution at point x.

Parameters: x – point to evaluate log(P(x))
Returns: Log of probability density.

virtual double getLogDensityGradient(double x) const override¶

Gets the Gradient of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate grad(log(P(x)))
Returns: Gradient of log of probability density.

virtual double getLogDensityHessian(double x) const override¶

Gets the second derivative of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate H(log(P(x)))
Returns: Hessian of log of probability density.

virtual double getRandomNumber() override¶

Draws and returns a random number from the distribution.

Returns: Random real number.

Public Members

double _alpha¶: [Conditional Variable Value]

std::string _alphaConditional¶: [Conditional Variable Reference]

double _beta¶: [Conditional Variable Value]

std::string _betaConditional¶: [Conditional Variable Reference]

template<typename valType, typename timerType> struct korali::cacheElement_t¶

#include <kcache.hpp>

Struct that defines an element present in Korali’s cache structure.

Public Members

valType value¶: Value of the element.

timerType time¶: Time when the element was last updated.

class korali::distribution::univariate::Cauchy : public korali::distribution::Univariate ¶

#include <cauchy.hpp>

Class declaration for module: Cauchy.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual double *getPropertyPointer(const std::string &property) override¶

Retrieves the pointer of a conditional value of a distribution property.

Parameters: property – Name of the property to find.
Returns: The pointer to the property..

virtual void updateDistribution() override¶: Updates the parameters of the distribution based on conditional variables.

virtual double getDensity(const double x) const override¶

Gets the probability density of the distribution at point x.

Parameters: x – point to evaluate P(x)
Returns: Value of the probability density.

virtual double getLogDensity(const double x) const override¶

Gets the Log probability density of the distribution at point x.

Parameters: x – point to evaluate log(P(x))
Returns: Log of probability density.

virtual double getLogDensityGradient(double x) const override¶

Gets the Gradient of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate grad(log(P(x)))
Returns: Gradient of log of probability density.

virtual double getLogDensityHessian(double x) const override¶

Gets the second derivative of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate H(log(P(x)))
Returns: Hessian of log of probability density.

virtual double getRandomNumber() override¶

Draws and returns a random number from the distribution.

Returns: Random real number.

Public Members

double _location¶: [Conditional Variable Value] Specifies the location of the peak of the distribution.

std::string _locationConditional¶: [Conditional Variable Reference] Specifies the location of the peak of the distribution.

double _scale¶: [Conditional Variable Value] Specifies the half-width at half-maximum (HWHM)

std::string _scaleConditional¶: [Conditional Variable Reference] Specifies the half-width at half-maximum (HWHM)

template<typename T> class korali::cBuffer¶

#include <cbuffer.hpp>

This class defines a circular buffer with overwrite policy on add.

Public Functions

inline cBuffer()¶: Default constructor.

inline cBuffer(size_t size)¶

Constructor with a specific size.

Parameters: size – The buffer size

inline size_t size()¶

Returns the current number of elements in the buffer.

Returns: The number of elements

inline void resize(size_t maxSize)¶

Returns the current number of elements in the buffer.

Parameters: maxSize – The buffer size

inline void add(const T &v)¶

Adds an element to the buffer.

Parameters: v – The element to add

inline std::vector<T> getVector()¶

Returns the elements of the buffer in a vector format.

Returns: The vector with the circular buffer elements

inline void clear()¶: Eliminates all contents of the buffer.

inline T &operator[](size_t pos)¶

Accesses an element at the required position.

Parameters: pos – The access position
Returns: The element corresponding to the position

Private Members

size_t _maxSize¶: Size of buffer container.

size_t _size¶: Number of elements already added.

std::unique_ptr<T[]> _data¶: Container for data.

size_t _start¶: Position of the start of the buffer.

size_t _end¶: Position of the end of the buffer.

class korali::solver::optimizer::CMAES : public korali::solver::Optimizer ¶

#include <CMAES.hpp>

Class declaration for module: CMAES.

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void prepareGeneration()¶: Prepares generation for the next set of evaluations.

void sampleSingle(size_t sampleIdx, const std::vector<double> &randomNumbers)¶

Evaluates a single sample.

Parameters

sampleIdx – Index of the sample to evaluate
randomNumbers – Random numbers to generate sample

void adaptC(int hsig)¶

Adapts the covariance matrix.

Parameters: hsig – Sign

void updateSigma()¶: Updates scaling factor of covariance matrix.

void updateDistribution()¶: Updates mean and covariance of Gaussian proposal distribution.

void updateEigensystem(const std::vector<double> &M)¶

Updates the system of eigenvalues and eigenvectors.

Parameters: M – Input matrix

void numericalErrorTreatment()¶: Method that checks potential numerical issues and does correction. Not yet implemented.

void eigen(size_t N, const std::vector<double> &C, std::vector<double> &diag, std::vector<double> &Q) const¶

Function for eigenvalue decomposition.

Parameters

N – Matrix size
C – Input matrix
diag – Sorted eigenvalues
Q – eingenvectors of C

void sort_index(const std::vector<double> &vec, std::vector<size_t> &_sortingIndex, size_t N) const¶

Descending sort of vector elements, stores ordering in _sortingIndex.

Parameters

_sortingIndex – Ordering of elements in vector
vec – Vector to sort
N – Number of current samples.

void initMuWeights(size_t numsamples)¶

Initializes the weights of the mu vector.

Parameters: numsamples – Length of mu vector

void initCovariance()¶: Initialize Covariance Matrix and Cholesky Decomposition.

void checkMeanAndSetRegime()¶: Check if mean of proposal distribution is inside of valid domain (does not violate constraints), if yes, re-initialize internal vars. Method for CCMA-ES.

void updateConstraints()¶: Update constraint evaluationsa. Method for CCMA-ES.

void updateViabilityBoundaries()¶: Update viability boundaries. Method for CCMA-ES.

void handleConstraints()¶: Process samples that violate constraints. Method for CCMA-ES.

void reEvaluateConstraints()¶: Reevaluate constraint evaluations. Called in handleConstraints. Method for CCMA-ES.

void updateDiscreteMutationMatrix()¶: Update mutation matrix for discrete variables. Method for discrete/integer optimization.

void discretize(std::vector<double> &sample)¶

Discretize variables to given granularity using arithmetic rounding.

Parameters: sample – Sample to discretize

virtual void setInitialConfiguration() override¶: Configures CMA-ES.

virtual void runGeneration() override¶: Executes sampling & evaluation generation.

virtual void printGenerationBefore() override¶: Console Output before generation runs.

virtual void printGenerationAfter() override¶: Console output after generation.

virtual void finalize() override¶: Final console output at termination.

Public Members

size_t _populationSize¶: Specifies the number of samples to evaluate per generation (preferably $4+3*log(N)$, where $N$ is the number of variables).

size_t _muValue¶: Number of best samples (offspring samples) used to update the covariance matrix and the mean (by default it is half the Sample Count).

std::string _muType¶: Weights given to the Mu best values to update the covariance matrix and the mean.

double _initialSigmaCumulationFactor¶: Controls the learning rate of the conjugate evolution path (by default this variable is internally calibrated).

double _initialDampFactor¶: Controls the updates of the covariance matrix scaling factor (by default this variable is internally calibrated).

int _useGradientInformation¶: Include gradient information for proposal distribution update.

float _gradientStepSize¶: Scaling factor for gradient step, only relevant if gradient information used.

int _isSigmaBounded¶: Sets an upper bound for the covariance matrix scaling factor. The upper bound is given by the average of the initial standard deviation of the variables.

double _initialCumulativeCovariance¶: Controls the learning rate of the evolution path for the covariance update (must be in (0,1], by default this variable is internally calibrated).

int _diagonalCovariance¶: Covariance matrix updates will be optimized for diagonal matrices.

int _mirroredSampling¶: Generate the negative counterpart of each random number during sampling.

size_t _viabilityPopulationSize¶: Specifies the number of samples per generation during the viability regime, i.e. during the search for a parameter vector not violating the constraints.

size_t _viabilityMuValue¶: Number of best samples used to update the covariance matrix and the mean during the viability regime (by default this variable is half the Viability Sample Count).

size_t _maxCovarianceMatrixCorrections¶: Max number of covairance matrix adaptions per generation during the constraint handling loop.

double _targetSuccessRate¶: Controls the updates of the covariance matrix scaling factor during the viability regime.

double _covarianceMatrixAdaptionStrength¶: Controls the covariane matrix adaption strength if samples violate constraints.

double _normalVectorLearningRate¶: Learning rate of constraint normal vectors (must be in (0, 1], by default this variable is internally calibrated).

double _globalSuccessLearningRate¶: Learning rate of success probability of objective function improvements.

korali::distribution::univariate::Normal *_normalGenerator¶: [Internal Use] Normal random number generator.

korali::distribution::univariate::Uniform *_uniformGenerator¶: [Internal Use] Uniform random number generator.

int _isViabilityRegime¶: [Internal Use] True if mean is outside feasible domain. During viability regime CMA-ES is working with relaxed constraint boundaries that contract towards the true constraint boundaries.

std::vector<double> _valueVector¶: [Internal Use] Objective function values.

std::vector<std::vector<double>> _gradients¶: [Internal Use] Gradients of objective function evaluations.

size_t _currentPopulationSize¶: [Internal Use] Actual number of samples used per generation (Population Size or Viability Population Size).

size_t _currentMuValue¶: [Internal Use] Actual value of mu (Mu Value or Viability Mu Value).

std::vector<double> _muWeights¶: [Internal Use] Calibrated Weights for each of the Mu offspring samples.

double _effectiveMu¶: [Internal Use] Variance effective selection mass.

double _sigmaCumulationFactor¶: [Internal Use] Increment for sigma, calculated from muEffective and dimension.

double _dampFactor¶: [Internal Use] Dampening parameter controls step size adaption.

double _cumulativeCovariance¶: [Internal Use] Controls the step size adaption.

double _chiSquareNumber¶: [Internal Use] Expectation of $||N(0,I)||^2$.

size_t _covarianceEigenvalueEvaluationFrequency¶: [Internal Use] Establishes how frequently the eigenvalues are updated.

double _sigma¶: [Internal Use] Determines the step size.

double _trace¶: [Internal Use] The trace of the initial covariance matrix.

std::vector<std::vector<double>> _samplePopulation¶: [Internal Use] Sample coordinate information.

size_t _finishedSampleCount¶: [Internal Use] Counter of evaluated samples to terminate evaluation.

std::vector<double> _currentBestVariables¶: [Internal Use] Best variables of current generation.

double _previousBestValue¶: [Internal Use] Best model evaluation from previous generation.

size_t _bestSampleIndex¶: [Internal Use] Index of the best sample in current generation.

double _previousBestEverValue¶: [Internal Use] Best ever model evaluation as of previous generation.

std::vector<size_t> _sortingIndex¶: [Internal Use] Sorted indeces of samples according to their model evaluation.

std::vector<double> _covarianceMatrix¶: [Internal Use] (Unscaled) covariance Matrix of proposal distribution.

std::vector<double> _auxiliarCovarianceMatrix¶: [Internal Use] Temporary Storage for Covariance Matrix.

std::vector<double> _covarianceEigenvectorMatrix¶: [Internal Use] Matrix with eigenvectors in columns.

std::vector<double> _auxiliarCovarianceEigenvectorMatrix¶: [Internal Use] Temporary Storage for Matrix with eigenvectors in columns.

std::vector<double> _axisLengths¶: [Internal Use] Axis lengths (sqrt(Evals))

std::vector<double> _auxiliarAxisLengths¶: [Internal Use] Temporary storage for Axis lengths.

std::vector<double> _bDZMatrix¶: [Internal Use] Temporary storage.

std::vector<double> _auxiliarBDZMatrix¶: [Internal Use] Temporary storage.

std::vector<double> _currentMean¶: [Internal Use] Current mean of proposal distribution.

std::vector<double> _previousMean¶: [Internal Use] Previous mean of proposal distribution.

std::vector<double> _meanUpdate¶: [Internal Use] Update differential from previous to current mean.

std::vector<double> _evolutionPath¶: [Internal Use] Evolution path for Covariance Matrix update.

std::vector<double> _conjugateEvolutionPath¶: [Internal Use] Conjugate evolution path for sigma update.

double _conjugateEvolutionPathL2Norm¶: [Internal Use] L2 Norm of the conjugate evolution path.

size_t _infeasibleSampleCount¶: [Internal Use] Keeps count of the number of infeasible samples.

double _maximumDiagonalCovarianceMatrixElement¶: [Internal Use] Maximum diagonal element of the Covariance Matrix.

double _minimumDiagonalCovarianceMatrixElement¶: [Internal Use] Minimum diagonal element of the Covariance Matrix.

double _maximumCovarianceEigenvalue¶: [Internal Use] Maximum Covariance Matrix Eigenvalue.

double _minimumCovarianceEigenvalue¶: [Internal Use] Minimum Covariance Matrix Eigenvalue.

int _isEigensystemUpdated¶: [Internal Use] Flag determining if the covariance eigensystem is up to date.

std::vector<std::vector<int>> _viabilityIndicator¶: [Internal Use] Evaluation of each constraint for each sample.

int _hasConstraints¶: [Internal Use] True if the number of constraints is higher than zero.

double _covarianceMatrixAdaptionFactor¶: [Internal Use] This is the beta factor that indicates how fast the covariance matrix is adapted.

int _bestValidSample¶: [Internal Use] Index of best sample without constraint violations (otherwise -1).

double _globalSuccessRate¶: [Internal Use] Estimated Global Success Rate, required for calibration of covariance matrix scaling factor updates.

double _viabilityFunctionValue¶: [Internal Use] Viability Function Value.

size_t _resampledParameterCount¶: [Internal Use] Number of resampled parameters due constraint violation.

size_t _covarianceMatrixAdaptationCount¶: [Internal Use] Number of Covariance Matrix Adaptations.

std::vector<double> _viabilityBoundaries¶: [Internal Use] Viability Boundaries.

std::vector<int> _viabilityImprovement¶: [Internal Use] Sample evaluations larger than fviability.

size_t _maxConstraintViolationCount¶: [Internal Use] Temporary counter of maximal amount of constraint violations attained by a sample (must be 0).

std::vector<size_t> _sampleConstraintViolationCounts¶: [Internal Use] Maximal amount of constraint violations.

std::vector<std::vector<double>> _constraintEvaluations¶: [Internal Use] Functions to be evaluated as constraint evaluations, if the return from any of them is > 0, then the constraint is met.

std::vector<std::vector<double>> _normalConstraintApproximation¶: [Internal Use] Normal approximation of constraints.

std::vector<double> _bestConstraintEvaluations¶: [Internal Use] Constraint evaluations for best ever.

int _hasDiscreteVariables¶: [Internal Use] Flag indicating if at least one of the variables is discrete.

std::vector<double> _discreteMutations¶: [Internal Use] Vector storing discrete mutations, required for covariance matrix update.

size_t _numberOfDiscreteMutations¶: [Internal Use] Number of discrete mutations in current generation.

size_t _numberMaskingMatrixEntries¶: [Internal Use] Number of nonzero entries on diagonal in Masking Matrix.

std::vector<double> _maskingMatrix¶: [Internal Use] Diagonal Matrix signifying where an integer mutation may be conducted.

std::vector<double> _maskingMatrixSigma¶: [Internal Use] Sigma of the Masking Matrix.

double _chiSquareNumberDiscreteMutations¶: [Internal Use] Expectation of $||N(0,I^S)||^2$ for discrete mutations.

double _currentMinStandardDeviation¶: [Internal Use] Current minimum standard deviation of any variable.

double _currentMaxStandardDeviation¶: [Internal Use] Current maximum standard deviation of any variable.

size_t _constraintEvaluationCount¶: [Internal Use] Number of Constraint Evaluations.

size_t _maxInfeasibleResamplings¶: [Termination Criteria] Maximum number of resamplings per candidate per generation if sample is outside of Lower and Upper Bound.

double _maxConditionCovarianceMatrix¶: [Termination Criteria] Specifies the maximum condition of the covariance matrix.

double _minStandardDeviation¶: [Termination Criteria] Specifies the minimal standard deviation for any variable in any proposed sample.

double _maxStandardDeviation¶: [Termination Criteria] Specifies the maximal standard deviation for any variable in any proposed sample.

class korali::conduit::Concurrent : public korali::Conduit ¶

#include <concurrent.hpp>

Class declaration for module: Concurrent.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool isRoot() override¶

Determines whether the caller rank/thread/process is root.

Returns: True, if it is root; false, otherwise.

virtual void initServer() override¶: Initializes the worker/server bifurcation in the conduit.

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void terminateServer() override¶: Finalizes the workers.

virtual void stackEngine(Engine *engine) override¶

Stacks a new Engine into the engine stack.

Parameters: engine – A Korali Engine

virtual void popEngine() override¶: Pops the current Engine from the engine stack.

virtual void listenWorkers() override¶: (Engine <- Worker) Receives all pending incoming messages and stores them into the corresponding sample’s message queue.

virtual void broadcastMessageToWorkers(knlohmann::json &message) override¶

(Engine -> Worker) Broadcasts a message to all workers

Parameters: message – JSON object with information to broadcast

virtual void sendMessageToEngine(knlohmann::json &message) override¶

(Sample -> Engine) Sends an update to the engine to provide partial information while the sample is still active

Parameters: message – Message to send to engine

virtual knlohmann::json recvMessageFromEngine() override¶

(Sample <- Engine) Blocking call that waits until any message incoming from the engine.

Returns: message from the engine.

virtual void sendMessageToSample(Sample &sample, knlohmann::json &message) override¶

(Engine -> Sample) Sends an update to a still active sample

Parameters

sample – The sample from which to receive an update
message – Message to send to the sample.

virtual size_t getProcessId() override¶

Returns the identifier corresponding to the executing process (to differentiate their random seeds)

Returns: The executing process id

Public Members

size_t _concurrentJobs¶: Specifies the number of worker processes (jobs) running concurrently.

std::vector<pid_t> _workerPids¶: PID of worker processes.

int _workerId¶: Worker Id for current workers - 0 for the master process.

std::vector<std::vector<int>> _resultContentPipe¶: OS Pipe to handle result contents communication coming from worker processes.

std::vector<std::vector<int>> _resultSizePipe¶: OS Pipe to handle result size communication coming from worker processes.

std::vector<std::vector<int>> _inputsPipe¶: OS Pipe to handle sample parameter communication to worker processes.

struct korali::problem::hierarchical::Psi::conditionalPriorInfo¶

Stores the pre-computed positions (pointers) of the conditional priors to evaluate for performance.

Public Members

std::vector<size_t> _samplePositions¶: Stores the position of the conditional prior.

std::vector<double*> _samplePointers¶: Stores the pointer of the conditional prior.

class korali::Conduit : public korali::Module ¶

#include <conduit.hpp>

Class declaration for module: Conduit.

Subclassed by korali::conduit::Concurrent, korali::conduit::Distributed, korali::conduit::Sequential

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void worker()¶: Lifetime function for korali workers.

inline virtual bool isRoot()¶

Determines whether the caller rank/thread/process is root.

Returns: True, if it is root; false, otherwise.

void workerProcessSample(const knlohmann::json &js)¶

(Worker Side) Starts the processing of a sample at the worker side

Parameters: js – Contains sample’s input data and metadata

void workerStackEngine(const knlohmann::json &js)¶

(Worker Side) Accepts and stacks an incoming Korali engine from the main process

Parameters: js – Contains Engine’s input data and metadata

void workerPopEngine()¶: (Worker Side) Pops the top of the engine stack

void start(Sample &sample)¶

Starts the execution of the sample.

Parameters: sample – A Korali sample

void wait(Sample &sample)¶

Waits for a given sample to finish. The experiment will not continue until the sample has been evaluated.

Parameters: sample – A Korali sample

void waitAll(std::vector<Sample> &samples)¶

Waits for a set of sample to finish. The experiment will not continue until all samples have been evaluated.

Parameters: samples – A list of Korali samples

size_t waitAny(std::vector<Sample> &samples)¶

Waits for a set of sample to finish. The experiment will not continue until at least one of the samples have been evaluated.

Parameters: samples – A list of Korali samples
Returns: Position in the vector of the sample that has finished.

virtual void stackEngine(Engine *engine) = 0¶

Stacks a new Engine into the engine stack.

Parameters: engine – A Korali Engine

virtual void popEngine() = 0¶: Pops the current Engine from the engine stack.

void runSample(Sample *sample, Engine *engine)¶

Starts the execution of a sample, given an Engine.

Parameters

sample – the sample to execute
engine – The Korali engine to use for its execution

virtual void initServer() = 0¶: Initializes the worker/server bifurcation in the conduit.

virtual void terminateServer() = 0¶: Finalizes the workers.

virtual void broadcastMessageToWorkers(knlohmann::json &message) = 0¶

(Engine -> Worker) Broadcasts a message to all workers

Parameters: message – JSON object with information to broadcast

virtual void listenWorkers() = 0¶: (Engine <- Worker) Receives all pending incoming messages and stores them into the corresponding sample’s message queue.

void listen(std::vector<Sample> &samples)¶

Start pending samples and retrieve any pending messages for them.

Parameters: samples – The set of samples

virtual void sendMessageToEngine(knlohmann::json &message) = 0¶

(Sample -> Engine) Sends an update to the engine to provide partial information while the sample is still active

Parameters: message – Message to send to engine

virtual knlohmann::json recvMessageFromEngine() = 0¶

(Sample <- Engine) Blocking call that waits until any message incoming from the engine.

Returns: message from the engine.

virtual void sendMessageToSample(Sample &sample, knlohmann::json &message) = 0¶

(Engine -> Sample) Sends an update to a still active sample

Parameters

sample – The sample from which to receive an update
message – Message to send to the sample.

virtual size_t getProcessId() = 0¶

Returns the identifier corresponding to the executing process (to differentiate their random seeds)

Returns: The executing process id

Public Members

std::queue<size_t> _workerQueue¶: Queue to store idle workers to assign samples to.

std::map<size_t, Sample*> _workerToSampleMap¶: Map that links workers to their currently-executing sample.

Public Static Functions

static void coroutineWrapper()¶: Wrapper function for the sample coroutine.

class korali::problem::reinforcementLearning::Continuous : public korali::problem::ReinforcementLearning ¶

#include <continuous.hpp>

Class declaration for module: Continuous.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool runOperation(std::string operation, korali::Sample &sample) override¶

Runs the operation specified on the given sample. It checks recursively whether the function was found by the current module or its parents.

Parameters

sample – Sample to operate on. Should contain in the ‘Operation’ field an operation accepted by this module or its parents.
operation – Should specify an operation type accepted by this module or its parents.

Returns

True, if operation found and executed; false, otherwise.

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

class korali::solver::agent::Continuous : public korali::solver::Agent ¶

#include <continuous.hpp>

Class declaration for module: Continuous.

Subclassed by korali::solver::agent::continuous::VRACER

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

std::vector<float> calculateImportanceWeightGradient(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy)¶

Calculates the gradient of teh importance weight wrt to the parameter of the 2nd (current) distribution evaluated at old action.

Parameters

action – The action taken by the agent in the given experience
oldPolicy – The policy for the given state used at the time the action was performed
curPolicy – The current policy for the given state

Returns

gradient of policy wrt curParamsOne and curParamsTwo

std::vector<float> calculateKLDivergenceGradient(const policy_t &oldPolicy, const policy_t &curPolicy)¶

Calculates the gradient of KL(p_old, p_cur) wrt to the parameter of the 2nd (current) distribution.

Parameters

oldPolicy – The policy for the given state used at the time the action was performed
curPolicy – The current policy for the given state

Returns

std::vector<float> generateTrainingAction(policy_t &curPolicy)¶

Function to generate randomized actions from neural network output.

Parameters: curPolicy – The current policy for the given state
Returns: An action vector

std::vector<float> generateTestingAction(const policy_t &curPolicy)¶

Function to generate deterministic actions from neural network output required for policy evaluation, respectively testing.

Parameters: curPolicy – The current policy for the given state
Returns: An action vector

virtual float calculateImportanceWeight(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy) override¶

Calculates importance weight of current action from old and new policies.

Parameters

action – The action taken
curPolicy – The current policy
oldPolicy – The old policy, the one used for take the action in the first place

Returns

The importance weight

virtual void getAction(korali::Sample &sample) override¶

Gathers the next action either from the policy or randomly.

Parameters: sample – Sample on which the action and metadata will be stored

virtual void initializeAgent() override¶: Initializes the internal state of the policy.

Public Members

std::string _policyDistribution¶: Specifies which probability distribution to use for the policy.

korali::distribution::univariate::Normal *_normalGenerator¶: [Internal Use] Gaussian random number generator to generate the agent’s action.

std::vector<float> _actionShifts¶: [Internal Use] Shifts required for bounded actions.

std::vector<float> _actionScales¶: [Internal Use] Scales required for bounded actions (half the action domain width).

size_t _policyParameterCount¶: [Internal Use] Stores the number of parameters that determine the probability distribution for the current state sequence.

std::vector<std::string> _policyParameterTransformationMasks¶: [Internal Use] Stores the transformations required for each parameter.

std::vector<float> _policyParameterScaling¶: [Internal Use] Stores the scaling required for the parameter after the transformation is applied.

std::vector<float> _policyParameterShifting¶: [Internal Use] Stores the shifting required for the parameter after the scaling is applied.

problem::reinforcementLearning::Continuous *_problem¶: Storage for the pointer to the (continuous) learning problem.

class korali::problem::bayesian::Custom : public korali::problem::Bayesian ¶

#include <custom.hpp>

Class declaration for module: Custom.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void evaluateLoglikelihood(korali::Sample &sample) override¶

Evaluates the log likelihood of the given sample, and stores it in sample[“Log Likelihood”].

Parameters: sample – A Korali Sample

virtual void evaluateLoglikelihoodGradient(korali::Sample &sample) override¶

Evaluates the gradient of the logLikelikood w.r.t. to the variables, and stores it in sample[“logLikelihood Gradient”].

Parameters: sample – A Korali Sample

virtual void evaluateFisherInformation(korali::Sample &sample) override¶

Evaluates the empirical Fisher information.

Parameters: sample – A Korali Sample

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

Public Members

std::uint64_t _likelihoodModel¶: Stores the user-defined likelihood model. It should return the value of the Log Likelihood of the given sample.

class korali::solver::optimizer::DEA : public korali::solver::Optimizer ¶

#include <DEA.hpp>

Class declaration for module: DEA.

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void setInitialConfiguration() override¶: Configures Differential Evolution/.

virtual void runGeneration() override¶: Executes sampling & evaluation generation.

virtual void printGenerationBefore() override¶: Console Output before generation runs.

virtual void printGenerationAfter() override¶: Console output after generation.

virtual void finalize() override¶: Final console output at termination.

Public Members

size_t _populationSize¶: Specifies the number of samples to evaluate per generation (preferably 5-10x the number of variables).

double _crossoverRate¶: Controls the rate at which dimensions of the samples are mixed (must be in [0,1]).

double _mutationRate¶: Controls the scaling of the vector differentials (must be in [0,2], preferably < 1).

std::string _mutationRule¶: Controls the Mutation Rate.

std::string _parentSelectionRule¶: Defines the selection rule of the parent vector.

std::string _acceptRule¶: Sets the accept rule after sample mutation and evaluation.

int _fixInfeasible¶: If set true, Korali samples a random sample between Parent and the voiolated boundary. If set false, infeasible samples are mutated again until feasible.

korali::distribution::univariate::Normal *_normalGenerator¶: [Internal Use] Normal random number generator.

korali::distribution::univariate::Uniform *_uniformGenerator¶: [Internal Use] Uniform random number generator.

std::vector<double> _valueVector¶: [Internal Use] Objective Function Values.

std::vector<double> _previousValueVector¶: [Internal Use] Objective Function Values from previous evaluations.

std::vector<std::vector<double>> _samplePopulation¶: [Internal Use] Sample variable information.

std::vector<std::vector<double>> _candidatePopulation¶: [Internal Use] Sample candidates variable information.

size_t _bestSampleIndex¶: [Internal Use] Index of the best sample in current generation.

double _previousBestEverValue¶: [Internal Use] Best ever model evaluation as of previous generation.

std::vector<double> _currentMean¶: [Internal Use] Current mean of population.

std::vector<double> _previousMean¶: [Internal Use] Previous mean of population.

std::vector<double> _currentBestVariables¶: [Internal Use] Best variables of current generation.

std::vector<double> _maxDistances¶: [Internal Use] Max distance between samples per dimension.

size_t _infeasibleSampleCount¶: [Internal Use] Keeps count of infeasible samples so far.

double _currentMinimumStepSize¶: [Internal Use] Minimum step size of any variable in the current generation.

size_t _maxInfeasibleResamplings¶: [Termination Criteria] Max number of mutations per sample per generation if infeasible (only relevant if Fix Infeasible is set False).

double _minValue¶: [Termination Criteria] Specifies the target fitness to stop minimization.

double _minStepSize¶: [Termination Criteria] Specifies the minimal step size of the population mean from one gneration to another.

Private Functions

void mutateSingle(size_t sampleIdx)¶

Mutate a sample.

Parameters: sampleIdx – Index of sample to be mutated.

void fixInfeasible(size_t sampleIdx)¶

Fix sample params that are outside of domain.

Parameters: sampleIdx – Index of sample that is outside of domain.

void updateSolver(std::vector<Sample> &samples)¶

Update the state of Differential Evolution.

Parameters: samples – Sample evaluations.

void initSamples()¶: Create new set of candidates.

void prepareGeneration()¶: Mutate samples and distribute them.

class korali::solver::learner::DeepSupervisor : public korali::solver::Learner ¶

#include <deepSupervisor.hpp>

Class declaration for module: DeepSupervisor.

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void resetOptimizer()¶: Resets the optimizer after passing new hyperparameters.

virtual std::vector<std::vector<float>> &getEvaluation(const std::vector<std::vector<std::vector<float>>> &input) override¶

For learner modules which have been trained, test returns an inferred output batch, from a batch of inputs to process.

Parameters: input – The inputs from which to infer outputs. Format: BxTxIC (B: Batch Size, T: Time steps, IC: Input channels)
Returns: The inferred batch outputs for the last given timestep. Format: BxOC (B: Batch Size, OC: Output channels)

virtual std::vector<float> getHyperparameters() override¶

Returns the hyperparameters required to continue training in the future.

Returns: The hyperparameters

virtual void setHyperparameters(const std::vector<float> &hyperparameters) override¶

Sets the hyperparameters required to continue training from a previous state.

Parameters: hyperparameters – The hyperparameters to use

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void runGeneration() override¶: Runs the current generation.

virtual void printGenerationAfter() override¶: Prints solver information after the execution of the current generation.

Public Members

knlohmann::json _neuralNetworkHiddenLayers¶: Sets the configuration of the hidden layers for the neural network.

knlohmann::json _neuralNetworkOutputActivation¶: Allows setting an aditional activation for the output layer.

knlohmann::json _neuralNetworkOutputLayer¶: Sets any additional configuration (e.g., masks) for the output NN layer.

std::string _neuralNetworkEngine¶: Specifies which Neural Network backend engine to use.

std::string _neuralNetworkOptimizer¶: Determines which optimizer algorithm to use to apply the gradients on the neural network’s hyperparameters.

std::vector<float> _hyperparameters¶: Stores the training neural network hyperparameters (weights and biases).

std::string _lossFunction¶: Function to calculate the difference (loss) between the NN inference and the exact solution and its gradients for optimization.

size_t _stepsPerGeneration¶: Represents the number of opitmization steps to run per each generation.

float _learningRate¶: Learning rate for the underlying ADAM optimizer.

int _l2RegularizationEnabled¶: Regulates if l2 regularization will be applied to the neural network.

int _l2RegularizationImportance¶: Importance weight of l2 regularization.

float _outputWeightsScaling¶: Specified by how much will the weights of the last linear transformation of the NN be scaled. A value of < 1.0 is useful for a more deterministic start.

float _currentLoss¶: [Internal Use] Current value of the loss function.

std::vector<float> _normalizationMeans¶: [Internal Use] Stores the current neural network normalization mean parameters.

std::vector<float> _normalizationVariances¶: [Internal Use] Stores the current neural network normalization variance parameters.

float _targetLoss¶: [Termination Criteria] Specifies the maximum number of suboptimal generations.

problem::SupervisedLearning *_problem¶: Korali Problem for optimizing NN weights and biases.

korali::Experiment _optExperiment¶: Korali Experiment for optimizing the NN’s weights and biases.

korali::fGradientBasedOptimizer *_optimizer¶: Gradient-based solver pointer to access directly (for performance)

NeuralNetwork *_neuralNetwork¶: A neural network to be trained based on inputs and solutions.

class korali::problem::reinforcementLearning::Discrete : public korali::problem::ReinforcementLearning ¶

#include <discrete.hpp>

Class declaration for module: Discrete.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual bool runOperation(std::string operation, korali::Sample &sample) override¶

Runs the operation specified on the given sample. It checks recursively whether the function was found by the current module or its parents.

Parameters

sample – Sample to operate on. Should contain in the ‘Operation’ field an operation accepted by this module or its parents.
operation – Should specify an operation type accepted by this module or its parents.

Returns

True, if operation found and executed; false, otherwise.

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

Public Members

std::vector<std::vector<float>> _possibleActions¶: The set of all possible actions.

class korali::solver::agent::Discrete : public korali::solver::Agent ¶

#include <discrete.hpp>

Class declaration for module: Discrete.

Subclassed by korali::solver::agent::discrete::dVRACER

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual float calculateImportanceWeight(const std::vector<float> &action, const policy_t &curPolicy, const policy_t &oldPolicy) override¶

Calculates importance weight of current action from old and new policies.

Parameters

action – The action taken
curPolicy – The current policy
oldPolicy – The old policy, the one used for take the action in the first place

Returns

The importance weight

std::vector<float> calculateImportanceWeightGradient(const size_t actionIdx, const std::vector<float> &curPvalues, const std::vector<float> &oldPvalues)¶

Calculates the gradient of importance weight wrt to NN output.

Parameters

actionIdx – Action from memory
curPvalues – todo
oldPvalues – todo

Returns

gradient of importance weight wrt NN output

std::vector<float> calculateKLDivergenceGradient(const std::vector<float> &oldPvalues, const std::vector<float> &curPvalues)¶

Calculates the gradient of KL(p_old, p_cur) wrt to the parameter of the 2nd (current) distribution.

Parameters

oldPvalues – todo
curPvalues – todo

Returns

gradient of KL wrt curParamsOne and curParamsTwo

virtual void getAction(korali::Sample &sample) override¶

Gathers the next action either from the policy or randomly.

Parameters: sample – Sample on which the action and metadata will be stored

virtual void initializeAgent() override¶: Initializes the internal state of the policy.

Public Members

float _randomActionProbability¶: Specifies the probability of taking a random action for the epsilon-greedy strategy.

problem::reinforcementLearning::Discrete *_problem¶: Storage for the pointer to the (discrete) learning problem.

class korali::conduit::Distributed : public korali::Conduit ¶

#include <distributed.hpp>

Class declaration for module: Distributed.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void checkRankCount()¶: Checks whether the number of MPI workers satisfies the requirement.

virtual void initServer() override¶: Initializes the worker/server bifurcation in the conduit.

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void terminateServer() override¶: Finalizes the workers.

virtual void stackEngine(Engine *engine) override¶

Stacks a new Engine into the engine stack.

Parameters: engine – A Korali Engine

virtual void popEngine() override¶: Pops the current Engine from the engine stack.

virtual void listenWorkers() override¶: (Engine <- Worker) Receives all pending incoming messages and stores them into the corresponding sample’s message queue.

virtual void broadcastMessageToWorkers(knlohmann::json &message) override¶

(Engine -> Worker) Broadcasts a message to all workers

Parameters: message – JSON object with information to broadcast

virtual void sendMessageToEngine(knlohmann::json &message) override¶

(Sample -> Engine) Sends an update to the engine to provide partial information while the sample is still active

Parameters: message – Message to send to engine

virtual knlohmann::json recvMessageFromEngine() override¶

(Sample <- Engine) Blocking call that waits until any message incoming from the engine.

Returns: message from the engine.

virtual void sendMessageToSample(Sample &sample, knlohmann::json &message) override¶

(Engine -> Sample) Sends an update to a still active sample

Parameters

sample – The sample from which to receive an update
message – Message to send to the sample.

virtual size_t getProcessId() override¶

Returns the identifier corresponding to the executing process (to differentiate their random seeds)

Returns: The executing process id

int getRootRank()¶

Determines which rank is the root.

Returns: The rank id of the root rank.

virtual bool isRoot() override¶

Determines whether the caller rank/thread/process is root.

Returns: True, if it is root; false, otherwise.

Public Members

int _ranksPerWorker¶: Specifies the number of MPI ranks per Korali worker (k).

int _rankId¶: ID of the current rank.

int _rankCount¶: Total number of ranks in execution.

int _workerCount¶: Number of Korali Teams in execution.

int _workerIdSet¶: Signals whether the worker has been assigned a team.

int _localRankId¶: Local ID the rank within its Korali Worker.

std::vector<std::vector<int>> _workerTeams¶: Storage that contains the rank teams for each worker.

std::vector<int> _rankToWorkerMap¶: Map that indicates to which worker does the current rank correspond to.

class korali::Distribution : public korali::Module ¶

#include <distribution.hpp>

Class declaration for module: Distribution.

Subclassed by korali::distribution::Multivariate, korali::distribution::Specific, korali::distribution::Univariate

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

gsl_rng *setRange(const std::string rangeString)¶

Creates and sets the RNG range (state and seed) of the random distribution.

Parameters: rangeString – The range to load, in string of hexadecimal values form
Returns: Pointer to the new range.

std::string getRange(gsl_rng *range) const¶

Gets a hexadecimal string from a given range’s state and seed.

Parameters: range – Range to read from
Returns: Hexadecimal string produced.

inline virtual void updateDistribution()¶: Updates the parameters of the distribution based on conditional variables.

inline virtual double *getPropertyPointer(const std::string &property)¶

Gets the pointer to a distribution property.

Parameters: property – The name of the property to update
Returns: Pointer to the property

Public Members

std::string _name¶: Defines the name of the distribution.

size_t _randomSeed¶: Defines the random seed of the distribution.

gsl_rng *_range¶: Stores the current state of the distribution in hexadecimal notation.

std::map<std::string, double*> _conditionalsMap¶: Map to store the link between parameter names and their pointers.

double _aux¶: Auxiliar variable to hold pre-calculated data to avoid re-processing information.

bool _hasConditionalVariables¶: Indicates whether or not this distribution contains conditional variables.

class korali::solver::agent::discrete::dVRACER : public korali::solver::agent::Discrete ¶

#include <dVRACER.hpp>

Class declaration for module: dVRACER.

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

void updateVtbc(size_t expId)¶

Update the V-target or current and previous experiences in the episode.

Parameters: expId – Current Experience Id

void calculatePolicyGradients(const std::vector<size_t> &miniBatch)¶

Calculates the gradients for the policy/critic neural network.

Parameters: miniBatch – The indexes of the experience mini batch

virtual std::vector<policy_t> runPolicy(const std::vector<std::vector<std::vector<float>>> &stateBatch) override¶

Function to pass a state time series through the NN and calculates the action probabilities, along with any additional information.

Parameters: stateBatch – The batch of state time series (Format: BxTxS, B is batch size, T is the time series lenght, and S is the state size)
Returns: A JSON object containing the information produced by the policies given the current state series

virtual knlohmann::json getAgentPolicy() override¶

Obtains the policy hyperaparamters from the learner for the agent to generate new actions.

Returns: The current policy hyperparameters

virtual void setAgentPolicy(const knlohmann::json &hyperparameters) override¶

Updates the agent’s hyperparameters.

Parameters: hyperparameters – The hyperparameters to update the agent.

virtual void trainPolicy() override¶: Trains the Agent’s policy, based on the new experiences.

virtual void resetAgentOptimizers() override¶: Resets the states of the optimizers.

virtual void printAgentInformation() override¶: Prints information about the training policy.

virtual void initializeAgent() override¶: Initializes the internal state of the policy.

Public Members

std::vector<float> _statisticsAverageActionSigmas¶: [Internal Use] Standard deviation of the actions in the minibatch.

learner::DeepSupervisor *_criticPolicyLearner¶: Pointer to training the actor network.

korali::Experiment _criticPolicyExperiment¶: Korali experiment for obtaining the agent’s action.

problem::SupervisedLearning *_criticPolicyProblem¶: Pointer to actor’s experiment problem.

struct korali::solver::sampler::ellipse_t¶

#include <Nested.hpp>

Ellipse object to generate bounds.

Public Functions

ellipse_t() = delete¶: Default c-tor (avoid empty initialization).

inline ellipse_t(size_t dim)¶

Init d-dimensional ellipse without covariance.

Parameters: dim – Dimension of ellipsoid.

void initSphere()¶: Init d-dimensional unit sphere.

void scaleVolume(double factor)¶

Scale volume.

Parameters: factor – Volume multiplicator.

Public Members

size_t dim¶: Dimension of ellipsoid.

size_t num¶: Number samples in ellipse.

double det¶: Determinant of covariance.

std::vector<size_t> sampleIdx¶: Indices of samples from live data set.

std::vector<double> mean¶: Mean vector of samples in ellipse.

std::vector<double> cov¶: Covariance Matrix of samples in ellipse.

std::vector<double> invCov¶: Inverse of Covariance Matrix.

std::vector<double> axes¶: Axes of the ellipse.

std::vector<double> evals¶: Eigenvalues of the ellipse.

std::vector<double> paxes¶: Principal axes of the ellipse.

double volume¶: Volume estimated from covariance.

double pointVolume¶: ‘True’ volume from which the subset of samples were sampled from.

class korali::Engine¶

#include <engine.hpp>

A Korali Engine initializes the conduit and experiments, and guides their execution.

Public Functions

Engine()¶

void saveProfilingInfo(const bool forceSave = false)¶

Saves the profiling information to the specified path.

Parameters: forceSave – Saves even if the current generation does not divide _profilingFrequency. Reserved for last generation.

void initialize()¶: Initialization stage of the Korali Engine.

void run(std::vector<Experiment> &experiments)¶

Stores a set experiments into the experiment list and runs them to completion.

Parameters: experiments – Set of experiments.

void run(Experiment &experiment)¶

Stores a single experiment into the experiment list and runs it to completion.

Parameters: experiment – The experiment to run.

void start()¶: Runs the stored list of experiments.

knlohmann::json &operator[](const std::string &key)¶

C++ wrapper for the getItem operator.

Parameters: key – A C++ string acting as JSON key.
Returns: The referenced JSON object content.

knlohmann::json &operator[](const unsigned long int &key)¶

C++ wrapper for the getItem operator.

Parameters: key – A C++ integer acting as JSON key.
Returns: The referenced JSON object content.

pybind11::object getItem(const pybind11::object key)¶

Gets an item from the JSON object at the current pointer position.

Parameters: key – A pybind11 object acting as JSON key (number or string).
Returns: A pybind11 object

void setItem(const pybind11::object key, const pybind11::object val)¶

Sets an item on the JSON object at the current pointer position.

Parameters

key – A pybind11 object acting as JSON key (number or string).
val – The value of the item to set.

void serialize(knlohmann::json &js)¶

Serializes Engine’s data into a JSON object.

Parameters: js – Json object onto which to store the Engine data.

Public Members

Conduit *_conduit¶: A pointer to the execution conduit. Shared among all experiments in the engine.

std::string _verbosityLevel¶: Verbosity level of the Engine (‘Silent’, ‘Minimal’ (default), ‘Normal’ or ‘Detailed’).

std::vector<Experiment*> _experimentVector¶: Stores the list of experiments to run.

cothread_t _thread¶: Stores the main execution thread (coroutine).

std::string _profilingPath¶: Saves the output path for the profiling information file.

std::string _profilingDetail¶: Specifies how much detail will be saved in the profiling file (None, Full)

double _profilingFrequency¶: Specifies every how many generation will the profiling file be updated.

std::chrono::time_point<std::chrono::high_resolution_clock> _profilingLastSave¶: Stores the timepoint of the last time the profiling information was saved.

KoraliJson _js¶: Stores the JSON based configuration for the engine.

bool _isDryRun¶: Determines whether this is a dry run (no conduit initialization nor execution)

Experiment *_currentExperiment¶: (Worker) Stores a pointer to the current Experiment being processed

Public Static Functions

static Engine *deserialize(const knlohmann::json &js)¶

Deserializes JSON object and returns a Korali Engine.

Parameters: js – Json object onto which to store the Engine data.
Returns: The Korali Engine

class korali::solver::Executor : public korali::Solver ¶

#include <executor.hpp>

Class declaration for module: Executor.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual void runGeneration() override¶: Generate a sample and evaluate it.

virtual void printGenerationBefore() override¶: Console Output before generation runs.

virtual void printGenerationAfter() override¶: Console output after generation.

Public Members

size_t _executionsPerGeneration¶: Specifies the number of model executions per generation. By default this setting is 0, meaning that all executions will be performed in the first generation. For values greater 0, executions will be split into batches and split int generations for intermediate output.

size_t _sampleCount¶: [Internal Use] Number of samples to execute.

class korali::Experiment : public korali::Module ¶

#include <experiment.hpp>

Class declaration for module: Experiment.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

Experiment()¶

~Experiment()¶

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void finalize() override¶: Finalizes Module. Deallocates memory and produces outputs.

pybind11::object getItem(const pybind11::object key)¶

Gets an item from the JSON object at the current pointer position.

Parameters: key – A pybind11 object acting as JSON key (number or string).
Returns: A pybind11 object

void setItem(const pybind11::object key, const pybind11::object val)¶

Sets an item on the JSON object at the current pointer position.

Parameters

key – A pybind11 object acting as JSON key (number or string).
val – The value of the item to set.

bool loadState(const std::string &path)¶

Load the state of an experiment from a Korali result file.

Parameters: path – Path from which to load the experiment state.
Returns: true, if file was found; false, otherwise

void saveState()¶: Saves the state into the experiment’s result path.

void run()¶: Start the execution of the current experiment.

knlohmann::json &operator[](const std::string &key)¶

C++ wrapper for the getItem operator.

Parameters: key – A C++ string acting as JSON key.
Returns: The referenced JSON object content.

std::vector<std::vector<float>> getEvaluation(const std::vector<std::vector<std::vector<float>>> &inputBatch)¶

For learner modules which have been trained, test returns an inferred output batch, from a batch of inputs to process.

Parameters: inputBatch – The inputs from which to infer outputs. Format: TxBxIC (T: Time steps, B: Batch Size, IC: Input channels)
Returns: The inferred outputs. Format: BxOC (Time steps, B: Batch Size, OC: Output channels)

void setSeed(knlohmann::json &js)¶

Initializes seed to a random value based on current time if not set by the user (i.e. Random Seed is 0).

Parameters: js – Json object onto which to store the Experiment data.

Public Members

size_t _randomSeed¶: Specifies the initializing seed for the generation of random numbers. If 0 is specified, Korali will automatically derivate a new seed base on the current time.

int _preserveRandomNumberGeneratorStates¶: Indicates that the engine must preserve the state of their RNGs for reproducibility purposes.

std::vector<korali::distribution::Univariate*> _distributions¶: Represents the distributions to use during execution.

std::vector<korali::Variable*> _variables¶: Sample coordinate information.

korali::Problem *_problem¶: Represents the configuration of the problem to solve.

korali::Solver *_solver¶: Represents the state and configuration of the solver algorithm.

std::string _fileOutputPath¶: Specifies the path of the results directory.

int _fileOutputEnabled¶: Specifies whether the partial results should be saved to the results directory.

size_t _fileOutputFrequency¶: Specifies how often (in generations) will partial result files be saved on the results directory. The default, 1, indicates that every generation’s results will be saved. 0 indicates that only the latest is saved.

int _storeSampleInformation¶: Specifies whether the sample information should be saved to samples.json in the results path.

std::string _consoleOutputVerbosity¶: Specifies how much information will be displayed on console when running Korali.

size_t _consoleOutputFrequency¶: Specifies how often (in generations) will partial results be printed on console. The default, 1, indicates that every generation’s results will be printed.

size_t _currentGeneration¶: [Internal Use] Indicates the current generation in execution.

int _isFinished¶: [Internal Use] Indicates whether execution has reached a termination criterion.

size_t _runID¶: [Internal Use] Specifies the Korali run’s unique identifier. Used to distinguish run results when two or more use the same output directory.

std::string _timestamp¶: [Internal Use] Indicates the current time when saving a result file.

KoraliJson _js¶: JSON object to store the experiment’s configuration.

Logger *_logger¶: A pointer to the Experiment’s logger object.

Engine *_engine¶: A pointer to the parent engine.

KoraliJson _sampleInfo¶: JSON object to details of all the samples that have been executed, if requested by the user.

size_t _experimentId¶: Experiment Identifier.

cothread_t _thread¶: Experiment’s coroutine (thread). It is swapped among other experiments, and sample threads.

bool _isInitialized¶: Flag to indicate that the experiment has been initialized to prevent it from re-initializing upon resuming.

double _resultSavingTime¶: [Profiling] Measures the amount of time taken by saving results

bool _overrideEngine = false¶: For testing purposes, this field establishes whether the engine is the one to run samples (default = false) or a custom function (true)

std::function<void(Sample&)> _overrideFunction¶: For testing purposes, this field establishes which custom function to use to override the engine on sample execution for testing.

class korali::distribution::univariate::Exponential : public korali::distribution::Univariate ¶

#include <exponential.hpp>

Class declaration for module: Exponential.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual double *getPropertyPointer(const std::string &property) override¶

Retrieves the pointer of a conditional value of a distribution property.

Parameters: property – Name of the property to find.
Returns: The pointer to the property..

virtual void updateDistribution() override¶: Updates the parameters of the distribution based on conditional variables.

virtual double getDensity(const double x) const override¶

Gets the probability density of the distribution at point x.

Parameters: x – point to evaluate P(x)
Returns: Value of the probability density.

virtual double getLogDensity(const double x) const override¶

Gets the Log probability density of the distribution at point x.

Parameters: x – point to evaluate log(P(x))
Returns: Log of probability density.

virtual double getLogDensityGradient(double x) const override¶

Gets the Gradient of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate grad(log(P(x)))
Returns: Gradient of log of probability density.

virtual double getLogDensityHessian(double x) const override¶

Gets the second derivative of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate H(log(P(x)))
Returns: Hessian of log of probability density.

virtual double getRandomNumber() override¶

Draws and returns a random number from the distribution.

Returns: Random real number.

Public Members

double _location¶: [Conditional Variable Value]

std::string _locationConditional¶: [Conditional Variable Reference]

double _mean¶: [Conditional Variable Value]

std::string _meanConditional¶: [Conditional Variable Reference]

class korali::fAdaBelief : public korali::fAdam ¶

#include <fAdaBelief.hpp>

Class declaration for module: AdaBelief.

Public Functions

fAdaBelief(size_t nVars)¶

Default constructor for the optimizer.

Parameters: nVars – Variable-space dimensionality

virtual void processResult(float evaluation, std::vector<float> &gradient) override¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override¶: Restores the optimizer to the initial state.

Public Members

std::vector<float> _secondCentralMoment¶: [Internal Use] Old estimate of second moment of Gradient.

class korali::fAdagrad : public korali::fGradientBasedOptimizer ¶

#include <fAdagrad.hpp>

Class declaration for module: MADGRAD.

Public Functions

fAdagrad(size_t nVars)¶

Default constructor for the optimizer.

Parameters: nVars – Variable-space dimensionality

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) override¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override¶: Restores the optimizer to the initial state.

virtual void printInfo() override¶: Prints progress information.

Public Members

std::vector<float> _s¶: Squared Gradient Component.

float _epsilon¶: Safety addition on divide.

class korali::fAdam : public korali::fGradientBasedOptimizer ¶

#include <fAdam.hpp>

Class declaration for module: Adam.

Subclassed by korali::fAdaBelief

Public Functions

fAdam(size_t nVars)¶

Default constructor for the optimizer.

Parameters: nVars – Variable-space dimensionality

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) override¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override¶: Restores the optimizer to the initial state.

virtual void printInfo() override¶: Prints progress information.

Public Members

float _beta1¶: Beta for momentum update.

float _beta2¶: Beta for gradient update.

float _epsilon¶: Smoothing Term.

std::vector<float> _firstMoment¶: [Internal Use] Estimate of first moment of Gradient.

std::vector<float> _secondMoment¶: [Internal Use] Old estimate of second moment of Gradient.

float _minGradientNorm¶: [Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

float _maxGradientNorm¶: [Termination Criteria] Specifies the minimal norm for the gradient of function with respect to Parameters.

class korali::fCMAES¶

#include <fCMAES.hpp>

Class declaration for module: fCMAES.

Public Functions

fCMAES(size_t nVars, size_t populationSize = 0, size_t muSize = 0)¶

Default constructor for the optimizer.

Parameters

nVars – Variable-space dimensionality
populationSize – How many samples per generation to use
muSize – How many sample selections for covariance matrix adaptation

~fCMAES()¶: Default destructor for the optimizer.

void setSeed(size_t seed)¶

Defines the random number generator seed.

Parameters: seed – Random seed

bool checkTermination()¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

void prepareGeneration()¶: Prepares generation for the next set of evaluations.

void sampleSingle(size_t sampleIdx)¶

Evaluates a single sample.

Parameters: sampleIdx – Index of the sample to evaluate

void adaptC(int hsig)¶

Adapts the covariance matrix.

Parameters: hsig – Sign

void updateSigma()¶: Updates scaling factor of covariance matrix.

void updateDistribution(const std::vector<float> &evaluations)¶

Updates mean and covariance of Gaussian proposal distribution.

Parameters: evaluations – Model evaluations for all proposed samples

void updateEigensystem(std::vector<float> &M)¶

Updates the system of eigenvalues and eigenvectors.

Parameters: M – Input matrix

void eigen(size_t N, std::vector<float> &C, std::vector<float> &diag, std::vector<float> &Q) const¶

Function for eigenvalue decomposition.

Parameters

N – Matrix size
C – Input matrix
diag – Means
Q – Output Matrix

void sort_index(const std::vector<float> &vec, std::vector<size_t> &_sortingIndex, size_t N) const¶

Descending sort of vector elements, stores ordering in _sortingIndex.

Parameters

_sortingIndex – Ordering of elements in vector
vec – Vector to sort
N – Number of current samples.

void initMuWeights(size_t numsamples)¶

Initializes the weights of the mu vector.

Parameters: numsamples – Length of mu vector

void initCovariance()¶: Initialize Covariance Matrix and Cholesky Decomposition.

void printInfo()¶: Console output after generation.

void reset()¶: Restores the optimizer to the initial state.

bool isSampleFeasible(const std::vector<float> &sample)¶

Checks whether a proposed sample is feasible (all variables within acceptable range)

Parameters: sample – Sample to check
Returns: Whether the sample is feasible

Public Members

size_t _nVars¶: Number of problem variables.

size_t _currentGeneration¶: Counter for the current generation.

std::vector<float> _initialMeans¶: Indicates the initial gaussian means for all variables.

std::vector<float> _initialStandardDeviations¶: Indicates the initial gaussian standard deviations for all variables.

std::vector<float> _lowerBounds¶: Indicates the lower bounds for all variables.

std::vector<float> _upperBounds¶: Indicates the upper bounds for all variables.

std::default_random_engine _randomGenerator¶: Random number generator.

std::normal_distribution<float> _normalGenerator¶: Gaussian number generator.

size_t _populationSize¶: Specifies the number of samples to evaluate per generation (default: $4+3*log(N)$, where $N$ is the number of variables).

size_t _muValue¶: Number of best samples used to update the covariance matrix and the mean (by default it is half the Sample Count).

std::string _muType¶: Weights given to the Mu best values to update the covariance matrix and the mean.

float _initialSigmaCumulationFactor¶: Controls the learning rate of the conjugate evolution path (by default this variable is internally calibrated).

float _initialDampFactor¶: Controls the updates of the covariance matrix scaling factor (by default this variable is internally calibrated).

int _isSigmaBounded¶: Sets an upper bound for the covariance matrix scaling factor. The upper bound is given by the average of the initial standard deviation of the variables.

float _initialCumulativeCovariance¶: Controls the learning rate of the evolution path for the covariance update (must be in (0,1], by default this variable is internally calibrated).

int _isDiagonal¶: Covariance matrix updates will be optimized for diagonal matrices.

std::vector<float> _valueVector¶: [Internal Use] Objective function values.

std::vector<float> _previousValueVector¶: [Internal Use] Objective function values from previous generation.

std::vector<float> _muWeights¶: [Internal Use] Weights for each of the Mu samples.

float _effectiveMu¶: [Internal Use] Variance effective selection mass.

float _sigmaCumulationFactor¶: [Internal Use] Increment for sigma, calculated from muEffective and dimension.

float _dampFactor¶: [Internal Use] Dampening parameter controls step size adaption.

float _cumulativeCovariance¶: [Internal Use] Controls the step size adaption.

float _chiSquareNumber¶: [Internal Use] Expectation of $||N(0,I)||^2$.

size_t _covarianceEigenvalueEvaluationFrequency¶: [Internal Use] Establishes how frequently the eigenvalues are updated.

float _sigma¶: [Internal Use] Determines the step size.

bool _noUpdatePossible¶: [Internal Use] This flag remembers if it was not possible to produce an update and terminates

float _trace¶: [Internal Use] The trace of the initial covariance matrix.

std::vector<std::vector<float>> _samplePopulation¶: [Internal Use] Sample coordinate information.

size_t _finishedSampleCount¶: [Internal Use] Counter of evaluated samples to terminate evaluation.

float _currentBestValue¶: [Internal Use] Best model evaluation from current generation.

std::vector<float> _currentBestVariables¶: [Internal Use] Best variables of current generation.

std::vector<float> _bestEverVariables¶: [Internal Use] Best ever found variables.

float _previousBestValue¶: [Internal Use] Best model evaluation from previous generation.

size_t _bestSampleIndex¶: [Internal Use] Index of the best sample in current generation.

float _bestEverValue¶: [Internal Use] Best ever model evaluation.

float _previousBestEverValue¶: [Internal Use] Best ever model evaluation as of previous generation.

std::vector<size_t> _sortingIndex¶: [Internal Use] Sorted indeces of samples according to their model evaluation.

std::vector<float> _covarianceMatrix¶: [Internal Use] (Unscaled) covariance Matrix of proposal distribution.

std::vector<float> _auxiliarCovarianceMatrix¶: [Internal Use] Temporary Storage for Covariance Matrix.

std::vector<float> _covarianceEigenvectorMatrix¶: [Internal Use] Matrix with eigenvectors in columns.

std::vector<float> _auxiliarCovarianceEigenvectorMatrix¶: [Internal Use] Temporary Storage for Matrix with eigenvectors in columns.

std::vector<float> _axisLengths¶: [Internal Use] Axis lengths (sqrt(Evals))

std::vector<float> _auxiliarAxisLengths¶: [Internal Use] Temporary storage for Axis lengths.

std::vector<float> _bDZMatrix¶: [Internal Use] Temporary storage.

std::vector<float> _auxiliarBDZMatrix¶: [Internal Use] Temporary storage.

std::vector<float> _currentMean¶: [Internal Use] Current mean of proposal distribution.

std::vector<float> _previousMean¶: [Internal Use] Previous mean of proposal distribution.

std::vector<float> _meanUpdate¶: [Internal Use] Update differential from previous to current mean.

std::vector<float> _evolutionPath¶: [Internal Use] Evolution path for Covariance Matrix update.

std::vector<float> _conjugateEvolutionPath¶: [Internal Use] Conjugate evolution path for sigma update.

float _conjugateEvolutionPathL2Norm¶: [Internal Use] L2 Norm of the conjugate evolution path.

size_t _infeasibleSampleCount¶: [Internal Use] Keeps count of the number of infeasible samples.

float _maximumDiagonalCovarianceMatrixElement¶: [Internal Use] Maximum diagonal element of the Covariance Matrix.

float _minimumDiagonalCovarianceMatrixElement¶: [Internal Use] Minimum diagonal element of the Covariance Matrix.

float _maximumCovarianceEigenvalue¶: [Internal Use] Maximum Covariance Matrix Eigenvalue.

float _minimumCovarianceEigenvalue¶: [Internal Use] Minimum Covariance Matrix Eigenvalue.

int _isEigensystemUpdated¶: [Internal Use] Flag determining if the covariance eigensystem is up to date.

float _covarianceMatrixAdaptionFactor¶: [Internal Use] This is the beta factor that indicates how fast the covariance matrix is adapted.

float _globalSuccessRate¶: [Internal Use] Estimated Global Success Rate, required for calibration of covariance matrix scaling factor updates.

size_t _covarianceMatrixAdaptationCount¶: [Internal Use] Number of Covariance Matrix Adaptations.

float _currentMinStandardDeviation¶: [Internal Use] Current minimum standard deviation of any variable.

float _currentMaxStandardDeviation¶: [Internal Use] Current maximum standard deviation of any variable.

size_t _maxInfeasibleResamplings¶: [Termination Criteria] Maximum number of resamplings per candidate per generation if sample is outside of Lower and Upper Bound.

float _maxConditionCovarianceMatrix¶: [Termination Criteria] Specifies the maximum condition of the covariance matrix.

float _minValue¶: [Termination Criteria] Specifies the minimum target fitness to stop minimization.

float _maxValue¶: [Termination Criteria] Specifies the maximum target fitness to stop maximization.

float _minValueDifferenceThreshold¶: [Termination Criteria] Specifies the minimum fitness differential between two consecutive generations before stopping execution.

float _targetMaxStandardDeviation¶: [Termination Criteria] Finishes execution when the maximum standard deviation falls below this threshold

std::vector<float> _minMeanUpdates¶: [Termination Criteria] Specifies the mininum update to the variable means before triggering termination.

size_t _maxGenerations¶: Stores how many generations to run for.

gsl_vector *_gsl_eval¶: Internal GSL storage.

gsl_matrix *_gsl_evec¶: Internal GSL storage.

gsl_eigen_symmv_workspace *_gsl_work¶: Internal GSL storage.

class korali::fGradientBasedOptimizer¶

#include <fGradientBasedOptimizer.hpp>

Class declaration for module: GRADIENT_BASED_OPTIMIZER.

Subclassed by korali::fAdagrad, korali::fAdam, korali::fMadGrad, korali::fRMSProp

Public Functions

virtual ~fGradientBasedOptimizer() = default¶: Default destructor to avoid warnings.

virtual bool checkTermination() = 0¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) = 0¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void reset() = 0¶: Restores the optimizer to the initial state.

virtual void printInfo() = 0¶: Prints progress information.

Public Members

size_t _nVars¶: Number of problem variables.

float _eta¶: Learning Rate.

float _decay¶: Decay for gradient update.

size_t _currentGeneration¶: Counter for the current generation.

std::vector<float> _initialValues¶: Initial values for the variables.

size_t _maxGenerations¶: Indicates how many generations to run.

size_t _modelEvaluationCount¶: Keeps track of how many model evaluations performed.

std::vector<float> _currentValue¶: [Internal Use] Current value of parameters.

float _currentEvaluation¶: [Internal Use] Function evaluation for the current parameters.

float _bestEvaluation¶: [Internal Use] Smaller function evaluation

std::vector<float> _gradient¶: [Internal Use] Gradient of Function with respect to Parameters.

class korali::fMadGrad : public korali::fGradientBasedOptimizer ¶

#include <fMadGrad.hpp>

Class declaration for module: MADGRAD.

Public Functions

fMadGrad(size_t nVars)¶

Default constructor for the optimizer.

Parameters: nVars – Variable-space dimensionality

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) override¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override¶: Restores the optimizer to the initial state.

virtual void printInfo() override¶: Prints progress information.

Public Members

std::vector<float> _s¶: Gradient Component.

std::vector<float> _v¶: Squared Gradient Component.

std::vector<float> _z¶: Update rule.

float _epsilon¶: Safety addition on divide.

float _momentum¶: Update momentum.

struct korali::solver::sampler::fparam_s¶

#include <TMCMC.hpp>

Struct for TMCMC optimization operations.

Public Members

const double *loglike¶: Likelihood values in current generation.

size_t Ns¶: Population size of current generation.

double exponent¶: Annealing exponent of current generation.

double cov¶: Target coefficient of variation.

class korali::fRMSProp : public korali::fGradientBasedOptimizer ¶

#include <fRMSProp.hpp>

Class declaration for module: MADGRAD.

Public Functions

fRMSProp(size_t nVars)¶

Default constructor for the optimizer.

Parameters: nVars – Variable-space dimensionality

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void processResult(float evaluation, std::vector<float> &gradient) override¶

Takes a sample evaluation and its gradient and calculates the next set of parameters.

Parameters

evaluation – The value of the objective function at the current set of parameters
gradient – The gradient of the objective function at the current set of parameters

virtual void reset() override¶: Restores the optimizer to the initial state.

virtual void printInfo() override¶: Prints progress information.

Public Members

std::vector<float> _r¶: Square gradient contribution.

std::vector<float> _v¶: Update rule.

float _epsilon¶: Safety addition on divide.

float _decay¶: Decay of gradient contribution over time.

class korali::distribution::univariate::Gamma : public korali::distribution::Univariate ¶

#include <gamma.hpp>

Class declaration for module: Gamma.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual double *getPropertyPointer(const std::string &property) override¶

Retrieves the pointer of a conditional value of a distribution property.

Parameters: property – Name of the property to find.
Returns: The pointer to the property..

virtual void updateDistribution() override¶: Updates the parameters of the distribution based on conditional variables.

virtual double getDensity(const double x) const override¶

Gets the probability density of the distribution at point x.

Parameters: x – point to evaluate P(x)
Returns: Value of the probability density.

virtual double getLogDensity(double x) const override¶

Gets the Log probability density of the distribution at point x.

Parameters: x – point to evaluate log(P(x))
Returns: Log of probability density.

virtual double getLogDensityGradient(double x) const override¶

Gets the Gradient of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate grad(log(P(x)))
Returns: Gradient of log of probability density.

virtual double getLogDensityHessian(double x) const override¶

Gets the second derivative of the log probability density of the distribution wrt. to x.

Parameters: x – point to evaluate H(log(P(x)))
Returns: Hessian of log of probability density.

virtual double getRandomNumber() override¶

Draws and returns a random number from the distribution.

Returns: Random real number.

Public Members

double _shape¶: [Conditional Variable Value]

std::string _shapeConditional¶: [Conditional Variable Reference]

double _scale¶: [Conditional Variable Value]

std::string _scaleConditional¶: [Conditional Variable Reference]

class korali::solver::learner::GaussianProcess : public korali::solver::Learner ¶

#include <gaussianProcess.hpp>

Class declaration for module: GaussianProcess.

Public Functions

virtual bool checkTermination() override¶

Determines whether the module can trigger termination of an experiment run.

Returns: True, if it should trigger termination; false, otherwise.

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.

virtual void applyModuleDefaults(knlohmann::json &js) override¶

Applies the module’s default configuration upon its creation.

Parameters: js – JSON object containing user configuration. The defaults will not override any currently defined settings.

virtual void applyVariableDefaults() override¶: Applies the module’s default variable configuration to each variable in the Experiment upon creation.

virtual std::vector<std::vector<float>> &getEvaluation(const std::vector<std::vector<std::vector<float>>> &input) override¶

For learner modules which have been trained, test returns an inferred output batch, from a batch of inputs to process.

Parameters: input – The inputs from which to infer outputs. Format: BxTxIC (B: Batch Size, T: Time steps, IC: Input channels)
Returns: The inferred batch outputs for the last given timestep. Format: BxOC (B: Batch Size, OC: Output channels)

virtual std::vector<float> getHyperparameters() override¶

Returns the hyperparameters required to continue training in the future.

Returns: The hyperparameters

virtual void setHyperparameters(const std::vector<float> &hyperparameters) override¶

Sets the hyperparameters required to continue training from a previous state.

Parameters: hyperparameters – The hyperparameters to use

virtual void initialize() override¶: Initializes Module upon creation. May allocate memory, set initial states, and initialize external code.

virtual void runGeneration() override¶: Runs the current generation.

virtual void printGenerationAfter() override¶: Prints solver information after the execution of the current generation.

Public Members

std::string _covarianceFunction¶: Covariance function for the libgp library.

float _defaultHyperparameter¶: Default value of the hyperparameters, used to initialize the Gaussian Processes.

knlohmann::json _optimizer¶: Represents the state and configuration of the optimization algorithm.

size_t _gpInputDimension¶: [Internal Use] Dimension of the input space.

size_t _gpParameterDimension¶: [Internal Use] Number of the Gaussian Process’ parameters.

std::vector<float> _gpHyperparameters¶: [Internal Use] Gaussian Process’ hyperparameters.

int _terminateWithOptimizer¶: [Termination Criteria] Execution will end as soon as the internal optimizer reaches one of its termination criteria.

std::vector<std::vector<float>> _outputValues¶: Storage for the output values.

problem::SupervisedLearning *_problem¶: Korali engine for optimizing NN weights and biases.

std::unique_ptr<libgp::GaussianProcess> _gp¶: Pointer to the gaussian processes library.

Experiment _koraliExperiment¶: Korali experiment for optimizing the GP’s parameters.

class korali::distribution::univariate::Geometric : public korali::distribution::Univariate ¶

#include <geometric.hpp>

Class declaration for module: Geometric.

Public Functions

virtual void getConfiguration(knlohmann::json &js) override¶

Obtains the entire current state and configuration of the module.

Parameters: js – JSON object onto which to save the serialized state of the module.

virtual void setConfiguration(knlohmann::json &js) override¶

Sets the entire state and configuration of the module, given a JSON object.

Parameters: js – JSON object from which to deserialize the state of the module.