Parameter learning

Suppose we are building a Bayesian network but do not have a good way to generate the probabilities between the different variables. This is where parameter learning comes into play. Parameter learning, or just "learning", allows us to present the network with data that we observe in the real world, sometimes called evidence, and let the network learn the probabilities from the data through Bayesian inference.

In essence, parameter learning in Genius counts the combinations of times that certain events coincide with one another. For example, if nine out of ten times you observe that when the grass is wet it is rainy and the sprinkler is on, the associated probability for this entry would be $0.9$. By presenting the model with rows of data for each variable that we gather from the real world, probabilities can be populated in the network automatically.

Parameter learning in Genius, whether performed in the model editor or the Python SDK, requires an appropriate CSV file containing the data. Please see the section on the CSV data format for more information about the structure of a data file.

Although parameter learning may be used to fill in the probabilities in a network, be aware that these probabilities will always reflect biases in the data. If the data collected does not represent the true probability distributions being modeled then queries run on the model will not be accurate. It is always a good idea to inspect probabilities in networks and see if they make sense given some domain knowledge. The more, representative data that is available, the more accurate the learned probabilities will be.

Types of learning

There are three types of parameter learning paradigms supported in Genius:

• De novo learning - This form of learning entails updating a static discrete Bayesian network as new data becomes available without considering prior data or evidence. Since past probabilities are not taken into account in the calculating of probabilities this is equivalent to creating an entirely new model with a new set of data. This was the form of learning that was available in the Curie release.

• Continuous learning - This form of learning entails updating a static discrete Bayesian network as new data becomes available while taking into account prior probabilities calculated from previous data. This means that the Bayesian network keeps a memory of past probabilities when new data is presented.

• Online learning - This form of learning is equivalent to continuous learning in the case where we use a dynamic discrete Bayesian network or POMDP. In this case, data arrives at specific time points and it is used to calculate the state of a system at a particular time point. However, past probabilities are still taken into account which makes online learning a form of continuous learning with time-dependent variables.

Parameter learning is still possible when there are latent variables in the model but it requires some careful treatment. Genius uses the expectation-maximization (EM) algorithm for learning in the presence of latent variables which alternates repeatedly between two steps:

• Expectation: Infer the expected probability distribution for latent variables given currently estimated parameters.

• Maximization: Learn the most likely value (maximum probability) of the parameters.

As these steps alternate, the algorithm can hone in on an estimation of the parameters based on the available data while in the presence of latent variables.

Parameter learning in active inference models

It is possible to perform parameter learning in active inference models. As explained in the knowledge center article on partially observable Markov decision processes (POMDPs), an active inference model is built from an initial state prior, likelihood, state-transition, and prior preference. Each of these random variables are parameterized by categorical probability distributions. If these probabilities are known in advance they can be set when creating the model. If these probabilities are unknown they can be learned from some input data using online learning.

In active inference models, parameter learning is generally expected to take place on a slower time scale than state inference. That is, while the state of the environment is rapidly changing from time point to time point, parameter learning is much slower and probabilities accumulate over time.

In active inference models, parameters are learned using Dirichlet counts. Technically speaking, this means that priors on the parameters in the model are Dirichlet distributions representing the parameter space. Dirichlet distributions use pseudocounts which accumulate over time and the expected value of the resulting Dirichlet distribution is used to update the parameters.