Algorithms
Introduction
Inference is the process of querying one or more nodes/variables in Bayes Server, also known as making a prediction, or calculating the posterior probability.
Bayes Server includes a number of different inference algorithms which are described here.
As well as querying the probabilities of nodes/variables in a network (making predictions), inference may also be used during Parameter learning. This is why the Parameter learning window allows the inference algorithm to be changed.
Exact and approximate inference
The inference algorithms found in Bayes Server are categorized into exact and approximate inference algorithms. Exact algorithms will calculate exact probabilities (subject to normal computer rounding errors), whereas an approximate algorithm will attempt to calculate the correct answer.
Whenever possible, an exact algorithm should be used for parameter learning, even if an approximate algorithm is required to perform subsequent queries on the network. For example, some complex time series models can be trained using exact inference, but may require approximate inference for long range predictions.
Deterministic and non-deterministic
Algorithms are further categorized into deterministic and non-deterministic algorithms. Deterministic algorithms will return the same result each time, whereas non-deterministic algorithms may return slightly different results each time (e.g. due to random sampling).
Algorithms
| Algorithm | Exact / approximate | Deterministic / Non-deterministic | Description |
|---|---|---|---|
| Relevance tree | Exact | Deterministic | An efficient exact inference algorithm with numerous optimizations including those for querying many variables and time series predictions. |
| Variable elimination | Exact | Deterministic | An exact algorithm, loosely based on the well known Variable elimination algorithm, although with numerous optimizations. May not perform as well as the Relevance tree algorithm when querying large numbers of nodes/variables or temporal queries. |
| Loopy belief propagation | Approximate | Deterministic | An approximate inference algorithm which is not based on random sampling, so will return the same result each time. Unlike an exact algorithm, there is no guarantee that it will return the correct result. |
| Likelihood sampling | Approximate | Non-deterministic | An approximate inference algorithm based on random sampling. The more samples used, the better the accuracy. Results may differ each time due to its random nature. |
Hybrid inference
If using the Bayes Server API, you can mix exact and approximate inference if required. For example you can query a subset of nodes with exact inference, and others with approximate inference.
With some types of complex time series / sequence models (Dynamic Bayesian networks) it may be possible to use exact inference for parameter learning, but require approximate inference for long range predictions. In this case, an example of hybrid inference would be to use exact inference to predict the first 5 time steps into the future, and then approximate inference for 5 - 100 time steps into the future.
The image below shows a time series model that was learned using exact inference but requires approximate inference for long range predictions.
