Structural learning

Structural learning is the process of using data to learn the links of a Bayesian network or Dynamic Bayesian network.

Structural learning

Bayes Server supports the following algorithms for structural learning:

NOTE

You can chain algorithms together (e.g. Search & Score + Clustering). Simply finish the structural learning wizard multiple times, each time importing link constraints from a previous run.

Clustering

(since version 8.4)

Automatically determines an appropriate discrete latent variable to add to the model, which can model hidden patterns. Similar to a cluster model or a mixture model. Supports networks with discrete and/or continuous variables, as well as missing data.

NOTE

See our article on mixture models & clustering for more information.

This is an unsupervised algorithm, and in its most basic form is similar to a mixture model (a.k.a. cluster model). However this algorithm is more flexible, as an arbitrary structure can be mixed, and include both discrete and continuous variables. The appropriate number of clusters is automatically determined by the algorithm robustly using cross validation.

PC algorithm

A constraint based algorithm, which uses marginal and conditional independence tests to determine the structure of the network.

This algorithm supports the following:

NOTE

The algorithm does not currently support the automatic detection of latent variables.

References:

Significance level

The significance level is used during tests for independence and conditional independence by the algorithm to determine whether links should exist or not. A value of 0.05 or 0.01 is typical.

Max conditional

This controls the number of conditional nodes that are considered when the algorithm performs conditional independence tests.

A value of 3 means that when testing whether X and Y are independent, up to 3 other conditional variables will be considered in the tests.

Max temporal

This is the maximum order that should be considered for temporal links in dynamic Bayesian networks.

A value of 3 means that temporal nodes should not have links greater than order 3, i.e. up to 3 previous time steps.

Search & score algorithm

(since version 7.14)

The Search & Score algorithm performs a search of possible Bayesian network structures, and scores each to determine the best.

This algorithm currently supports the following:

References:

Hierarchical

(since version 7.20)

This algorithm, which is unique to Bayes Server, creates a hierarchy of nodes, built layer by layer. Each layer consists of a number of groups. Each group consists of similar variables as determined by the algorithm. Individual groups are connected via a parent discrete latent variable. Therefore the algorithm generates multiple latent variables, and latent variables themselves can be subsequently clustered in groups.

This algorithm currently supports the following:

An example output from the algorithm is shown below.

Structural learning

Chow-Liu algorithm

(since version 7.12)

Creates a Bayesian network which is a tree. The tree is constructed from a weighted spanning tree over a fully connected graph whose connections are weighted by a metric such as Mutual Information.

This algorithm currently supports the following:

References:

Root

An optional argument which determines the root of the generated tree.

TAN algorithm

(since version 7.12)

The Tree Augmented Naive Bayes algorithm (TAN) builds a Bayesian network that is focused on a particular target T (e.g. for classification). The algorithm creates a Chow-Liu tree, but using tests that are conditional on T (e.g. conditional mutual information). Then additional links are added from T to each node in the Chow-Liu tree.

This algorithm currently supports the following:

References

See the Chow-Liu algorithm.

Root

An optional argument which determines the root of the generated tree.

Target

Determines the target for the learning algorithm. For example, the node which you are trying to predict.

Chaining algorithms

Chaining of algorithms provides extra flexibility. For example you could start with a Chow-Liu tree which can be constructed very quickly. You may then wish to use the Search & Score algorithm to augment the tree with some more complex structure.

You could also start with a Search & Score, then use the Clustering algorithm.

When chaining, be sure to:

Defining variables

In order to perform structural learning from a data set, you must have first defined your variables/nodes. If required, these can be automatically determined using the Add nodes from data feature.

NOTE

Not all variables have to be mapped to data columns. Any variables that are not mapped are ignored during the learning process.

If required link constraints can be defined before learning. The different types of link constraint are documented in BayesServer.Learning.Structure.LinkConstraintMethod in the Bayes Server API. The link constraint failure modes are documented in BayesServer.Learning.Structure.LinkConstraintFailureMode in the Bayes Server API.