Mixed Bayesian Networks: A Mixture of Gaussian Distributions

Abstract:

Mixed Bayesian networks are probabilistic models associated with a graphical representation, where the graph is directed and the random variables are discrete or continuous. We propose a comprehensive method for estimating the density functions of continuous variables, using a graph structure and a set of samples. The principle of the method is to learn the shape of densities from a sample of continuous variables. The densities are approximated by a mixture of Gaussian distributions. The estimation algorithm is a stochastic version of the Expectation Maximization algorithm (Stochastic EM algorithm). The inference algorithm corresponding to our model is a variant of junction three method, adapted to our specific case. The approach is illustrated by a simulated example from the domain of pharmacokinetics. Tests show that the true distributions seem sufficiently fitted for practical application.

Keywords:

Belief Networks - Gaussian Mixtures - SEM Algorithm - Statistical Tests

• REFERENCES

• 1 Pearl J. Probabilistic Reasoning in Intelligent Systems. . San Mateo: Morgan Kaufmann; 1988
• 2 Golmard JL. Probabilistic inference in artificial intelligence: the method of Bayesian networks. In: Dubucs JP. eds. Philosophy of Probability. . Deventer: Kluwer; 1993: 257-91.
• 3 Neapolitan RE. Probabilistic Reasoning in Expert Systems. . New York: Wiley; 1990
• 4 Henrion M. An introduction to algorithms for inference in belief nets. In: Henrion M, Shachter RD, Kanal LN, Lemmer JF. eds. Uncertainty in Artificial Intelligence 5. . Amsterdam: Elsevier; 1990: 129-38.
• 5 Spiegelhalter DJ, Dawid AP, Lauritzen SL, Cowell RG. Bayesian analysis in expert systems. Research Report 92-6. Cambridge: MRC Biostatistics Unit; 1992
• 6 Heckerman DE, Horvitz EJ, Nathwani BN. Toward normative expert systems: Part I. The Pathfinder project. Meth Inform Med 1992; 31: 90-105.
• 7 Dempster AD. Covariance selection. Biometrics 1972; 28: 157-75.
• 8 Loehlin JC. Latent Variable Models. Hillsdale NJ: Lawrence Erlbaum Ass; 1987
• 9 Lauritzen SL. Mixed graphical association models (with discussion). Scand J of Stat 1989; 16: 273-306.
• 10 Lauritzen SL. Propagation of probabilities, means and variances in mixed graphical association models. J Amer Stat Assoc 1992; 87: 1098-108.
• 11 Lauritzen SL, Wermuth N. Graphical models for associations between variables, some of which are qualitative and some quantitative. Ann Stat 1989; 17: 31-57.
• 12 Olesen KG. Causal probabilistic networks with both discrete and continuous variables. IEEE Trans Pattern Anal and Mach Intell 1993; 15: 275-9.
• 13 Thomas A, Spiegelhalter DJ, Gilks WR. BUGS: a program to perform Bayesian inference using Gibbs sampling. In: Bernardo JM, Berger JO, Dawid AP, Smith FM. eds. Bayesian Statistics 4. . Oxford: University Press; 1992: 837-12.
• 14 Gelfand AE, Smith AFM. Sampling-based approaches to calculating marginal densities. J Amer Stat Assoc 1990; 85: 398-109.
• 15 Lange N. Graphs and stochastic relaxation for hierarchical Bayes modelling. Stat in Med 1992; 11: 2001-16.
• 16 Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm (with discussion). J Roy Soc B 1977; 39: 1-38.
• 17 Celeux G, Diebolt J. The SEM algorithm: a probabilistic teacher algorithm derived from the EM algorithm for the mixture problem. Comput Statist Quart 1985; 02: 73-82.
• 18 Lauritzen SL, Spiegelhalter DJ. Local computations with probabilities on graphical structures and their application to expert systems (with discussion). J Roy Statist Soc B 1988; 50: 157-224.
• 19 Jensen FV, Olesen KG, Andersen SK. An algebra of Bayesian belief universes for knowledge-based systems. Networks 1990; 20: 637-59.
• 20 Steimer JL, Mallet A, Golmard JL, Boisvieux JF. Alternative approaches to estimation of population pharmacokinetic parameters: Comparison with the nonlinear mixed-effect model. In: Drug Metabolism Reviews 15. . New York: Marcel Dekker Inc; 1984: 130-62.