Learning bipartite Bayesian networks under monotonicity restrictions

ABSTRACT

Learning parameters of a probabilistic model is a necessary step in machine learning tasks. We present a method to improve learning from small datasets by using monotonicity conditions. Monotonicity simplifies the learning and it is often required by users. We present an algorithm for Bayesian Networks parameter learning. The algorithm and monotonicity conditions are described, and it is shown that with the monotonicity conditions we can better fit underlying data. Our algorithm is tested on artificial and empiric datasets. We use different methods satisfying monotonicity conditions: the proposed gradient descent, isotonic regression EM, and non-linear optimization. We also provide results of unrestricted EM and gradient descent methods. Learned models are compared with respect to their ability to fit data in terms of log-likelihood and their fit of parameters of the generating model. Our proposed method outperforms other methods for small sets, and provides better or comparable results for larger sets.

KEYWORDS:

• Bayesian networks
• monotonicity
• parameter learning
• isotonic regression
• gradient method
• computerized adaptive testing

Disclosure statement

No potential conflict of interest was reported by the authors.

ORCID

Martin Plajner  http://orcid.org/0000-0001-8388-1832

Jiří Vomlel http://orcid.org/0000-0001-5810-4038

Notes

1 In our case, points are specifying the score obtained in the question i. The interpretation of points is very complex and has to be viewed as per question because we use the CAT framework. In this context, getting one point in one question is not the same as one point in another.

2 Note that for $n_i$ this formula always holds since $\sum _{t=0}^{n_i}P(X_i=t|S_j=s_{j,k},\mathit{s})=1\mathrm{\forall }i,\mathrm{\forall }j,\mathrm{\forall }k$.

3 As we use only reparameterized parameters in our gradient method, we provide only formulas with the reparametrization, i.e. the parameter vector $\mathit{\mu }$ as was introduced in Section 2.2

4 The reason to include regular EM and unrestricted gradient methods is to further verify the benefit of using the monotonicity constraints. We want to provide a reader with a comparison also between the restricted and unrestricted cases.

5 The test assignment and its solution are accessible in the Czech language at: http://www.statnimaturita-matika.cz/wp-content/uploads/matematika-test-zadani-maturita-2015-jaro.pdf.

Additional information

Funding

This work was supported by the Czech Science Foundation (Project No. 16-12010S and Project No. 19-04579S) and by the Grant Agency of the Czech Technical University in Prague [grant number SGS17/198/OHK4/3T/14].

Notes on contributors

Martin Plajner

Martin Plajner is a PhD student in Mathematical Engineering at the Czech Technical University in Prague, Faculty of Nuclear Sciences and Physical Engineering since 2013. He cooperates with the Institute of Information Theory and Automation of the Czech Academy of Sciences as a research PhD student under the supervision of Jiří Vomlel. His research interests are mainly centered on Computerized Adaptive Testing, Probabilistic Methods in Artificial Intelligence, and Bayesian Networks. He published research papers discussing these topics at various international conferences such as Conference on Probabilistic Graphical Models (PGM).

Jiří Vomlel

Jiří Vomlel obtained his PhD degree in Artificial Intelligence from the Czech Technical University in Prague. He spent three years at Aalborg University in Denmark where he worked as a research assistant in the research unit of Decision Support Systems. Since 2002, he has been a researcher in the Institute of Information Theory and Automation of the Czech Academy of Sciences. His researcher interests are Probabilistic Methods in Artificial Intelligence, Bayesian Networks, Computerized Adaptive Testing, Decision Theoretic Troubleshooting, and other applications of probabilistic graphical models. He has published about 50 research papers in scientific journals and peer-reviewed conference proceedings. He serves as an Area Editor of the International Journal of Approximate Reasoning and as regular program committee member of the Conference on Uncertainty in Artificial Intelligence (UAI), the International Joint Conference on Artificial Intelligence (IJCAI), and the International Conference on Probabilistic Graphical Models (PGM).