Microsoft Researchers Propose Neural Graphical Models (NGMs): A New Type of Probabilistic Graphical Models (PGM) that Learns to Represent the Probability Function Over the Domain Using a Deep Neural Network

In the domain of reasoning under uncertainty, probabilistic graphical models (PGMs) have long been a prominent tool for data analysis. These models provide a structured framework for representing relationships between various features in a dataset and can learn underlying probability distributions that capture the functional dependencies between these features. Whether it’s learning from data, performing inference, or generating samples, graphical models offer valuable capabilities for exploring complex domains. However, they also come with limitations, often constrained by restrictions on variable types and the complexity of operations involved.

Traditional PGMs have proven effective in various domains but are flexible. Many graphical models are designed to work exclusively with continuous or categorical variables, limiting their applicability to data that spans different types. Moreover, specific restrictions, such as continuous variables not being allowed as parents of categorical variables in directed acyclic graphs (DAGs), can hinder their flexibility. Additionally, traditional graphical models may be limited in the types of probability distributions they can represent, often favoring multivariate Gaussian distributions.

Microsoft researchers propose a groundbreaking solution to these challenges in their recent “Neural Graphical Models” paper presented at the 17th European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU 2023). They introduce Neural Graphical Models (NGMs), a novel type of PGM that leverages deep neural networks to learn and efficiently represent probability functions over a domain. What sets NGMs apart is their ability to transcend the limitations commonly associated with traditional PGMs.

NGMs offer a versatile framework for modeling probability distributions without imposing constraints on variable types or distributions. This means they can handle various input data types, including categorical, continuous, images, and embeddings. Moreover, NGMs provide efficient solutions for inference and sampling, making them a powerful tool for probabilistic modeling.

The core idea behind NGMs is to utilize deep neural networks to parametrize probability functions over a given domain. This neural network can be trained efficiently by optimizing a loss function that simultaneously enforces adherence to the specified dependency structure (provided as an input graph, either directed or undirected) and fits the data. Unlike traditional PGMs, NGMs are not restricted by common constraints and can seamlessly handle diverse data types.

To delve deeper into NGMs, let’s explore their performance through experimental validations conducted on real and synthetic datasets:

1. Infant Mortality Data: The researchers used data from the Centers for Disease Control and Prevention (CDC), focusing on pregnancy and birth variables for live births in the U.S. The dataset also included information on infant mortality. Predicting infant mortality is challenging due to the rarity of such events. Nevertheless, NGMs demonstrated impressive inference accuracy when compared to other methods. They outperformed logistic regression and Bayesian networks and performed on par with Explainable Boosting Machines (EBM) for categorical and ordinal variables.
2. Synthetic Gaussian Graphical Model Data: In addition to real-world data, the researchers evaluated NGMs on synthetic data generated from Gaussian Graphical Models. NGMs showcased their capability to adapt to complex data structures and perform well in this synthetic environment.
3. Lung Cancer Data: Another dataset, sourced from Kaggle and related to lung cancer, was used to validate NGMs further. While the specific results on this dataset were not discussed in detail, it demonstrates the applicability of NGMs across various domains.

One remarkable feature of NGMs is their ability to handle situations where traditional models struggle, particularly in predicting low-probability events. For example, NGMs excel in predicting the cause of death among infants, even when it’s a rare occurrence. This highlights the robustness of NGMs and potential in domains where precision on infrequent outcomes is critical.

In conclusion, Neural Graphical Models (NGMs) significantly advance probabilistic graphical modeling. By combining the flexibility and expressiveness of deep neural networks with the structural advantages of graphical models, NGMs offer a powerful and versatile solution. They break free from the constraints imposed by traditional PGMs, allowing practitioners to work with a broader range of data types and distributions. With their demonstrated success in handling complex dependencies and accurately predicting rare events, NGMs hold great promise for addressing real-world challenges across diverse domains. Researchers and data scientists are encouraged to explore the capabilities of NGMs and leverage their potential to enhance probabilistic modeling efforts.

Check out the Paper and Reference Article. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 30k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..