This AI Paper Reveals a New Approach to Understand Deep Learning Models: Unpacking the ‘Where’ and ‘What’ with Concept Relevance Propagation (CRP)

The field of Machine Learning and Artificial Intelligence has become very important. We have new advancements that have been there with each day. The area is impacting all spheres. By utilizing finely developed neural network architectures, we have models that are distinguished by extraordinary accuracy within their respective sectors.

Despite their accurate performance, we must still fully understand how these neural networks function. We must know the mechanisms governing attribute selection and prediction inside these models to observe and interpret results.

The intricate and nonlinear nature of deep neural networks (DNNs) often leads to conclusions that may exhibit bias towards undesired or undesirable traits. The inherent opacity of their reasoning poses a challenge, making it challenging to apply machine learning models across various relevant application domains. It isn’t easy to understand how an AI system makes its decisions.

Consequently, Prof. Thomas Wiegand (Fraunhofer HHI, BIFOLD), Prof. Wojciech Samek (Fraunhofer HHI, BIFOLD), and Dr. Sebastian Lapuschkin (Fraunhofer HHI) introduced the concept of relevance propagation (CRP) in their paper. This innovative method offers a pathway from attribution maps to human-understandable explanations, allowing for the elucidation of individual AI decisions through concepts understandable to humans.

They highlight CRP as an advanced explanatory method for deep neural networks to complement and enrich existing explanatory models. By integrating local and global perspectives, CRP addresses the ‘where’ and ‘what’ questions about individual predictions. The AI ideas CRP uses, their spatial representation in the input, and the individual neural network segments responsible for their consideration are all revealed by CRP, in addition to the relevant input variables impacting the choice.

As a result, CRP describes decisions made by AI in terms that people can comprehend. 

The researchers emphasize that this approach of explainability examines an AI’s full prediction process from input to output. The research group has already created techniques for using heat maps to demonstrate how AI algorithms make judgments.

Dr. Sebastian Lapuschkin, head of the research group Explainable Artificial Intelligence at Fraunhofer HHI, explains the new technique in more detail. He said that CRP transfers the explanation from the input space, where the image with all its pixels is located, to the semantically enriched concept space formed by higher neural network layers. 

The researchers further said that the next phase of AI explainability, known as CRP, opens up a world of new opportunities for researching, evaluating, and enhancing the performance of AI models.

Insights into the representation and composition of ideas within the model and a quantitative evaluation of their influence on predictions can be acquired by exploring model designs and application domains using CRP-based studies. These investigations leverage the power of CRP to delve into the intricate layers of the model, unraveling the conceptual landscape and assessing the quantitative impact of various ideas on predictive outcomes. 

Check out the Paper. All Credit For This Research Goes To the Researchers on This Project. Also, don’t forget to join our 31k+ ML SubReddit, 40k+ Facebook Community, Discord Channel, and Email Newsletter, where we share the latest AI research news, cool AI projects, and more.

If you like our work, you will love our newsletter..