Explainable Object Detection Using Prototypical Features

Pavlos Rath-Manakidis, M.Sc.
Lead Researcher

Henry Huick
Research Assistant

Prof. Dr. Laurenz Wiskott
Supervisor

Theory of Neural Systems

Funding:

By the German Federal Ministry of Education and Research (BMBF) within the Future of Value Creation-Research on Production, Services and Work program.

Making AI Vision Understandable

This research project, ProtoP-OD, aims to make object detection models, a main type of artificial intelligence (AI) vision systems, more transparent and interpretable. Current AI models often act like "black boxes," making it difficult to understand how they arrive at a decision, which can be problematic in critical applications like medical imaging or autonomous systems where trust and reliability are paramount. ProtoP-OD tackles this by identifying objects using "prototypical parts"—small, understandable local features that are aligned with specific object classes. For instance, instead of just detecting a "person," the model might first identify prototypical parts like a "head," "torso," or "legs" and then combine this information. This process allows for a visual inspection of what the model is "seeing" and focusing on, making its decision-making process more intuitive and explainable.

The core innovation is a "prototype neck" module that is integrated into existing detection transformer architectures. This module forces the AI to represent image content through a sparse set of these learned prototypes before making a detection. The system is trained with a special "alignment loss" to ensure these prototypes meaningfully correspond to the object classes they are supposed to represent. The result is an AI that not only detects objects but also provides a clearer explanation of why it detected them by showing which prototypical parts were activated. This inherent interpretability helps users better understand the AI's strengths and weaknesses, calibrate their trust in its outputs, and potentially interact with or even edit the model's understanding. While this approach may involve a slight trade-off in detection performance, the significant gain in explanatory power is crucial for fostering reliable and effective human-AI collaboration.

Pre-Print

Publications