Explainable inference for multi-modal foundation models in anomaly detection
An advanced system for anomaly detection using multi-modal data, integrating few-shot learning and transductive inference to explain anomalies in food safety contexts.
Overview
This innovative research focuses on developing an explainable inference system for anomaly detection using multi-modal foundation models. The technology leverages diverse data modalities such as RGBD, hyper-spectral, and NIR images to identify anomalies with few-shot learning techniques. By contextualizing anomalies within the broader dataset, the system provides actionable insights particularly useful in ensuring food safety.
Technical specifications
- Utilizes multi-modal data inputs to enhance anomaly detection capabilities.
- Employs state-of-the-art few-shot learning and anomaly detection methods for high-dimensional datasets.
- Integrates self-supervised learning models with self-attention-based networks and modality-specific augmentations.
- Capable of reconstructing missing modalities by leveraging remaining data inputs.
- Develops vision-language models to contextualize and explain anomalies through transductive inference and textual descriptions.
Technology readiness level
Currently at TRL 2, this technology is in the early stages of development. It involves creating prototype models and conducting initial testing with multi-modal datasets in collaboration with partners. Further validation and development are planned to improve detection rates and application in food safety scenarios.
About University of Pennsylvania
The University of Pennsylvania is a private, comprehensive research university with a large, interdisciplinary research enterprise on an urban campus in Philadelphia. For industry, the university combines an integrated academic health system with a research and technology park featuring co‑located labs, pilot and prototyping spaces, and access to shared instrumentation. Dedicated corporate engagement and technology transfer teams streamline sponsored research, IP, and startup formation, with incubator and venture resources to accelerate commercialization. Faculty attract substantial competitive federal funding from agencies such as the National Institutes of Health, National Science Foundation, Department of Energy, and the Department of Defense.
delivered to your inbox.