Self-Generating Plant Digital Twins Using Reinforcement Learning and Large Language Models

Overview

This research develops self-generating plant digital twins by integrating reinforcement learning (RL) with large language models (LLMs) and data-driven modeling techniques. The goal is to automatically construct knowledge graphs and governing equations for a wide variety of plants with minimal human supervision. An RL agent is trained to build these models by issuing queries to an LLM, allowing the system to adapt dynamically as plant conditions change. A second RL capability is being explored to review generated graphs, identify gaps, and formulate targeted questions for the LLM to fill in missing information.

The approach addresses two key challenges: generating accurate graph structures without requiring large labeled training datasets, and verifying that the information embedded in the graphs is correct. By automating digital twin creation, this technology can accelerate process modeling for engineering teams, reduce manual effort, and support faster decision-making in plant operations and design.

Technical specifications

Key features:

• Reinforcement learning agent that selects graph structures and content to build knowledge graphs for plant modeling
• Integration with large language models to generate descriptive text and equations through prompt-based queries
• Baseline knowledge graph development without RL to establish reference structures
• Reward metrics and exploration policies designed to guide the RL agent toward accurate graph generation
• Automated prompt engineering for LLM interactions to populate graph components
• Distributed optimization techniques to validate correctness and build consensus across graph components for new plants
• Generalization strategy enabling the RL agent, trained on multiple plants, to generate graphs for previously unseen plants

Technology readiness level

This technology is at an early research stage. Initial work involves formulating the RL problem and establishing baseline knowledge graphs for individual plants. Reward functions and exploration policies are being designed to train the agent. Future validation will test the agent's ability to generalize across plant types using distributed optimization to ensure model accuracy. The research is currently at a concept-to-proof stage and is not yet ready for commercial deployment.