Physics-informed AI simulations, such as physics-informed neural networks (PINNs), are beginning to replace artificial neural network models (ANNs), which are regarded as black box models. Physics-informed models yield more accurate and more trustworthy predictions than ANN simulations.
Both models are data driven, but ANNs require vast quantities of operational data, said Herman Van der Auweraer, director of technology innovation at Siemens Digital Industries Software.
Artificial neural network models also are very complex and can take a lot of time to acquire a benchmark of data, said Robert X. Gao, professor and chair of the department of mechanical and aerospace engineering at Case Western Reserve University.
“The lack of transparency and interpretability of AI/machine learning [ML] models has been well recognized as a bottleneck for the widespread adoption of AI/ML in manufacturing. Inherently, manufacturing is about applying physical principles and laws to process materials into useful products for industrial or commercial applications,” he said.
Although some people may describe neural networks as AI, neural networks are simply a very smart way to predict between known data points (interpolation), said Peter Mas, director at engineering services at Siemens Software. Using ANN, accurately predicting values for points outside the range of data (extrapolation) is not possible, he said.
For example, events like wind turbine faults that occur rarely are not captured by the data and could lead to erroneous representation, Van der Auweraer said.
“No neural network technique can predict in areas where it has never been before,” Mas said. “To do that, you need to include the physics law that will tell you what trend the extrapolation will have to follow.”
If a black box model has too little data, it won’t capture the proper behavior of the system being modeled, said Bijan Sayyar Rodsari, director of advanced analytics at Rockwell Automation.
But simply throwing more data at a black box system is not always the answer.
If not protected against over-training, such a system latches onto every anomaly in the data and fails to make accurate predictions, he said. Regarding anomalies, often a manufacturer doesn’t get the chance to collect an extensive amount of data because the machine is shut down as soon as an anomaly is discovered, he added.
Because of the lack of data and insight, manufacturers using black box systems won’t push machines all the way to the sweet spot for maximum efficiency, choosing instead to err well within the safety margin, Rodsari said.
“Your purely data-driven model is bound to be constrained by the data you put in,” Rodsari said.
Nor will they be willing to run closed-loop systems, he said. “If the operator does not understand what the model is doing, they are always going to be skeptical about allowing this model to run in a closed loop.”
The Russian proverb, “Trust but verify,” simply doesn’t work with black box models. Verifying how the modeling system came up with its predictions is nearly impossible. Without the ability to verify inputs and outputs, many operators in manufacturing do not trust the models—and with good reason because the results may not be accurate.
The only way to know for sure if the initial processing was correct is by trial and error, Rodsari said.
Here’s a simple example from Mas: If an advanced neural network model has data showing that four apples weigh 1 kilogram and eight apples weigh 2.1 kg, the model can likely correctly predict the weight of six apples because six falls within its known data points and is part of the linear trend. But if queried on the possible weight of 12 apples (a number outside its data points) the ANN model will give an answer but potentially 5 kg, or another off-base number, instead of the more reasonable prediction of 3-3.3 kg.
“These black box models have proven to be difficult to interpret,” Rodsari said. “While good for creating a match for variables of interest, they’re not helpful for explaining that relationship. That hinders the ability of people who have to put these models to use and make a judgment as to whether results are correct or not. For most applications in the manufacturing space, you need the ability to convince operations that this relationship is meaningful, help them to have a means of monitoring the quality, and prevent decisions that are going to be damaging to the operation.
“You need the ability to provide some visibility into the nature of the model so the operator can trust it.”
Meanwhile, physics-informed AI simulations can make predictions based on significantly less data because they use data that’s higher-quality and more relevant to the machine and the problem at hand.
As the name implies, physics-informed AI incorporates relevant data, physical laws, and prior knowledge, such as performance parameters and norms from the machine being modeled, Gao said.
“The laws of physics contain an enormous wealth of information in a very condensed way,” Van der Auweraer said. “Physics-based simulation methods are using these physics laws. So, they can give an extremely powerful head start to AI systems by bringing in this knowledge instead of having to wait until enough representative data comes along.”
With that encoded physical knowledge, physics-informed AI models can make predictions based on less data, Rodsari said.
“We want to build models that are helpful to manufacturers in real time,” he said.
Physics-informed AI models allow AI to learn from data in process, emulating a brain learning, and can improve as more data becomes available, Mas said.
Manufacturing engineers can then modify and tailor their existing structures and systems to make the model work for their factory.
“When you observe data, there are typically relationships that you know, especially for an engineer,” Mas said. “There is known physics and unknown physics. This is how physics-informed AI works. Instead of just treating the full data relations as a black box, you can also impose some physical equations on the data like ‘conservation of energy’ or more complex things, such as ‘wave-like behavior,’ so that the machine learning algorithm will balance between the data and the physics. This is typically done through a loss function, which is the target function for the algorithm to minimize error with data while satisfying the physics.”
The first PINN applications are emerging in manufacturing processes with complex models and relations, such as in additive manufacturing, Van der Auweraer said.
Other early adopters will be in the food industry or pharmaceutical processing industry where complex processes may hinder a pure simulation-based approach and where the AI in a PINN approach may yield promising results, Van der Auweraer and Mas said.
PINN models also can complement or replace labor-intensive lab testing and design, Mas said, combining the existing strengths of lab testing and the benefits of physics-based simulations to accurately design new material and products in much less time using less lab testing.
The biggest challenge is that machine learning is being done today by data scientists who write scripts in their preferred data science language, Mas said, as opposed to available and accessible to the engineer who develops a product through application-oriented, low-code platforms like the ROM Building application that Siemens is currently developing within its Simcenter portfolio.
“You can start with the generic structure,” Gao said. “The innovation is up to the engineers to creatively modify and adapt.”
The most powerful approach may be combining the physics relationships inside the artificial neural network, complementing that network or as a specific layer or structure within the neural network, Van der Auweraer said.
That would transform the black box to a gray box.
“Such a network could start to be trained from high-quality simulations. It can contain internal physics relations for a more condensed and powerful network that will also be trained faster and can ultimately be further trained by any data that may come along during its lifecycle, he said.
Connect With Us
