Draft:Physics-Enhanced Machine Learning

Physics-Enhanced Machine Learning

Physics-Enhanced Machine Learning (PEML) is a subfield of machine learning that combines physics-based modelling and data-driven algorithms, and aims to combine the complimentary strengths of physical knowledge and machine learning algorithms to overcome the limitations of using either approach alone. PEML incorporates known physical laws and domain knowledge into the machine learning process so that models not only fit observed data patterns, but also respect the governing physics equations of the model.

This concept is closely related to scientific machine learning (SciML), Physics-Informed Machine Learning (PIML), Physics-Enhanced Artificial Intelligence (PEAI) and Physics-Guided Machine Learning, but is distinct in its emphasis on enhancing predictive capability through the use of physics-based components in hybrid or constrained models.

Background and Motivation

The motivation for PEML arose from challenges often encountered in engineering and real-world problems: The volume of useful data is generally limited; predictions obtained from modelling can make accurate, but physically implausible predictions that violate known physics; uncertainties could not easily be dealt with; and traditional machine learning models are not very explainable or interpretable. By infusing physics into machine learning, it ensures better generalisation to unseen conditions, improved physical consistency of predictions, and a greater explainability of the learnt models. This approach has gained traction since 2019, where techniques have been used to encompass strategies where "prior physics knowledge is embedded to the learner" , particularly in dealing with dynamical systems in engineering.

Early examples of the need for PEML arose from fields such as structural mechanics and environmental science, where purely data-driven methods struggled with limited data or lacked reliability. For instance, in structural engineering, traditional physics-based simulations can be very accurate but often require costly modelling and still face uncertainty in loads or material properties; on the other hand, data-driven models may fit experimental data but fail to generalise outside those conditions. PEML approaches were developed to bridge this gap, effectively creating a "spectrum" between the extremes of purely physics-based (white-box modelling) and purely data-driven (black-box modelling), known as grey-box or hybrid modelling. In practice, this means a PEML model can leverage governing equations or simulation data to inform the learning process, thus requiring less training data and yielding outputs that obey physical laws.

Methods and Techniques

PEML encompasses a range of methods that integrate domain-specific physical knowledge into the machine learning process. These techniques differ in how physics is incorporated, whether through loss functions, model structures, feature design, or data generation, and can be summarised into three different categories: Physics-Informed, Physics-Guided, and Physics-Encoded Machine Learning.

Physics-Informed Learning

See also: Physics-informed neural networks

Physics-Informed learning techniques integrate physical laws directly into the machine learning process to simulate complex systems using partial differential equations (PDEs) and embedding physical constraints directly into a machine learning algorithm, such as a neural network. An example of physics-informed learning is through the use of Physics-Informed Neural Networks (PINNs), which implement composite loss functions that balance errors with PDE residuals, effectively blending sparse observations with physical constraints. They are most suitable for irregular geometries due to their ability to operate in a meshless paradigm by sampling random collocation points. Physics-informed learning excels in multi-physics scenarios such as electroconvection, molecular dynamics, and real-time 4D flow reconstruction from MRI observations.

Physics-Guided Learning

In many PEML approaches, physical knowledge is introduced into the learning process not by altering the model itself, but through data preprocessing and feature engineering. This strategy enables conventional machine learning algorithms to work with inputs that already encode important physical structure, enhancing both accuracy and interpretability. Common techniques include:

1. Physics-based feature extraction. Raw data is transformed into features with physical meaning, such as dimensionless numbers (e.g. Reynolds number or Mach number), wavelet coefficients, or energy spectra. For example, Mohan et al. used a wavelet transform to extract turbulence-related features from velocity fields, embedding known physics of turbulent cascades into the model inputs.
2. Simulation or theory-driven feature augmentation. Outputs of simplified physical models (or residuals between observed and predicted behaviour) are used as additional features, reducing the learning burden of the machine learning model by letting it focus only on the discrepancy. This technique has been used in chemical kinetics applications, where delta learning was applied to graph neural networks (GNNs) to enhance activation energy predictions in chemical reactions, or in Quantitative Structure-Activity Relationships (QSARs), where molecular descriptors derived from quantum chemistry calculations or physical models are used as features to predict chemical properties.
3. Physical domain transformations. Data is mapped into domains where physics-relevant patterns are more easily captured. For example, signal processing often employs Fourier transforms to reveal frequency content, allowing oscillatory features to be revealed. This enables machine learning algorithms, such as convolutional neural networks (CNNs) to apply standard vision models, yielding better generalisation and efficiency by learning from spectrograms instead of raw waveforms.

These preprocessing methods are especially useful when physical insight is available, but the system is too complex for fully mechanistic modelling. By encoding physics into the data, standard machine learning architectures such as multi-layer perceptrons (MLPs) can be trained without needing architecture-specific changes. This class of approach enables physics-guided learning, where training data already obeys physical laws. As a result, the learned mapping is inherently constrained by the input features, and the model does not need to discover fundamental physical relationships from scratch, since key patterns are already embedded in the data.

Physics-Encoded Learning

Physics-encoded learning, otherwise known as hybrid modelling, combines physics-based components with data-driven components in a singular framework. This approach is useful when the underlying physical laws are partially understood but insufficient to describe the full system behaviour, and are computationally expensive to simulate. In such methods, the final model $M$ integrates the physics-based model $G$ and the data-driven correction term $D$, along with additional biases to narrow the solution space to physically plausible outputs such that the system formulates:$$f(x)\approx M[G(\mathbf {x} _{model},\mathbf {\theta } _{p})\cup D(\mathbf {x} _{measured},\mathbf {y} _{measured},\mathbf {\theta } _{D})+biases]$$Common examples of physics-encoded learning include Gaussian Process (GP) latent force models and Physics-Informed Sparse Identification of Nonlinear Dynamics (PhI-SINDy), which have been used to model multiple degree-of-freedom (MDOF) oscillator with multiple Coulomb friction contacts under harmonic load using both synthetic and experimental noisy experiments with multiple sources of discontinuous nonlinearities.