Scientific Machine Learning: Air Force Center of Excellence on Multi-Fidelity Modeling of Rocket Combustor Dynamics

Predicting combustion instabilities requires new capabilities in scientific machine learning, nonlinear model reduction, data-driven reduced models, and multi-fidelity modeling.

Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems

Qian, E., Kramer, B., Peherstorfer, B., and Willcox, K., Physica D: Nonlinear Phenomena, Volume 406, May 2020, 132401.

We present Lift & Learn, a physics-informed method for learning low-dimensional models for large-scale dynamical systems. The method exploits knowledge of a system's governing equations to identify a coordinate transformation in which the system dynamics have quadratic structure. This transformation is called a lifting map because it often adds auxiliary variables to the system state. The lifting map is applied to data obtained by evaluating a model for the original nonlinear system. This lifted data is projected onto its leading principal components, and low-dimensional linear and quadratic matrix operators are fit to the lifted reduced data using a least-squares operator inference procedure. Analysis of our method shows that the Lift & Learn models are able to capture the system physics in the lifted coordinates at least as accurately as traditional intrusive model reduction approaches. This preservation of system physics makes the Lift & Learn models robust to changes in inputs. Numerical experiments on the FitzHugh-Nagumo neuron activation model and the compressible Euler equations demonstrate the generalizability of our model.

[back to citation]

Learning physics-based reduced-order models for a single-injector combustion process

Swischuk, R., Kramer, B., Huang, C. and Willcox, K. AIAA Journal, published online March 2020. Also in Proceedings of 2020 AIAA SciTech Forum & Exhibition, Orlando FL, January, 2020. Also Oden Institute Report 19-13.

This paper presents a physics-based data-driven method to learn predictive reduced-order models (ROMs) from high-fidelity simulations, and illustrates it in the challenging context of a single-injector combustion process. The method combines the perspectives of model reduction and machine learning. Model reduction brings in the physics of the problem, constraining the ROM predictions to lie on a subspace defined by the governing equations. This is achieved by defining the ROM in proper orthogonal decomposition (POD) coordinates, which embed the rich physics information contained in solution snapshots of a high-fidelity computational fluid dynamics (CFD) model. The machine learning perspective brings the flexibility to use transformed physical variables to define the POD basis. This is in contrast to traditional model reduction approaches that are constrained to use the physical variables of the high-fidelity code. Combining the two perspectives, the approach identifies a set of transformed physical variables that expose quadratic structure in the combustion governing equations and learns a quadratic ROM from transformed snapshot data. This learning does not require access to the high-fidelity model implementation. Numerical experiments show that the ROM accurately predicts temperature, pressure, velocity, species concentrations, and the limit-cycle amplitude, with speedups of more than five orders of magnitude over high-fidelity models. Our ROM simulation is shown to be predictive 200% past the training interval. ROM-predicted pressure traces accurately match the phase of the pressure signal and yield good approximations of the limit-cycle amplitude.

[back to citation]

Multifidelity method for locating aeroelastic flutter boundaries

Marques, A., Lam, R., Chaudhuri, A., Opgenoord, M. and Willcox, K., AIAA Journal, Vol. 58, No. 4, April 2020, pp. 1772-1784. Also in 21st AIAA Non-Deterministic Approaches Conference (AIAA Scitech), San Diego, CA, January 2019. DOI 10.2514/1.J058663.

This paper introduces a multifidelity method that can produce accurate estimates of the flutter boundary at a reduced cost by combining information from low- and high-fidelity aeroelastic models. Estimating the flutter boundary in the presence of non-linear aerodynamic phenomena is challenging because high-fidelity aeroelastic models are expensive to evaluate, and flutter analysis requires many model evaluations. On the other hand, relatively inexpensive approximate aeroelastic models (low-fidelity models) exist and are routinely applied to reduce the cost of estimating flutter, albeit with lower accuracy. The multifidelity method introduced here uses an active learning algorithm to leverage information from low-fidelity models. A multifidelity statistical surrogate is used to fit damping coefficient estimates computed with different aeroelastic models. This surrogate is used to estimate the uncertainty in the prediction of the flutter boundary, which drives the selection of new evaluations. The effectiveness of the multifidelity method is demonstrated by estimating the aeroelastic flutter boundary of a typical section model at a cost 85\% lower when compared to the bisection method. Four aeroelastic models are considered in this example: three models (including the high-fidelity model) use a computational fluid dynamics solver based on the Euler equations, whereas one model uses a two-dimensional doublet-lattice method.

[back to citation]

Balanced Truncation Model Reduction for Lifted Nonlinear Systems

Kramer, B., and Willcox, K., arXiv:1907.12084. Submitted.

We present a balanced truncation model reduction approach for a class of nonlinear systems with time-varying and uncertain inputs. First, our approach brings the nonlinear system into quadratic-bilinear (QB) form via a process called lifting, which introduces transformations via auxiliary variables to achieve the specified model form. Second, we extend a recently developed QB balanced truncation method to be applicable to such lifted QB systems that share the common feature of having an indefinite system matrix. We illustrate this framework and the multi-stage lifting transformation on a tubular reactor model. In the numerical results we show that our proposed approach can obtain reduced-order models that are more accurate than proper orthogonal decomposition reduced-order models in situations where the latter are sensitive to the choice of training data.

[back to citation]

Information Reuse for Importance Sampling in Reliability-Based Design Optimization

Chaudhuri, A., Kramer,B., and Willcox, K., Reliability Engineering and System Safety, published online January 2020.

This paper introduces a new approach for importance-sampling-based reliability-based design optimization (RBDO) that reuses information from past optimization iterations to reduce computational effort. RBDO is a two-loop process-an uncertainty quantification loop embedded within an optimization loop-that can be computationally prohibitive due to the numerous evaluations of expensive high-fidelity models to estimate the probability of failure in each optimization iteration. In this work, we use the existing information from past optimization iterations to create efficient biasing densities for importance sampling estimates of probability of failure. The method involves two levels of information reuse: (1) reusing the current batch of samples to construct an a posteriori biasing density with optimal parameters, and (2) reusing the a posteriori biasing densities of the designs visited in past optimization iterations to construct the biasing density for the current design. We demonstrate for the RBDO of a benchmark speed reducer problem and a combustion engine problem that the proposed method leads to computational savings in the range of 51% to 76%, compared to building biasing densities from scratch in each iteration.

[back to citation]

Transform & Learn: A data-driven approach to nonlinear model reduction

Qian, E., Kramer, B., Marques, A., Willcox, K., AIAA Aviation 2019 Forum, June 2019, Dallas, TX. DOI 10.2514/6.2019-3707.

This paper presents Transform & Learn, a physics-informed surrogate modeling approach that unites the perspectives of model reduction and machine learning. The proposed method uses insight from the physics of the problem - in the form of partial differential equation (PDE) models - to derive a state transformation in which the system admits a quadratic representation. Snapshot data from a high-fidelity model simulation are transformed to the new state representation and subsequently are projected onto a low-dimensional basis. The quadratic reduced model is then learned via a least-squares-based operator inference procedure. The state transformation thus plays two key roles in the proposed method: it allows the task of nonlinear model reduction to be reformulated as a structured model learning problem, and it parametrizes the machine learning problem in a way that recovers efficient, generalizable models. The proposed method is demonstrated on two PDE examples. First, we transform the Euler equations in conservative variables to the specific volume state representation, yielding low-dimensional Transform & Learn models that achieve a 0.05% relative state error when compared to a high-fidelity simulation in the conservative variables. Second, we consider a model of the Continuously Variable Resonance Combustor, a single element liquid-fueled rocket engine experiment. We show that the specific volume representation of this model also has quadratic structure and that the learned quadratic reduced models can accurately predict the growing oscillations of an unstable combustor.

[back to citation]

Reusing information for multifidelity active learning in reliability-based design optimization

Chaudhuri, A., Marques, A., Lam, R., and Willcox, K., 21st AIAA Non-Deterministic Approaches Conference (AIAA Scitech), San Diego, CA, January 2019. DOI 10.2514/6.2019-1222.

This paper develops a multifidelity method to reuse information from optimization history for adaptively refining surrogates in reliability-based design optimization (RBDO). RBDO can be computationally prohibitive due to numerous evaluations of the expensive high-fidelity models to estimate the probability of failure of the system in each optimization iteration. In this work, the high-fidelity model evaluations are replaced by cheaper-to-evaluate adaptively refined surrogate evaluations in the probability of failure estimation. The method reuses the past optimization iterations as an information source for devising an efficient multifidelity active learning (adaptive sampling) algorithm to refine the surrogates that best locate the failure boundary. We implement the information-reuse method using a multifidelity extension of efficient global reliability analysis that combines the expected feasibility function with a weighted lookahead information gain criterion to pick both the next sample location and information source used to evaluate the sample.

[back to citation]

Projection-based model reduction: Formulations for physics-based machine learning

Swischuk, R., Mainini, L., Peherstorfer, B. and Willcox, K., Computers and Fluids, Vol. 179, pp. 704-717, January 2019.

This paper considers the creation of parametric surrogate models for applications in science and engineering where the goal is to predict high-dimensional output quantities of interest, such as pressure, temperature and strain fields. The proposed methodology develops a low-dimensional parametrization of these quantities of interest using the proper orthogonal decomposition (POD), and combines this parametrization with machine learning methods to learn the map between the input parameters and the POD expansion coefficients. The use of particular solutions in the POD expansion provides a way to embed physical constraints, such as boundary conditions and other features of the solution that must be preserved. The relative costs and effectiveness of four different machine learning techniques—neural networks, multivariate polynomial regression, k-nearest-neighbors and decision trees—are explored through two engineering examples. The first example considers prediction of the pressure field around an airfoil, while the second considers prediction of the strain field over a damaged composite panel. The case studies demonstrate the importance of embedding physical constraints within learned models, and also highlight the important point that the amount of model training data available in an engineering setting is often much less than it is in other machine learning applications, making it essential to incorporate knowledge from physical models.

[back to citation]

Nonlinear model order reduction via lifting transformations and proper orthogonal decomposition

Kramer, B. and Willcox, K., Vol. 57 No. 6, pp. 2297-2307, 2019.

This paper presents a structure-exploiting nonlinear model reduction method for systems with general nonlinearities. First, the nonlinear model is lifted to a model with more structure via variable transformations and the introduction of auxiliary variables. The lifted model is equivalent to the original model-it uses a change of variables, but introduces no approximations. When discretized, the lifted model yields a polynomial system of either ordinary differential equations or differential algebraic equations, depending on the problem and lifting transformation. Proper orthogonal decomposition (POD) is applied to the lifted models, yielding a reduced-order model for which all reduced-order operators can be pre-computed. Thus, a key benefit of the approach is that there is no need for additional approximations of nonlinear terms, in contrast with existing nonlinear model reduction methods requiring sparse sampling or hyper-reduction. Application of the lifting and POD model reduction to the FitzHugh-Nagumo benchmark problem and to a tubular reactor model with Arrhenius reaction terms shows that the approach is competitive in terms of reduced model accuracy with state-of-the-art model reduction via POD and discrete empirical interpolation, while having the added benefits of opening new pathways for rigorous analysis and input-independent model reduction via the introduction of the lifted problem structure.

[back to citation]

Contour location via entropy reduction leveraging multiple information sources

Marques, A., Lam, R. and Willcox, K., Advances In Neural Information Processing Systems (NeurIPS), 31, pp. 5223-5233, 2018.

We introduce an algorithm to locate contours of functions that are expensive to evaluate. The problem of locating contours arises in many applications, including classification, constrained optimization, and performance analysis of mechanical and dynamical systems (reliability, probability of failure, stability, etc.). Our algorithm locates contours using information from multiple sources, which are available in the form of relatively inexpensive, biased, and possibly noisy approximations to the original function. Considering multiple information sources can lead to significant cost savings. We also introduce the concept of contour entropy, a formal measure of uncertainty about the location of the zero contour of a function approximated by a statistical surrogate model. Our algorithm locates contours efficiently by maximizing the reduction of contour entropy per unit cost.

[back to citation]

Multifidelity preconditioning of the cross-entropy method for rare event simulation and failure probability estimation

Peherstorfer, B., Kramer, B. and Willcox, K., SIAM/ASA Journal on Uncertainty Quantification, Vol. 6, No. 2, pp. 737-761, 2018.

Accurately estimating rare event probabilities with Monte Carlo can become costly if for each sample a computationally expensive high-fidelity model evaluation is necessary to approximate the system response. Variance reduction with importancesampling significantly reduces the number of required samples if a suitable biasing density is used. This work introduces a multifidelity approach that leverages a hierarchy of low-cost surrogate models to efficiently construct biasing densities for importance sampling. Our multifidelity approach is based on the cross-entropy method that derives a biasing density via an optimization problem. We approximate the solution of the optimization problem at each level of the surrogate-model hierarchy, reusing the densities found on the previous levels to precondition the optimization problem on the subsequent levels. With the preconditioning, an accurate approximation of the solution of the optimization problem at each level can be obtained from a few model evaluations only. In particular, at the highest level, only few evaluations of the computationally expensive high-fidelity model are necessary. Our numerical results demonstrate that our multifidelity approach achieves speedups of several orders of magnitude in a thermal and a reacting-flow example compared to the single-fidelity cross-entropy method that uses a single model alone.

[back to citation]

Multifidelity probability estimation via fusion of estimators

Kramer, B., Marques, A., Peherstorfer, B., Villa, U. and Willcox, K., Journal of Computational Physics, 392:385-402, 2019.

This paper develops a multifidelity method that enables estimation of failure probabilities for expensive-to-evaluate models via a new combination of techniques, drawing from information fusion and importance sampling. We use low-fidelity models to derive biasing densities for importance sampling and then fuse the importance sampling estimators such that the fused multifidelity estimator is unbiased and has mean-squared error lower than or equal to that of any of the importance sampling estimators alone. The presented general fusion method combines multiple probability estimators with the goal of further variance reduction. By fusing all available estimators, the method circumvents the challenging problem of selecting the best biasing density and using only that density for sampling. A rigorous analysis shows that the fused estimator is optimal in the sense that it has minimal variance amongst all possible combinations of the estimators. The asymptotic behavior of the proposed method is demonstrated on a convection-diffusion-reaction PDE model for which n=1.e5 samples can be afforded. To illustrate the proposed method at scale, we consider a model of a free plane jet and quantify how uncertainties at the flow inlet propagate to a quantity of interest related to turbulent mixing. The computed fused estimator has similar root-mean-squared error to that of an importance sampling estimator using a density computed from the high-fidelity model. However, it reduces the CPU time to compute the biasing density from 2.5 months to three weeks.

[back to citation]

Model reduction for reacting flow applications

Nguyen, V.B., Buffoni, M., Willcox, K. and Khoo, B.C., International Journal of Computational Fluid Dynamics, Volume 28, Issue 3-4, May 2014.

A model reduction approach based on Galerkin projection, proper orthogonal decomposition (POD), and the discrete empirical interpolation method (DEIM) is developed for chemically reacting flow applications. Such applications are challenging for model reduction due to the strong coupling between fluid dynamics and chemical kinetics, a wide range of temporal and spatial scales, highly nonlinear chemical kinetics, and long simulation run-times. In our approach, the POD technique combined with Galerkin projection reduces the dimension of the state (unknown chemical concentrations over the spatial domain), while the DEIM approximates the nonlinear chemical source term. The combined method provides an efficient offline–online solution strategy that enables rapid solution of the reduced-order models. Application of the approach to an ignition model of a premixed H2/O2/Ar mixture with 19 reversible chemical reactions and 9 species leads to reduced-order models with state dimension several orders of magnitude smaller than the original system. For example, a reduced-order model with state dimension of 60 accurately approximates a full model with a dimension of 91,809. This accelerates the simulation of the chemical kinetics by more than two orders of magnitude. When combined with the full-order flow solver, this results in a reduction of the overall computational time by a factor of approximately 10. The reduced-order models are used to analyse the sensitivity of outputs of interest with respect to uncertain input parameters describing the reaction kinetics.

[back to citation]

Projection-Based Model Reduction for Reacting Flows

Buffoni, M. and Willcox, K., AIAA-2010-5008, presented at 40th Fluid Dynamics Conference and Exhibit, Chicago, IL, June 28-July 1, 2010.

This paper presents a method based on the proper orthogonal decomposition for model reduction of reacting flow applications. Simulation of reacting flows requires the numerical integration of a system of nonlinear partial differential equations that couples conservation laws, equations of state, and equations describing the chemical source terms. Solution of this coupled system is particularly challenging due to the stiffness of the embedded kinetics and the high computational cost of integrating the nonlinear source term that arises from the chemical models. Existing model reduction approaches are based on separation of reaction timescales, and do not always result in sufficient levels of reduction. The proper orthogonal decomposition approach is combined with the discrete empirical interpolation method to achieve efficient computation of the nonlinear term in the reduced model. Results are shown for parameterized steady and unsteady models describing a two-dimensional premixed H2-Air flame governed by a nonlinear convection-diffusion-reaction equation.

[back to citation]