Funded by AFOSR and AFRL, grant FA9550-17-1-0195

In collaboration with Bill Anderson (Purdue), Karthik Duraisamy (Michigan, lead PI), Cheng Huang (Michigan), Boris Kramer (MIT), Alexandre Marques (MIT), Charles Merkle (Purdue), Benjamin Peherstorfer (NYU), Swanand Sardeshmukh (Purdue)

Predicting combustion instabilities requires new capabilities in nonlinear model reduction, data-driven reduced models, and multi-fidelity modeling.

The goal of this Center of Excellence is to advance the state-of-the-art in Reduced Order Models (ROMs) and enable efficient prediction of instabilities in liquid fueled rocket combustion systems. The challenges of modeling rocket combustors are manifold. The resulting combustion models are highly non-linear, multi-scale, poorly-conditioned and very high dimensional and it is generally unclear as to which spatial and temporal scales are important to capture transient events and combustor stability. For reduced-order models, the information loss resulting from unresolved modes require new closure strategies. With regards to data-driven models, full scale engine data not available, necessitating new approaches for multi-domain formulations.

MIT is contributing to the Center of Excellence by developing new methods for localized, adaptive and data-driven nonlinear reduced models. We are also developing a multi-fidelity Bayesian optimization strategy that leverages parametric reduced models for evaluating stability boundaries.

Swischuk, R., Kramer, B., Huang, C. and Willcox, K. , Learning physics-based reduced-order models for a single-injector combustion process. Oden Institute Report 19-13. Submitted.

Kramer, B., and Willcox, K. , Balanced Truncation Model Reduction for Lifted Nonlinear Systems. arXiv:1907.12084. Submitted.

Chaudhuri, A., Kramer,B., and Willcox, K. , Information Reuse for Importance Sampling in Reliability-Based Design Optimization. ACDL Technical Report TR-2019-01. Submitted.

Qian, E., Kramer, B., Marques, A., Willcox, K., Transform & Learn: A data-driven approach to nonlinear model reduction.. AIAA Aviation 2019 Forum, June 2019, Dallas, TX. DOI 10.2514/6.2019-3707.

Marques, A., Lam, R., Chaudhuri, A., Opgenoord, M. and Willcox, K., A multifidelity method for locating aeroelastic flutter boundaries. 21st AIAA Non-Deterministic Approaches Conference (AIAA Scitech), San Diego, CA, January 2019. DOI 10.2514/6.2019-0438.

Chaudhuri, A., Marques, A., Lam, R., and Willcox, K., Reusing information for multifidelity active learning in reliability-based design optimization. 21st AIAA Non-Deterministic Approaches Conference (AIAA Scitech), San Diego, CA, January 2019. DOI 10.2514/6.2019-1222.

Swischuk, R., Mainini, L., Peherstorfer, B. and Willcox, K., Projection-based model reduction: Formulations for physics-based machine learning, Computers and Fluids, Vol. 179, pp. 704-717, January 2019.

Kramer, B. and Willcox, K., Nonlinear model order reduction via lifting transformations and proper orthogonal decomposition. * AIAA Journal*, Vol. 57 No. 6, pp. 2297-2307, 2019.

Kramer, B., Marques, A., Peherstorfer, B., Villa, U. and Willcox, K., Multifidelity probability estimation via fusion of estimators. *Journal of Computational Physics*, 392:385-402, 2019.

Marques, A., Lam, R. and Willcox, K., Contour location via entropy reduction leveraging multiple information sources. Advances In Neural Information Processing Systems (NeurIPS) 31, pp. 5223-5233, 2018. Supplementary material.

Peherstorfer, B., Kramer, B. and Willcox, K. Multifidelity preconditioning of the cross-entropy method for rare event simulation and failure probability estimation. SIAM/ASA J. Uncertainty Quantification, 6(2), 737-761, 2018.

Nguyen, V.B., Buffoni, M., Willcox, K. and Khoo, B.C., Model reduction for reacting flow applications. International Journal of Computational Fluid Dynamics , Volume 28, Issue 3-4, May 2014.

Buffoni, M. and Willcox, K., Projection-Based Model Reduction for Reacting Flows. AIAA-2010-5008, presented at 40th Fluid Dynamics Conference and Exhibit, Chicago, IL, June 28-July 1, 2010

Swischuk, R., Kramer, B., Huang, C. and Willcox, K. , *Oden Institute Report 19-13*, submitted.

This paper presents a physics-based data-driven method to learn predictive reduced-order models (ROMs) from high-fidelity simulations, and illustrates it in the challenging context of a single-injector combustion process. The method combines the perspectives of model reduction and machine learning. Model reduction brings in the physics of the problem, constraining the ROM predictions to lie on a subspace defined by the governing equations. This is achieved by defining the ROM in proper orthogonal decomposition (POD) coordinates, which embed the rich physics information contained in solution snapshots of a high-fidelity computational fluid dynamics (CFD) model. The machine learning perspective brings the flexibility to use transformed physical variables to define the POD basis. This is in contrast to traditional model reduction approaches that are constrained to use the physical variables of the high-fidelity code. Combining the two perspectives, the approach identifies a set of transformed physical variables that expose quadratic structure in the combustion governing equations and learns a quadratic ROM from transformed snapshot data. This learning does not require access to the high-fidelity model implementation. Numerical experiments show that the ROM accurately predicts temperature, pressure, velocity, species concentrations, and the limit-cycle amplitude, with speedups of more than five orders of magnitude over high-fidelity models. ROM-predicted pressure traces accurately match the phase of the pressure signal and yield good approximations of the limit-cycle amplitude.

Kramer, B., and Willcox, K., *arXiv:1907.12084*. Submitted.

We present a balanced truncation model reduction approach for a class of nonlinear systems with time-varying and uncertain inputs. First, our approach brings the nonlinear system into quadratic-bilinear (QB) form via a process called lifting, which introduces transformations via auxiliary variables to achieve the specified model form. Second, we extend a recently developed QB balanced truncation method to be applicable to such lifted QB systems that share the common feature of having an indefinite system matrix. We illustrate this framework and the multi-stage lifting transformation on a tubular reactor model. In the numerical results we show that our proposed approach can obtain reduced-order models that are more accurate than proper orthogonal decomposition reduced-order models in situations where the latter are sensitive to the choice of training data.

Chaudhuri, A., Kramer,B., and Willcox, K., *ACDL Technical Report*, TR-2019-01. Submitted.

This paper introduces a new approach for importance-sampling-based reliability-based design optimization (RBDO) that reuses information from past optimization iterations to reduce computational effort. RBDO is a two-loop process-an uncertainty quantification loop embedded within an optimization loop-that can be computationally prohibitive due to the numerous evaluations of expensive high-fidelity models to estimate the probability of failure in each optimization iteration. In this work, we use the existing information from past optimization iterations to create efficient biasing densities for importance sampling estimates of probability of failure. The method involves two levels of information reuse: (1) reusing the current batch of samples to construct an *a posteriori* biasing density with optimal parameters, and (2) reusing the *a posteriori* biasing densities of the designs visited in past optimization iterations to construct the biasing density for the current design. We demonstrate for the RBDO of a benchmark speed reducer problem and a combustion engine problem that the proposed method leads to computational savings in the range of 51% to 76%, compared to building biasing densities from scratch in each iteration.

Qian, E., Kramer, B., Marques, A., Willcox, K., *AIAA Aviation 2019 Forum*, June 2019, Dallas, TX. DOI 10.2514/6.2019-3707.

This paper presents Transform & Learn, a physics-informed surrogate modeling approach that unites the perspectives of model reduction and machine learning. The proposed method uses insight from the physics of the problem - in the form of partial differential equation (PDE) models - to derive a state transformation in which the system admits a quadratic representation. Snapshot data from a high-fidelity model simulation are transformed to the new state representation and subsequently are projected onto a low-dimensional basis. The quadratic reduced model is then learned via a least-squares-based operator inference procedure. The state transformation thus plays two key roles in the proposed method: it allows the task of nonlinear model reduction to be reformulated as a structured model learning problem, and it parametrizes the machine learning problem in a way that recovers efficient, generalizable models. The proposed method is demonstrated on two PDE examples. First, we transform the Euler equations in conservative variables to the specific volume state representation, yielding low-dimensional Transform & Learn models that achieve a 0.05% relative state error when compared to a high-fidelity simulation in the conservative variables. Second, we consider a model of the Continuously Variable Resonance Combustor, a single element liquid-fueled rocket engine experiment. We show that the specific volume representation of this model also has quadratic structure and that the learned quadratic reduced models can accurately predict the growing oscillations of an unstable combustor.

Marques, A., Lam, R., Chaudhuri, A., Opgenoord, M. and Willcox, K., *21st AIAA Non-Deterministic Approaches Conference (AIAA Scitech)*, San Diego, CA, January 2019. DOI 10.2514/6.2019-0438.

Aeroelastic flutter is a dangerous dynamic instability that can affect aerospace vehicles in certain flight conditions. Producing an accurate estimate of the aeroelastic flutter boundary is a challenge because high-fidelity aeroelastic models (e.g., based on fluid-structure interaction) are expensive to evaluate, and aeroelastic flutter analysis requires a large number of model evaluations. Many relatively inexpensive approximate aeroelastic models (low-fidelity models) exist, and are routinely applied to reduce the cost of estimating aeroelastic flutter. We introduce a multifidelity method that can produce accurate estimates of the aeroelastic flutter boundary by combining information from low- and high-fidelity models This method constructs a mulifidelity statistical surrogate to fit data from all available models, learning the correlations between them. The statistical surrogate is used to measure the uncertainty in the aeroelastic flutter boundary predicted with a given set of model evaluations. The method then automatically selects new model evaluations using an active learning approach such that the uncertainty in the estimate of aeroelastic flutter boundary is reduced the most, by unit cost, at every iteration. We demonstrate the effectiveness of the multifidelity method by locating the aeroelastic flutter boundary of a typical section model.

Chaudhuri, A., Marques, A., Lam, R., and Willcox, K., *21st AIAA Non-Deterministic Approaches Conference (AIAA Scitech)*, San Diego, CA, January 2019. DOI 10.2514/6.2019-1222.

This paper develops a multifidelity method to reuse information from optimization history for adaptively refining surrogates in reliability-based design optimization (RBDO). RBDO can be computationally prohibitive due to numerous evaluations of the expensive high-fidelity models to estimate the probability of failure of the system in each optimization iteration. In this work, the high-fidelity model evaluations are replaced by cheaper-to-evaluate adaptively refined surrogate evaluations in the probability of failure estimation. The method reuses the past optimization iterations as an information source for devising an efficient multifidelity active learning (adaptive sampling) algorithm to refine the surrogates that best locate the failure boundary. We implement the information-reuse method using a multifidelity extension of efficient global reliability analysis that combines the expected feasibility function with a weighted lookahead information gain criterion to pick both the next sample location and information source used to evaluate the sample.

Swischuk, R., Mainini, L., Peherstorfer, B. and Willcox, K., *Computers and Fluids*, Vol. 179, pp. 704-717, January 2019.

This paper considers the creation of parametric surrogate models for applications in science and engineering where the goal is to predict high-dimensional output quantities of interest, such as pressure, temperature and strain fields. The proposed methodology develops a low-dimensional parametrization of these quantities of interest using the proper orthogonal decomposition (POD), and combines this parametrization with machine learning methods to learn the map between the input parameters and the POD expansion coefficients. The use of particular solutions in the POD expansion provides a way to embed physical constraints, such as boundary conditions and other features of the solution that must be preserved. The relative costs and effectiveness of four different machine learning techniques—neural networks, multivariate polynomial regression, k-nearest-neighbors and decision trees—are explored through two engineering examples. The first example considers prediction of the pressure field around an airfoil, while the second considers prediction of the strain field over a damaged composite panel. The case studies demonstrate the importance of embedding physical constraints within learned models, and also highlight the important point that the amount of model training data available in an engineering setting is often much less than it is in other machine learning applications, making it essential to incorporate knowledge from physical models.

Kramer, B. and Willcox, K., Vol. 57 No. 6, pp. 2297-2307, 2019.

This paper presents a structure-exploiting nonlinear model reduction method for systems with general nonlinearities. First, the nonlinear model is lifted to a model with more structure via variable transformations and the introduction of auxiliary variables. The lifted model is equivalent to the original model-it uses a change of variables, but introduces no approximations. When discretized, the lifted model yields a polynomial system of either ordinary differential equations or differential algebraic equations, depending on the problem and lifting transformation. Proper orthogonal decomposition (POD) is applied to the lifted models, yielding a reduced-order model for which all reduced-order operators can be pre-computed. Thus, a key benefit of the approach is that there is no need for additional approximations of nonlinear terms, in contrast with existing nonlinear model reduction methods requiring sparse sampling or hyper-reduction. Application of the lifting and POD model reduction to the FitzHugh-Nagumo benchmark problem and to a tubular reactor model with Arrhenius reaction terms shows that the approach is competitive in terms of reduced model accuracy with state-of-the-art model reduction via POD and discrete empirical interpolation, while having the added benefits of opening new pathways for rigorous analysis and input-independent model reduction via the introduction of the lifted problem structure.

Marques, A., Lam, R. and Willcox, K., *Advances In Neural Information Processing Systems (NeurIPS)*, 31, pp. 5223-5233, 2018.

We introduce an algorithm to locate contours of functions that are expensive to evaluate. The problem of locating contours arises in many applications, including classification, constrained optimization, and performance analysis of mechanical and dynamical systems (reliability, probability of failure, stability, etc.). Our algorithm locates contours using information from multiple sources, which are available in the form of relatively inexpensive, biased, and possibly noisy approximations to the original function. Considering multiple information sources can lead to significant cost savings. We also introduce the concept of contour entropy, a formal measure of uncertainty about the location of the zero contour of a function approximated by a statistical surrogate model. Our algorithm locates contours efficiently by maximizing the reduction of contour entropy per unit cost.

Peherstorfer, B., Kramer, B. and Willcox, K., *SIAM/ASA Journal on Uncertainty Quantification*, Vol. 6, No. 2, pp. 737-761, 2018.

Accurately estimating rare event probabilities with Monte Carlo can become costly if for each sample a computationally expensive high-fidelity model evaluation is necessary to approximate the system response. Variance reduction with importancesampling significantly reduces the number of required samples if a suitable biasing density is used. This work introduces a multifidelity approach that leverages a hierarchy of low-cost surrogate models to efficiently construct biasing densities for importance sampling. Our multifidelity approach is based on the cross-entropy method that derives a biasing density via an optimization problem. We approximate the solution of the optimization problem at each level of the surrogate-model hierarchy, reusing the densities found on the previous levels to precondition the optimization problem on the subsequent levels. With the preconditioning, an accurate approximation of the solution of the optimization problem at each level can be obtained from a few model evaluations only. In particular, at the highest level, only few evaluations of the computationally expensive high-fidelity model are necessary. Our numerical results demonstrate that our multifidelity approach achieves speedups of several orders of magnitude in a thermal and a reacting-flow example compared to the single-fidelity cross-entropy method that uses a single model alone.

Kramer, B., Marques, A., Peherstorfer, B., Villa, U. and Willcox, K., *Journal of Computational Physics*, 392:385-402, 2019.

This paper develops a multifidelity method that enables estimation of failure probabilities for expensive-to-evaluate models via a new combination of techniques, drawing from information fusion and importance sampling. We use low-fidelity models to derive biasing densities for importance sampling and then fuse the importance sampling estimators such that the fused multifidelity estimator is unbiased and has mean-squared error lower than or equal to that of any of the importance sampling estimators alone. The presented general fusion method combines multiple probability estimators with the goal of further variance reduction. By fusing all available estimators, the method circumvents the challenging problem of selecting the best biasing density and using only that density for sampling. A rigorous analysis shows that the fused estimator is optimal in the sense that it has minimal variance amongst all possible combinations of the estimators. The asymptotic behavior of the proposed method is demonstrated on a convection-diffusion-reaction PDE model for which n=1.e5 samples can be afforded. To illustrate the proposed method at scale, we consider a model of a free plane jet and quantify how uncertainties at the flow inlet propagate to a quantity of interest related to turbulent mixing. The computed fused estimator has similar root-mean-squared error to that of an importance sampling estimator using a density computed from the high-fidelity model. However, it reduces the CPU time to compute the biasing density from 2.5 months to three weeks.

Nguyen, V.B., Buffoni, M., Willcox, K. and Khoo, B.C., *International Journal of Computational Fluid Dynamics*, Volume 28, Issue 3-4, May 2014.

A model reduction approach based on Galerkin projection, proper orthogonal decomposition (POD), and the discrete empirical interpolation method (DEIM) is developed for chemically reacting flow applications. Such applications are challenging for model reduction due to the strong coupling between fluid dynamics and chemical kinetics, a wide range of temporal and spatial scales, highly nonlinear chemical kinetics, and long simulation run-times. In our approach, the POD technique combined with Galerkin projection reduces the dimension of the state (unknown chemical concentrations over the spatial domain), while the DEIM approximates the nonlinear chemical source term. The combined method provides an efficient offline–online solution strategy that enables rapid solution of the reduced-order models. Application of the approach to an ignition model of a premixed H2/O2/Ar mixture with 19 reversible chemical reactions and 9 species leads to reduced-order models with state dimension several orders of magnitude smaller than the original system. For example, a reduced-order model with state dimension of 60 accurately approximates a full model with a dimension of 91,809. This accelerates the simulation of the chemical kinetics by more than two orders of magnitude. When combined with the full-order flow solver, this results in a reduction of the overall computational time by a factor of approximately 10. The reduced-order models are used to analyse the sensitivity of outputs of interest with respect to uncertain input parameters describing the reaction kinetics.

Buffoni, M. and Willcox, K., AIAA-2010-5008, presented at 40th Fluid Dynamics Conference and Exhibit, Chicago, IL, June 28-July 1, 2010.

This paper presents a method based on the proper orthogonal decomposition for model reduction of reacting flow applications. Simulation of reacting flows requires the numerical integration of a system of nonlinear partial differential equations that couples conservation laws, equations of state, and equations describing the chemical source terms. Solution of this coupled system is particularly challenging due to the stiffness of the embedded kinetics and the high computational cost of integrating the nonlinear source term that arises from the chemical models. Existing model reduction approaches are based on separation of reaction timescales, and do not always result in sufficient levels of reduction. The proper orthogonal decomposition approach is combined with the discrete empirical interpolation method to achieve efficient computation of the nonlinear term in the reduced model. Results are shown for parameterized steady and unsteady models describing a two-dimensional premixed H2-Air flame governed by a nonlinear convection-diffusion-reaction equation.