## Abstract

The present study aims to assess the potential of the neural ordinary differential equations (NODE) network for reliable and computationally efficient implementation of chemistry in combustion simulations. Investigations are performed using a hydrogen-air pairwise mixing stirred reactor (PMSR). The PMSR is a zero-dimensional case affordable to study combustion chemistry entailing a similar numerical solution procedure as probability density function methods for turbulent combustion simulations. A systematic approach is presented to apply the NODE, solely trained on canonical constant pressure homogeneous reactor data, to predict complex chemistry and mixing interactions in PMSR. The reactor involves combustion of hydrogen in air described by a finite-rate mechanism with 9 chemical species and 21 reaction steps. The NODE network is shown to accurately capture the evolution of thermochemical variables for different mixing and chemical timescales. It also exhibits a significant reduction in numerical stiffness resulting in improving the computational efficiency and enabling the use of explicit solvers for the integration of chemical kinetics. The assessment results based on PMSR show that compared to direct integration of detailed kinetics, the NODE can achieve significant computational time speedup for a comparable accuracy.

## 1 Introduction and Research Background

Turbulent combustion lies at the core of numerous technologies for power generation, transportation, propulsion, heating, and other industrial processes. The simulations of turbulent combustion are inherently complex owing to the intricacies of turbulent flows, chemical kinetics, and the interactions between them. The brute-force approach of direct numerical simulation (DNS) involves resolving all spatiotemporal scales in a flow field, which is highly precise but exorbitantly expensive for real-world problems such as internal combustion engines and gas turbine combustors. Alternatively, the averaging or filtering approaches in Reynolds-averaged Navier–Stokes (RANS) and large eddy simulation (LES) offer tractable ways to carry out turbulent combustion simulations. However, averaging or filtering leads to closure problems, particularly for the highly nonlinear chemical reaction source term. In LES, for instance, the computational expense is mitigated by filtering the governing equations so that only large scales of the flow field are resolved while motions at the small scales, generally referred to as subgrid scales, are modeled. However, modeling reactive scalars at the subgrid scales is challenging since the fluctuations at the unresolved small scales, predominantly caused by molecular mixing and reactions, strongly influence the filtered chemical reaction source term and vice versa. An effective approach for this purpose is the filtered density function (FDF) methodology which offers an exact way to formulate the reaction source term in a closed form, eliminating the need for its further modeling [1]. The FDF is essentially the LES equivalent of probability density function (PDF) method [2,3] in RANS; we thus generally refer to both as PDF henceforward.

The joint PDF of reactive scalars carries information about species concentrations and temperature evolution in a highly dimensional state space which makes obtaining the direct numerical solution of its transport equation extremely challenging. Hence, the method of Lagrangian Monte Carlo is often used to solve for the joint PDF of reactive scalars [24]. In this approach, the PDF transport equation is cast into an equivalent system of stochastic differential equations (SDEs) which is solved by the Monte Carlo method to determine the composition evolution of Lagrangian particles. The evolution of a large ensemble of such notional particles essentially provides an equivalent solution to the PDF transport equation for the scalars. A common approach that facilitates the solution of SDEs for each particle involves the use of a simple splitting method to decouple the mixing process (due to convection and diffusion) from the chemical reaction effects and to solve the corresponding equations in two separate steps [57]. For typical flame calculations, the chemical reaction step requires an enormous number of integrations of the chemical kinetics stiff ordinary differential equations (ODEs) for all particles. Therefore, developing modeling techniques that can effectively reduce stiff chemical kinetics integration cost leads to significant savings in computations, which is the main motivation of the present work.

A prevailing strategy to moderate the computational cost of chemistry calculations is through the reduction of size and stiffness of the detailed chemical mechanism. Most widely employed mechanism reduction methodologies include sensitivity analysis, directed relation graph (DRG) [8], principal component analysis (PCA) [9], computational singular perturbation (CSP) [10], quasi-steady-state approximations (QSSA) [11], partial-equilibrium approximations (PEA) [12], rate-controlled constrained-equilibrium (RCCE) [1318], and intrinsic low dimensional manifold (ILDM) [19]. Although these approaches assist in alleviating computational demand, the resulting reduced mechanisms often retain some degree of numerical stiffness, necessitating the use of stiff ODE solvers to obtain the solution. Consequently, in some of the previous works, the reduced mechanisms are coupled with storage and retrieval methods such as look-up tables (LUT) [20] and in situ adaptive tabulation (ISAT) [7] to further decrease the cost of chemical kinetics integration. Despite the acceleration in computation time through tabulation methods, their application to PDF simulations is hindered by the necessity to perform direct integration of stiff chemistry at the initial stages of the simulation, which can make the tabulation prohibitively demanding for practical applications. Additionally, as more regions of the composition state space are accessed, the storage requirements and average retrieval times from tables can grow considerably.

In addition to the aforementioned classical modeling techniques, numerous researchers have focused on data-driven approaches based on diverse artificial neural network (ANN) topologies to address the high computational cost of chemistry integration and the increased storage requirement of tabulation methods. The pioneer works of Refs. [21,22] utilized a multilayer perceptron (MLP) topology neural network to tabulate relatively smaller finite-rate chemical kinetics and established its applicability in the simulation of turbulent flames. As the number of species increases, the composition space expands exponentially. To address the increased complexity in such cases, studies [23,24] advanced to clustering algorithms such as self-organizing-map (SOM) [25] and K-means [26] to cluster the composition space into several subdomains in order to manage complex chemical mechanisms. These studies reported good agreement with direct integration calculations, as well as significant savings in computational time and memory requirements. They did, however, identify a significant difficulty in the generation of representative datasets, which made the generalization of ANNs to actual conditions countered in real flames difficult. Later Refs. [27,28] encountered this issue by generating training data through an abstract problem (such as laminar flamelets) spanning the expected composition space and applying the trained ANN to simulate real turbulent flames.

Most prior research studies appear to have primarily focused on using an MLP neural network as a single-step chemistry integrator, with only minor differences in their network architectures. The species concentration and enthalpy at a given time are inputs to the ANN model, and the outputs are the species concentration/chemical source terms at the next time-step. Such ANN models have limited applicability in multistep predictions because they are trained by minimizing single-step prediction errors. The model could accumulate larger errors in longer-time predictions by recursively feeding the outputs from the previous time-step to the ANN as new inputs. This problem was addressed in Ref. [29], which adopted the neural network architecture of the residual network model (ResNet) to minimize multistep prediction error. Unlike MLP ANNs, ResNets are specifically designed for modeling time-series data. Recently, a class of machine learning algorithms known as neural ordinary differential equations (NODE) has emerged as superior alternatives to ResNets [30]. The NODE network is better suited for chemical kinetics integration because they have simpler architecture and better multistep prediction accuracy than simple MLPs and ResNets. Additionally, the NODE network can reduce the stiffness of the system, which can enable the use of an explicit ODE solver resulting in faster computations. The work of Ref. [31] was among the first to demonstrate the use of the NODE network in combustion chemistry. Because of the nonlinearity associated with the dynamics of various species, this study concluded that attempting to learn the source terms for all species concurrently was too unstable. As a result, separate networks were trained for each major species and temperature, with minor species being neglected. Subsequently, a methodical strategy was introduced in Ref. [32] to regularize and construct a single NODE network to learn the dynamics of coupled chemical kinetics. On a similar line, the work of Ref. [33] employed a stiffness-reduced autoencoder and NODE to develop a reduced-order model for methane. The study was specifically focused on training and demonstrating the model performance in a plug flow reactor and a continuous stirred tank reactor.

In the present study, we demonstrate the performance of a novel approach for coupling chemical kinetics NODE network with turbulent mixing to predict reacting systems. The NODE network is solely trained on canonical constant pressure homogeneous reactor data. This is a critical issue to address in order to ensure that the NODE network provides accurate predictions when used for complex turbulent combustion simulations. To our knowledge, no particular study has investigated the performance of the NODE network for chemical kinetics in the presence of the turbulent mixing. Investigations are carried out using a reacting pairwise mixing stirred reactor (PMSR) as the numerical solution procedure in PMSR is analogous to that in PDF methods [57,17,34]. The direct integration of detailed kinetic model (DKM) in the chemistry fractional step is replaced by NODE network integration and the computational performance of the NODE network is assessed by comparing the results with the former. Additional tests are performed to determine the sensitivity of the NODE network to numerical parameters such as error tolerances, ODE solvers, and the global time-step size. Although this methodology is directly relevant to simulations using the PDF method, it is also equally applicable to other approaches requiring direct integration of chemical mechanisms while employing splitting schemes such as DNS, unsteady flamelets, conditional moment closure (CMC), eddy dissipation concept (EDC), and partially stirred reactor (PaSR) models [35,36].

The paper is organized as follows: Sec. 2 outlines the basics of PMSR followed by a description of the NODE network-based chemistry integrator framework and NODE network adaptation in PMSR solution methodology. Section 3 discusses the performance and numerical efficiency of the NODE network for H2-air combustion in PMSR for various configurations and mixing intensities. That includes investigation results pertaining to evaluating the applicability of the NODE network using various ODE solvers, error tolerance, and global time-steps. Finally, Sec. 4 provides the concluding remarks along with an overview of the capabilities of the proposed methodology.

## 2 Research Methodology

### 2.1 Pairwise Mixing Stirred Reactor Formulation.

In the computational implementation of PDF methods, the fluid within the solution domain is represented by a large number of stochastic particles. The thermochemical state of a reactive gaseous flow can be characterized at any point and time (t) by the mass fractions Yi (i = 1, 2, …, ns) of the ns species, the enthalpy h, and the pressure p. We consider a broad class of low Mach number flows in which p deviates by a very small fraction from a fixed reference pressure p0, so that at any given p0, the state of the system is determined by composition vector Φ ≡ (Y1, Y2, …, Yns, h) [2]. The evolution of Φ is governed by
$dΦdt=M(Φ)+S(Φ)$
(1)
where M is the mixing term comprising the rate of change due to the mixing of particles and S is the chemical reaction source term (with Sns+1 ≈ 0 for low Mach number flows). Equation (1) is typically solved numerically using a zero-order operator splitting scheme that separates mixing and chemistry contributions to the evolution of Φ [5]. For example, the solution is advanced from time t0 for a small time-step dt by the following procedure:
1. From the initial condition Φ0, the mixing equation
$dΦdt=M(Φ)$
(2)
is integrated for a time dt, and the solution at the intermediate step, denoted by Φ*, is obtained.
2. From the intermediate step Φ*, the reaction equation

$dΦdt=S(Φ)$
(3)
is integrated for a time dt, to obtain a close approximation to Φ(t0 + dt). When using finite-rate chemical kinetics with DKM, direct integration of Eq. (3) is challenging owing to the high nonlinearity and stiffness of chemical source terms.
In the present study, we employ the NODE network for efficient integration of chemical kinetics, i.e., to obtain the solution for Eq. (3), and compare it with the solution obtained by directly integrating DKM. The details of the NODE network inference are described in Sec. 2.2. The chemical mechanism considered is that of H2-air consisting of nine species: H2, H, O2, OH, O, H2O, H2O2, HO2, and N2 along with 21 reaction steps [37,38]. We utilize PMSR to assess the performance of the NODE network as PMSR corresponds to zero-dimensional PDF method calculations, presenting events occurring at a single computational cell in more detailed turbulent reactive flow simulations using, e.g., the LES-FDF approach. Moreover, variation of mixing and chemical timescales in PMSR not only provides a computationally efficient representation of distributed turbulent reacting flow conditions found in many practical devices but also offers a stringent test to evaluate the performance of various turbulence and combustion models. The mixing process in PMSR involves macro-mixing, characterized by residence (τr) and pairing (τp) timescales, and micro-mixing characterized by mixing timescale (τm). The former two are associated with larger scale mixing events due to the bulk motion of fluid while the latter describes molecular-scale mixing. PMSR essentially represents perfect macro-mixing, i.e., no large-scale spatial gradients of scalars, but imperfect micro-mixing, i.e., incomplete mixing at the molecular scales. The reactor, at any time-step t, consists of an even number N of particles, initially arranged in pairs (p, q) such that the particles (1, 2), (3, 4)…., (N − 1, N) are partners. At a specific time-step dt, for each discrete time k dt, where k is an integer, three events occur corresponding to inflow, outflow, and pairing, causing the composition of the ith particle, Φ(i)(t), to change discontinuously. The inflow and outflow events consist of randomly selecting N dt/2τr pairs and replacing their compositions with inflow compositions. The pairing event consists of randomly selecting N dt/2τp pairs of particles, different from the inflow particles. Then, these pairing particles and the inflowing particles are randomly shuffled so that they most likely change partners. Between these discrete times, the compositions of pairs of particles (p, q) evolve by a mixing fractional step
$dΦ(p)dt=(Φ(p)−Φ(q))τmdΦ(q)dt=(Φ(q)−Φ(p))τm$
(4)
In the reaction fraction step, each particle evolves according to Eq. (3).

### 2.2 NODE Network Framework for H2-Air Chemical Kinetics.

Figure 1 encapsulates the methodology of the framework for the NODE network-based chemistry integrator. The procedure depicted briefly describes (1) the generation of training data by performing simulations of zero-dimensional constant pressure homogeneous reactor, (2) the preprocessing and regularization of training data to facilitate stable training of the NODE network, (3) the NODE network architecture and training methodology, and (4) the workflow to employ optimized NODE network to obtain the solution of Eq. (3). The detailed description is as follows:

1. We select the canonical constant pressure homogeneous reactor to generate the training data as the reaction equation (Eq. (3)) to be solved describes the evolution of species in the same reactor. During training, enthalpy is replaced by temperature as the latter has a nonzero source term and its redundancy offers an accuracy check during testing (by comparing it with the temperature obtained using the mixture composition). The training samples are generated using 30 constant pressure reactor simulations with initial temperatures ranging from 950 K to 1200 K and equivalence ratio in the range of 0.5–1.5 at atmospheric pressure (101,325 Pa) in order to generate representative composition space for real flames. The training of the NODE network depends solely on data and is not sensitive to the choice of ODE solvers used for data generation. Each of the 30 cases has its solution advanced from t = 0 to 10−2 s with a fixed dt = 10−6 s, resulting in 104 uniformly spaced points along the temporal composition trajectory for each simulation. As a result, a 3 × 105 sample training dataset is obtained. In this study, we utilize the ODE solver LSODA via scipy toolkit [39] for direct integration of Eq. (3) with DKM description of chemistry. LSODA is a hybrid solver, switching automatically between the nonstiff Adams method and the stiff backward differentiation formula (BDF) solver.

2. The use of a full dataset (3 × 105 data points) for training is unnecessary because most composition states reach equilibrium after ignition. Hence, we strategically sample the subset from the complete solution and regularize it in order to accelerate the NODE network training process. The samples of uneven log-spaced 50 points are chosen of 104 time-steps for each simulation. A half of sampled points are placed before the ignition delay time (τign) and the rest after the ignition. An example of sampling is shown in Fig. 1 for three cases corresponding to constant pressure hydrogen-air reactor at initial temperatures of T0 = 1000 K, 1100 K, 1200 K, and stoichiometric equivalence ratio. The ignition delay time is defined as the instant when the reactor temperature reaches T0 + 400 K. The chosen samples seem to capture the system dynamics of ignition curves well and enable training of the NODE network model with manageable computational requirements. The wide range of species concentration magnitudes is another challenge with training datasets for chemical reaction applications; for example, the major species mass fraction values are of the order 10−3 − 100, while the minor species mass fractions are of the order of 10−3 − 10−8. Therefore, for network stability, all the composition variables are normalized based on the respective minimum (Φmin) and maximum (Φmax) values obtained from the complete training dataset
$Φ¯=Φ−ΦminΦmax−Φmin$
(5)
3. Equations (3) and (5) are encapsulated in the neural ODE network and mathematically represented as
$dΦ¯(t)dt=Net(Φ¯(t),θ,t)$
(6)
where the NODE network, $Net(Φ¯(t),θ,t)$, is the representation of the chemical reaction source term of $Φ¯(t)$ by the neural network with network parameters θ. The network consists of three hidden layers constituting 150 neurons each. Each hidden layer is followed by a nonlinear ELU activation function. The explicit Runge–Kutta method of order (4)5 (dopri5) [40] with absolute and relative error tolerances 10−9 and 10−7 is chosen, respectively, for integration during the training. Embedding of an explicit solver ensures that the network learns the nonstiff representation of the chemical reaction source term through training. To improve learning performance, the network is trained by sending data in batches. One training batch consists of 50 filtered time instances on the composition space evolution trajectory for 30 initial conditions. As a result, the network learns from all training points at the same time in each epoch. Each epoch begins with a forward pass to calculate the mean absolute error (MAE) of a training batch, followed by backpropagation to obtain loss function derivatives with respect to input parameters. Then, the neural network weights are adjusted to minimize the MAE loss function using the adaptive moment estimation (Adam) optimization technique for gradient descent. The learning rate is automatically varied from 10−2 to 10−4 until the MAE is reduced to less than $5%$. We adopt torchdiffeq algorithmic package implemented in PyTorch [30,40] to train the NODE network for chemical kinetics. Further details about NODE network formulation and training details can be found in Ref. [32].
4. Once the network is trained with the desired accuracy, it can be integrated with any ODE solver to obtain its state at any arbitrary time tN > t0. For example, with the normalized initial values $Φ¯0$ at time t0, integrating forward to time tN using an ODE solver gives
$Φ¯N=Φ¯0+∫t0tNNet(Φ¯,θ,t)dt$
(7)
Subsequently, the normalized predicted values $Φ¯N$ can be converted to absolute composition vector ΦN using Eq. (5).

Fig. 1
Fig. 1
Close modal

## 3 Results and Discussion

### 3.1 Predictive Performance of NODE Network in PMSR.

At first, we consider a single PMSR test case for the demonstration. The test case has three incoming streams: air ($79%$ N2, 21% O2) at 1200 K; H2 at 500 K; and a pilot stream consisting of an equilibrium mixture corresponding to the inflow of the fuel/air conditions. The mass flowrates of these streams are in the ratio 0.873 : 0.027 : 0.1, respectively. Initially (t = 0), all particles are set to a mixture composition of inlet fuel and oxidizer streams resulting in an average temperature of 1004 K. Pressure is atmospheric throughout. Stochastic simulations with 1000 Monte Carlo particles are conducted over a span of five residence times to reach the statistically stationary state. DKM integration is performed using a hybrid LSODA solver [39] with an absolute error tolerance of ηa = 10−8 for species mass fraction and 10−6 for temperature, and relative error tolerance of ηr = 10−6. Tolerances are chosen based on the solver’s ability to produce converged solutions for a given set of operating conditions. The larger temperature ηa is because temperature values are of 2–3 orders of magnitude larger than those of species mass fractions. The numerical specifications in PMSR calculations are summarized in Table 1. The NODE network is integrated with an explicit ODE solver based on the Runge–Kutta method of order 5(4) (RK45) [39] with optimal error tolerances obtained with the procedure described in the previous study, i.e., ηa = 10−5 and ηr = 10−4 [32].

Table 1

Numerical specifications

ParameterSpecified value
Number of particles, N1000
Residence time, τr10−3 s
Mixing time, τm10−4 s
Pairing time, τp10−4 s
Global time-step, dt2.5 × 10−5 s
Total number of time-step, Nt200
Pressure, p1 bar
Mixture fraction of inlet stream, Z0.03
Mixture fraction of pilot stream, Zpilot0.1
Temperature of inflow air particles, T0,air1200 K
Temperature of inlet fuel particles, T0,fuel500 K
ODE solver for integration of DKMLSODA
DKM absolute error tolerance, ηa10−8
DKM relative error tolerance, ηr10−6
ODE solver for integration of NODE networkRK45
NODE network absolute error tolerance, ηa10−5
NODE network relative error tolerance, ηr10−4
ParameterSpecified value
Number of particles, N1000
Residence time, τr10−3 s
Mixing time, τm10−4 s
Pairing time, τp10−4 s
Global time-step, dt2.5 × 10−5 s
Total number of time-step, Nt200
Pressure, p1 bar
Mixture fraction of inlet stream, Z0.03
Mixture fraction of pilot stream, Zpilot0.1
Temperature of inflow air particles, T0,air1200 K
Temperature of inlet fuel particles, T0,fuel500 K
ODE solver for integration of DKMLSODA
DKM absolute error tolerance, ηa10−8
DKM relative error tolerance, ηr10−6
ODE solver for integration of NODE networkRK45
NODE network absolute error tolerance, ηa10−5
NODE network relative error tolerance, ηr10−4
At a given time, the stochastic nature of PMSR gives rise to the spread of composition variables among the particles. Thus, the results are presented using the ensemble mean and variance of a scalar (〈ϕ〉 and $⟨ϕ′2⟩$, respectively)
$⟨ϕ⟩=1N∑i=1Nϕi$
(8)
$⟨ϕ′2⟩=1N∑i=1N(ϕi−⟨ϕ⟩)2$
(9)
The normalized global error in NODE network predictions is defined by
$εG,⟨ϕ⟩=1Nt∑j=1Nt|⟨ϕ⟩j,DKM−⟨ϕ⟩j,NODEϕmax−ϕmin|$
(10)
where ϕmax and ϕmin are the maximum and minimum values of the scalar obtained from the same training dataset as in Eq. (5).

Figure 2(a) compares the evolution of ensemble average temperature and species mass fraction evolution in PMSR obtained with the NODE network and DKM on a log-scale plot. The NODE network predicts the average temperature in the reactor and ignition delay time accurately compared to DKM. The evolution of major species: fuel (H2), oxidizer (O2), and product (H2O), along with the highly reactive OH radical is also well captured by the NODE network. The NODE network agreeably captures the monotonic increase in temperature and product, depletion of oxidizer and fuel, and nonmonotonic behavior of OH along with the fluctuations caused by stochastic particles. The similarities in instantaneous particle temperature predicted by both methods are also evident from Figs. 2(b) and 2(c). There are visible inflowing particles carrying fuel and oxidizer at respective inflow temperatures throughout the simulation. At initial times, the particles are clustered around specified initial temperatures of 1004 K. As these particles mix with higher temperature particles, they ignite and eventually reach a statistically stationary state close to the equilibrium state.

Fig. 2
Fig. 2
Close modal

To further demonstrate the applicability of the NODE network in a broader composition space, results are presented for nonpremixed and premixed inlet stream configurations. In a nonpremixed configuration, the fuel and air enter the reactor as separate streams similar to the aforementioned test case. Whereas, in premixed configuration, only one inlet stream carrying the fuel and air mixture is used. For the premixed inlet, the equivalence ratio of the inflowing mixture is varied from lean to rich conditions at T0 = 1000 K, and the dynamics of temperature evolution are captured as shown in Figs. 3(a) and 3(b). The NODE network shows an excellent comparison in 〈T〉 and root mean square (RMS) of T, $⟨T′2⟩$, for all three equivalence ratios. The global errors in all 〈T〉 and other species mass fractions predominantly stay below $2%$ and $10%$, respectively, as depicted in Fig. 3(c). With a premixed inlet, the majority of particles in the reactor act as an independent constant pressure homogeneous reactor. Hence, 〈T〉 profiles look similar to temperature evolution in the constant pressure homogeneous reactor. Also, the trends in ignition delay (τign) and equilibrium temperatures (Teq) are consistent with the homogeneous reactor [32]. For example, changing the equivalence ratio of a stoichiometric mixture to a lean or rich mixture results in longer chemical time scales characterized by τign. The stoichiometric mixture has the highest Teq and fastest ignition delay and the lean mixture has the lowest Teq. Analyzing the statistics, it is observed that the NODE network exhibits smoother profiles compared to DKM. We speculate that this is due to the removal of small timescales during the stiffness reduction of the underlying chemical kinetics by the NODE network.

Fig. 3
Fig. 3
Close modal

Figure 4(a) shows the 〈T〉 evolution in PMSR for lean to rich mixture fractions of inlet streams in nonpremixed configuration. For H2-air the stoichiometric mixture fraction is Zst ≈ 0.03. The inlet temperatures also vary according to mixture fraction values, resulting in further expansion of the composition state space occupied by particles. The larger mixture fraction contributes to increased fuel in the reactor, resulting in higher equilibrium temperatures. However, unlike the premixed case, the leaner case ignites faster indicating the importance of temperature in the ignition process due to the nonlinear dependence of reaction rates on temperature. The aggregate effect is faster ignition with increasing average inflow temperature. These trends are clearly evident in Figs. 4(a) and 4(b). Also, the mean and RMS of T compare well with the DKM. As depicted in Fig. 4(c), the largest errors are observed for H radical for fuel-rich conditions. However, the global errors in 〈T〉 and other species mass fractions predominantly stay below $4%$ and $10%$, respectively.

Fig. 4
Fig. 4
Close modal

Overall, the performance of the NODE network in Figs. 3 and 4 demonstrates the generalizability of the NODE network for a wide composition space and varying chemical timescales.

### 3.2 Effect of Mixing Timescales.

In realistic turbulent reacting flows, there is a strong interaction between chemical kinetics and mixing at various scales. Considering that the NODE network is solely trained on constant pressure homogeneous reactor with no mixing, it is important to establish the accuracy and performance of the NODE network when interacting with varying mixing timescales. In PMSR, micro-scale mixing is controlled by mixing timescale. On the other hand, mixing processes occurring at larger scales are characterized by the residence and pairing timescales. In this section, we present the results of the NODE network for varying residence, pairing, and mixing timescales, respectively. Apart from the specific timescales, the remaining numerical specifications of DKM and NODE network simulations presented in this section are the same as those summarized in Table 1.

The effect of residence timescale in PMSR is depicted in Fig. 5. Timescale τr is associated with large-scale mixing in the reactor. As observed from Figs. 5(a) and 5(b), NODE network predictions of 〈T〉 and $⟨T′2⟩$ are in good agreement with DKM for all the residence times. Higher residence times correspond to a lower influx of fresh particles and thus, fewer hot particles to ignite the reactor. As a result, ignition delay times are increased with increasing residence times causing a longer time required to reach an equilibrium temperature. It is also clear that the average temperature at a stationary state decreases with decreasing τr. This can be attributed to the more frequent replacement of ignited particles with colder inflowing particles which in turn reduces the average temperature of the system. In extreme cases of very small τr values, inflow/outflow processes dominate and more particles get removed from the reactor before ignition, causing the average temperature of the reactor to fluctuate around the temperatures of incoming particles (≈1083 K). Figure 5(c) shows the global errors for all species mass fractions and temperatures corresponding to three residence times. In the majority of the cases, global error in averaged species mass fractions remains below $11%$ including highly reactive radicals such as O, H2O2, and HO2. However, the errors in minor species OH and H, two short-lived highly reactive radicals, rise to $25%$ for high τr. One possible explanation is that these radicals are known to be good QSSA species candidates which have shorter timescales and remain low in concentration [10]. In the process of removing stiffness, the NODE network loses some information about the fast species. Despite these local errors, the NODE network quite satisfactorily predicts the global ignition characteristics such as equilibrium temperature and ignition delay times.

Fig. 5
Fig. 5
Close modal

The pairing timescale controls macro-mixing in PMSR by exchanging the particle partners at random. The smaller pairing times entail the frequent exchange of partners, hence more effective mixing of hot and fresh particles. This results in higher Teq. This tendency can be observed from Figs. 6(a) and 6(b) with exact similarity. Also, the pairing time has almost no effect on the initial rate of increase in temperature or ignition delays. The global error in scalars remains below $10%$ except in the minor species radicals OH and H similar to the effects of τr. However, the 〈T〉 and $⟨T′2⟩$ predicted by the NODE network match well with those from DKM for all three pairing timescales. We also observe a peculiar behavior of sudden drop in Teq for τp = 5 × 10−4, 10−3 s after ignition. This can be attributed to the local extinction of particles because of an inefficient supply of reactants caused by slow macro-mixing. Nevertheless, the NODE network is able to accurately capture this phenomenon as well.

Fig. 6
Fig. 6
Close modal

At last, the interaction between micro-mixing and chemistry is studied by varying the mixing timescale. Figures 7(a) and 7(b) show the comparison between the NODE network and DKM predictions of 〈T〉 and $⟨T′2⟩$ for three τm values. For small mixing timescales such as τm = 10−4 s, mixing is fast and Teq reaches the corresponding chemical equilibrium state value, Teq = 2550 K. However, at larger mixing timescales the equilibrium temperature decreases due to slower reaction progress caused by imperfect micro-mixing. It is also observed that as mixing only tends to redistribute energy among particles at the initial times, τm has almost no influence on the initial rate of increase in temperature or ignition delays. Similar trends are noticed for even smaller τm values (not shown). The NODE network shows good agreement between the instantaneous mean and RMS temperature values for all the mixing timescales. As shown in Fig. 7(c), the global error in scalars predominantly remains below $10%$ except for OH radical.

Fig. 7
Fig. 7
Close modal

Similar results are obtained when experiments are conducted using various combinations of residence, pairing, and mixing timescales between 10−4 and 10−2 s. This study indicates that the NODE network accurately captures the combustion characteristics such as ignition, extinction, and statistics of thermochemical scalars. Overall, the NODE network can satisfactorily capture the interaction of chemistry and mixing over a wide range of timescales. The higher than average errors in H and OH radicals could be a result of system stiffness reduction by the NODE network leading to loss of information for very small timescales. However, further investigations are needed for conclusive elucidation.

### 3.3 Numerical Efficiency of NODE Network.

The computational performance of the NODE network or DKM depends on numerous factors involved in obtaining the solution to Eq. (1) including splitting stepsize, ODE solver type, and error tolerances. The relative (ηr) and absolute (ηa) error tolerances have an impact on how well the adaptive ODE solvers with stepsize control such as LSODA and RK45 perform. The former controls the error in the solution relative to dependent variable values. The latter provides a threshold below which the error is disregarded. This threshold determines the accuracy when the solution approaches zero. In this section, we conduct two studies: first to investigate the performance of the NODE network for varying global time-step or the splitting time-step size, and second to explore the performance scalability of the NODE network with the increased number of chemistry integrations by increasing the number of particles.

In the numerical implementation of the zero-order splitting scheme for reacting flows, it is recommended to select the global time-step such that it is at least an order of magnitude smaller than the mixing timescale, i.e., $dt=110τm$ [6]. This ensures obtaining accurate solutions compared to direct integration of the original fully coupled equations, Eq. (1). Hence, to study the effects of the global time-step, we select a reference case where a PMSR simulation is performed with DKM using a highly accurate implicit BDF solver for $dt=116τm$. For the operating conditions under consideration in this study, the richer mixtures exhibit the worst-case errors as observed in Fig. 4. Thus, for a stricter assessment of the numerical performance of the NODE network, a richer mixture fraction of Z = 0.05 with the specifications of the remaining parameters in Table 1 is selected for further studies. At first, to examine the performance of the NODE network, the global step dt is increased from $dt=116τm$ to $11.1τm$ and performance parameters: normalized CPU time ($Γ^$), the total number of function evaluations (Σ), and relative errors are recorded as shown in Fig. 8. The CPU times are normalized with respect to the maximum CPU time for each plot in Figs. 8 and 9. Parameter Σ indicates the number of times the chemical reaction source term is evaluated by the ODE solver and provides a strong indicator of the numerical stiffness of the system. The tests are carried out for three ODE solvers: hybrid solver LSODA, explicit solver RK45, and lower order explicit Runge–Kutta method of order 3(2) (RK23) [39]. To study the sensitivity of the NODE network to error tolerances, sets of three error tolerances are selected: (ηa, ηr) = (10−5, 10−4), (10−4, 10−4), (10−3, 10−3). For comparison, similar tests are carried out with DKM. It is important to emphasize that the stiffness of DKM does not allow the use of an explicit solver and lower error tolerances. Thus, DKM tests are performed with two solvers: implicit solver BDF and hybrid solver LSODA, for the error tolerances specified in Table 1. The results of these test cases are compared with the reference case. Since the reference and test cases use different time-steps, the resulting total number of time-steps (Nt) in these simulations are different and the global errors cannot be obtained from Eq. (10). Therefore, the relative errors in ignition delay time and equilibrium temperature in these cases are calculated as
$εr,τign=|τign,ref−τign,testdt|$
(11)
$εr,Teq=|Teq,ref−Teq,testTmax−Tmin|$
(12)
Here, τign is defined as the instant when 〈T〉 reaches T0 + 400 K, Teq is the mean temperature of statistically stationary state and Tmax and Tmin are the maximum and minimum temperature values of the training dataset in Eq. (5).
Fig. 8
Fig. 8
Close modal
Fig. 9
Fig. 9
Close modal

Figure 8 shows that irrespective of error tolerance values and ODE solver type, the NODE network is consistently faster than DKM. It requires a fewer number of function evaluations than DKM, indicating a reduction in the stiffness of the system. For DKM simulations with a highly accurate BDF solver the relative error $εr,τign$ tends to increase with the increase in dt, while $εr,Teq$ first increases and then plateaus at $1%$. However, for hybrid LSODA solver, $εr,τign$ increases with dt and $εr,Teq$ has a minimum at $dt=12τm$. The NODE network exhibits similar error behavior for all the solvers and tolerances which establishes the applicability of splitting schemes for the NODE network. This indicates the robustness of the NODE network for various ODE solvers and error tolerances, and their ability to work with larger dt without sacrificing accuracy. The speedup factors of 4 to 6 are recorded with $dt=12τm$ for NODE network cases compared to the corresponding DKM case with BDF solver. Furthermore, even when comparing the DKM case with LSODA, the observed speedup factors range from 2 to 3.

To further assess the scalability of the NODE network with the number of ODE integrations, the number of particles in PMSR is increased from 100 to 10,000 and performance is recorded in Fig. 9 for the same set of ODE solvers and error tolerances as described in Fig. 8. For each simulation, the simulations are progressed in time for 200 time-steps, requiring the total number of ODE integrations of 2 × 104 to 2 × 106. Irrespective of error tolerance values and ODE solvers, the NODE network is consistently faster than the DKM owing to the reduced stiffness of the system. The speedup factors for the NODE network with respect to DKM simulations with BDF solvers remain approximately constant as the number of ODE integrations is increased. However, these factors tend to increase with reduced error tolerances. For example, for the RK45 solvers, the speedup factor of 5 is observed for (ηa, ηr) = (10−5, 10−4), and it is increased to a factor of 8 for (ηa, ηr) = (10−3, 10−3). For the lower order solver RK23, this speedup reaches a factor of 11. The global error in 〈T〉 is the largest for N = 100 which can be partly attributed to the statistical variations of the mean. The mean error reduces to less than $2%$ for N = 10,000, indicating that statistical convergence is achieved with an increase in the number of particles. This statistical convergence can be clearly observed for DKM, as well as the NODE network, in Fig. 9(c).

In summary, this investigation concludes that the NODE network is highly robust, efficient, and accurate in capturing the complex dynamics of chemistry and mixing. NODE network offers great flexibility in the selection of ODE solvers and error tolerances to yield an order of magnitude speedup compared to highly accurate DKM solutions using implicit ODE solvers. This indicates the great potential of NODE in accelerating computations in turbulent combustion applications.

## 4 Conclusions

In this study, we illustrate the methodology of coupling chemical kinetics, formulated with NODE, with mixing to simulate mixing/reacting systems in the context of PDF methods. A proof-of-concept is demonstrated in hydrogen-air combustion in the PMSR which corresponds to zero-dimensional PDF calculations. The hydrogen-air chemistry is described by a finite-rate mechanism involving 9 chemical species and 21 reaction steps. Results show that the NODE simulations accurately capture the combustion characteristics of PMSR for a wide range of composition spaces and mixing timescales. It is observed that in extreme conditions such as larger residence and mixing times the NODE introduces relatively larger errors in H and OH radicals which are speculated to be the effect of stiffness reduction. Nevertheless, these errors do not have a significant effect on global combustion characteristics including ignition delay and equilibrium temperatures. Subsequent investigations to evaluate the numerical efficiency of the NODE network affirm the robustness of NODE to larger splitting time-steps, ODE solver type, and error tolerances. Despite the chosen solver and tolerances, the NODE network is shown to be faster than respective DKM simulations. In the best case, with a selection of optimal solvers and tolerances, the NODE network achieves 11 times speedup for comparable accuracy to DKM simulations under specified operating conditions in this study. This analysis demonstrates the accuracy and computational efficiency of the NODE network in describing the dynamics of reacting systems in the presence of mixing. This warrants further development and application of the proposed methodology for large-scale turbulent combustion simulations.

## Acknowledgment

This study is supported in part by the Office of the Vice President for Research at the University of Connecticut through the Research Excellence Program. The authors acknowledge the computing resources provided by the high-performance computing facilities at the University of Connecticut.

## Conflict of Interest

This article does not include research in which human participants were involved. Informed consent not applicable. This article does not include any research in which animal participants were involved.

## Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

## References

1.
Givi
,
P.
,
2006
, “
Filtered Density Function for Subgrid Scale Modeling of Turbulent Combustion
,”
AIAA J.
,
44
(
1
), pp.
16
23
.
2.
Pope
,
S. B.
,
1985
, “
PDF Methods for Turbulent Reactive Flows
,”
Prog. Energy Combust. Sci.
,
11
(
2
), pp.
119
192
.
3.
Pope
,
S. B.
,
2000
,
Turbulent Flows
,
Cambridge University Press
,
Cambridge
.
4.
Pope
,
S. B.
,
1981
, “
A Monte Carlo Method for the PDF Equations of Turbulent Reactive Flow
,”
Combust. Sci. Technol.
,
25
(
5–6
), pp.
159
174
.
5.
Yang
,
B.
, and
Pope
,
S.
,
1998
, “
An Investigation of the Accuracy of Manifold Methods and Splitting Schemes in the Computational Implementation of Combustion Chemistry
,”
Combust. Flame
,
112
(
1–2
), pp.
16
32
.
6.
Saxena
,
V.
, and
Pope
,
S. B.
,
1999
, “
PDF Simulations of Turbulent Combustion Incorporating Detailed Chemistry
,”
Combust. Flame
,
117
(
1–2
), pp.
340
350
.
7.
Pope
,
S. B.
,
1997
, “
Computationally Efficient Implementation of Combustion Chemistry Using In Situ Adaptive Tabulation
,”
Combust. Theory Modell.
,
1
(
1
), pp.
41
63
.
8.
Lu
,
T.
, and
Law
,
C. K.
,
2005
, “
A Directed Relation Graph Method for Mechanism Reduction
,”
Proc. Combust. Inst.
,
30
(
1
), pp.
1333
1341
.
9.
Vajda
,
S.
,
Valko
,
P.
, and
Turanyi
,
T.
,
1985
, “
Principal Component Analysis of Kinetic Models
,”
Int. J. Chem. Kinet.
,
17
(
1
), pp.
55
81
.
10.
Lu
,
T.
,
Ju
,
Y.
, and
Law
,
C. K.
,
2001
, “
Complex CSP for Chemistry Reduction and Analysis
,”
Combust. Flame
,
126
(
1–2
), pp.
1445
1455
.
11.
Lu
,
T.
, and
Law
,
C. K.
,
2006
, “
Systematic Approach to Obtain Analytic Solutions of Quasi Steady State Species in Reduced Mechanisms
,”
J. Phys. Chem. A
,
110
(
49
), pp.
13202
13208
.
12.
Rein
,
M.
,
1992
, “
The Partial-Equilibrium Approximation in Reacting Flows
,”
Phys. Fluids A
,
4
(
5
), pp.
873
886
.
13.
Law
,
R.
,
Metghalchi
,
M.
, and
Keck
,
J. C.
,
1989
, “
Rate-Controlled Constrained Equilibrium Calculations of Ignition Delay Times in Hydrogen-Oxygen Mixtures
,”
Proc. Combust. Inst.
,
22
(
1
), pp.
1705
1713
.
14.
Keck
,
J. C.
,
1990
, “
Rate-Controlled Constrained-Equilibrium Theory of Chemical Reactions in Complex Systems
,”
Prog. Energy Combust. Sci.
,
16
(
2
), pp.
125
154
.
15.
Hamiroune
,
J.
,
Bishnu
,
P.
,
Metghalchi
,
M.
, and
Keck
,
J. C.
,
1998
, “
Rate-Controlled Constrained-Equilibrium Method Using Constraint Potentials
,”
Combust. Theor. Model.
,
2
(
1
), pp.
81
94
.
16.
Janbozorgi
,
M.
,
Ugarte
,
S.
,
Metghalchi
,
H.
, and
Keck
,
J. C.
,
2009
, “
Combustion Modeling of Mono-carbon Fuels Using the Rate-Controlled Constrained-Equilibrium Method
,”
Combust. Flame
,
156
(
10
), pp.
1871
1885
.
17.
,
F.
,
Janbozorgi
,
M.
,
Sheikhi
,
M. R. H.
, and
Metghalchi
,
H.
,
2016
, “
A Study of Interactions Between Mixing and Chemical Reaction Using the Rate-Controlled Constrained-Equilibrium Method
,”
J. Non-Equilibrium Thermodyn.
,
41
(
4
), pp.
257
278
.
18.
,
F.
,
Yu
,
G.
, and
Metghalchi
,
H.
,
2018
,
Fundamentals of Rate-Controlled Constrained-Equilibrium Method
,
Springer
,
Singapore
.
19.
Maas
,
U.
, and
Pope
,
S. B.
,
1992
, “
Simplifying Chemical Kinetics: Intrinsic Low-Dimensional Manifolds in Composition Space
,”
Combust. Flame
,
88
(
3–4
), pp.
239
264
.
20.
Chen
,
J.-Y.
,
Kollmann
,
W.
, and
Dibble
,
R.
,
1989
, “
PDF Modeling of Turbulent Nonpremixed Methane Jet Flames
,”
Combust. Sci. Technol.
,
64
(
4–6
), pp.
315
346
.
21.
Blasco
,
J. A.
,
Fueyo
,
N.
,
Dopazo
,
C.
, and
Ballester
,
J.
,
1998
, “
Modelling the Temporal Evolution of a Reduced Combustion Chemical System With an Artificial Neural Network
,”
Combust. Flame
,
113
(
1–2
), pp.
38
52
.
22.
Christo
,
F. C.
,
Masri
,
A. R.
,
Nebot
,
E. M.
, and
Pope
,
S. B.
,
1996
, “
An Integrated PDF/Neural Network Approach for Simulating Turbulent Reacting Systems
,”
Symp. (Int.) Combust.
,
26
(
1
), pp.
43
48
.
23.
Blasco
,
J. A.
,
Fueyo
,
N.
,
Larroya
,
J. C.
,
Dopazo
,
C.
, and
Chen
,
Y. J.
,
1999
, “
A Single-Step Time-Integrator of a Methane-Air Chemical System Using Artificial Neural Networks
,”
Comput. Chem. Eng.
,
23
(
9
), pp.
1127
1133
.
24.
Chen
,
J. Y.
,
Blasco
,
J. A.
,
Fueyo
,
N.
, and
Dopazo
,
C.
,
2000
, “
An Economical Strategy for Storage of Chemical Kinetics: Fitting In Situ Adaptive Tabulation With Artificial Neural Networks
,”
Proc. Combust. Inst.
,
28
(
1
), pp.
115
121
.
25.
Blasco
,
J. A.
,
Fueyo
,
N.
,
Dopazo
,
C.
, and
Chen
,
J.
,
2000
, “
A Self-Organizing-Map Approach to Chemistry Representation in Combustion Applications
,”
Combust. Theory Modell.
,
4
(
1
), p.
61
.
26.
Nguyen
,
H.-T.
,
Domingo
,
P.
,
Vervisch
,
L.
, and
Nguyen
,
P.-D.
,
2021
, “
Machine Learning for Integrating Combustion Chemistry in Numerical Simulations
,”
Energy AI
,
5
, p.
100082
.
27.
Chatzopoulos
,
A. K.
, and
Rigopoulos
,
S.
,
2013
, “
A Chemistry Tabulation Approach Via Rate-Controlled Constrained Equilibrium (RCCE) and Artificial Neural Networks (ANNs), With Application to Turbulent Non-premixed CH4/H2/N2 Flames
,”
Proc. Combust. Inst.
,
34
(
1
), pp.
1465
1473
.
28.
Franke
,
L. L.
,
Chatzopoulos
,
A. K.
, and
Rigopoulos
,
S.
,
2017
, “
Tabulation of Combustion Chemistry Via Artificial Neural Networks (ANNs): Methodology and Application to LES-PDF Simulation of Sydney Flame L
,”
Combust. Flame
,
185
, pp.
245
260
.
29.
Ji
,
W.
, and
Deng
,
S.
,
2021
, “
KiNet: A Deep Neural Network Representation of Chemical Kinetics
,”
arXiv Preprint
. https://arxiv.org/abs/2108.00455
30.
Chen
,
R. T. Q.
,
Rubanova
,
Y.
,
Bettencourt
,
J.
, and
Duvenaud
,
D.
,
2018
, “
Neural Ordinary Differential Equations
,”
,
31
, pp.
6571
6583
.
31.
Owoyele
,
O.
, and
Pal
,
P.
,
2021
, “
ChemNODE: A Neural Ordinary Differential Equations Framework for Efficient Chemical Kinetic Solvers
,”
Energy AI
,
7
, p.
100118
.
32.
Bansude
,
S.
,
Imani
,
F.
, and
Sheikhi
,
R.
,
2022
, “
A Data-Driven Framework for Computationally Efficient Integration of Chemical Kinetics Using Neural Ordinary Differential Equations
,”
SSRN Electronic Journal
.
33.
Dikeman
,
H. E.
,
Zhang
,
H.
, and
Yang
,
S.
,
2022
, “
Stiffness-Reduced Neural ODE Models for Data-Driven Reduced-Order Modeling of Combustion Chemical Kinetics
,”
AIAA SCITECH 2022 Forum
,
San Diego, CA
,
Jan. 3–7
, p.
0226
.
34.
Rahimi
,
M. J.
,
Esposito
,
G.
,
Chelliah
,
H. K.
,
Hiremath
,
V.
, and
Pope
,
S. B.
,
2013
, “
Chemical Kinetic Model Reduction Based on Partially-Stirred Reactor Simulations With Comparable Chemical and Mixing Time Scales
,”
51st AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition 2013
,
Grapevine, TX
,
Jan. 7–10
, pp.
1
10
.
35.
Veynante
,
D.
, and
Vervisch
,
L.
,
2002
, “
Turbulent Combustion Modeling
,”
Prog. Energy Combust. Sci.
,
28
(
3
), pp.
193
266
.
36.
Magnussen
,
B.
,
1981
, “
On the Structure of Turbulence and a Generalized Eddy Dissipation Concept for Chemical Reaction in Turbulent Flow
,”
19th Aerospace Sciences Meeting
,
St. Louis, MO
,
Jan. 12–15
, p.
42
.
37.
Chemical Kinetic Mechanisms for Combustion Applications
,”
San Diego Mechanism web page, Mechanical and Aerospace Engineering (Combustion Research)
,
University of California at San Diego
, http://combustion.ucsd.edu
38.
Boivin
,
P.
,
2011
, “
Reduced-Kinetic Mechanisms for Hydrogen and Syngas Combustion Including Autoignition
39.
Virtanen
,
P.
,
Gommers
,
R.
,
Oliphant
,
T. E.
,
Haberland
,
M.
,
Reddy
,
T.
,
Cournapeau
,
D.
,
Burovski
,
E.
,
Peterson
,
P.
,
Weckesser
,
W.
,
Bright
,
J.
,
van der Walt
,
S. J.
,
Brett
,
M.
,
Wilson
,
J.
,
Millman
,
K. J.
,
Mayorov
,
N.
,
Nelson
,
A. R. J.
,
Jones
,
E.
,
Kern
,
R.
,
Larson
,
E.
,
Carey
,
C. J.
,
Polat
,
İ.
,
Feng
,
Y.
,
Moore
,
E. W.
,
VanderPlas
,
J.
,
Laxalde
,
D.
,
Perktold
,
J.
,
Cimrman
,
R.
,
Henriksen
,
I.
,
Quintero
,
E. A.
,
Harris
,
C. R.
,
Archibald
,
A. M.
,
Ribeiro
,
A. H.
,
Pedregosa
,
F.
, and
van Mulbregt
,
P.
,
2020
, “
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
,”
Nat. Methods
,
17
, pp.
261
272
.
40.
Chen
,
R. T. Q.
,
Amos
,
B.
, and
Nickel
,
M.
,
2021
, “
Learning Neural Event Functions for Ordinary Differential Equations
,”
International Conference on Learning Representations
,
Vienna, Austria
,
May 3–7
.