^{1}

^{1}

^{1}

^{1}

^{1}

For CFD results to be useful in IC engine analysis, simulation results should be accurate and consistent. However, with wide spread use of parallel computing nowadays, it has been reported that a model would not give the same results against the same input when the parallel computing environment is changed. The effect of parallel environment on simulation results needs to be carefully investigated and understood. In this paper, the solution inconsistency of parallel CFD simulations is investigated. First, the concept of solution inconsistency on parallel computing is reviewed, followed by a systematic CFD simulations specific to IC engine applications. The solution inconsistency against the number of CPU cores w as examined using a commercial CFD code CONVERGE. A test matrix was specifically designed to examine the core number effect on engine flow, spray and combustion submodels performance. It was found that the flow field simulation during the gas exchange process is the most sensitive to the number of cores among all submodels examined. An engineering solution was developed where local upwind scheme was used to control the variability, which showed good performance. The implication of the observed inconsistency was also discussed.

Numerical simulation of physical process has been an indispensable asset in product development. In particular, the automotive industries increasingly rely on Computer Aided Engineering (CAE) for product development. For any kind of numerical simulations to be useful in analysis, the solution should exhibit certain properties. The first one is the accuracy. A simulation is expected to reproduce or predict the physical process with acceptable accuracy to be used in analysis or product development. The solution accuracy depends on two factors. One is the model accuracy, which can be attributed to the assumptions and procedures used in the model development. In terms of CFD, Direct Numerical Simulation (DNS) is considered the most accurate, because it adapts the least assumptions in the modeling of fluid dynamics. On the other hand, models based on Reynolds averaging, commonly referred as RANS (Reynolds Averaged Navier-Stokes), adapt time-averaged governing equation with additional assumptions such as homogeneous isotropic turbulence, which makes it inapplicable in accurate modeling of instantaneous flow field, but more suited for an ensemble averaged flow field.

The other required property is the solution consistency. In other words, a simulation code should be able to give the same answer whenever and on which hardware the code is ran. In a serial code, the solution consistency is very good and has not been an issue for most cases. However, with recent wide spread use of parallel computing, it has been noticed and reported that a parallel code may not produce the same results. The solution consistency in parallel computing is not a trivial task and has been a research topic in computer science [

Understanding the sources of such non-deterministic behavior, and the effects thereof, is very important in analyzing the simulation data. This problem is not specific to CFD, but virtually all of parallel scientific computing is affected. Research has been conducted to identify possible sources of variations and how to eliminate them to retain solution determinacy. It has been argued that any parallel scientific computing should be deterministic [

In this study, the solution consistency from parallel CFD simulations was examined to understand how the simulation results are affected by the change in parallel computing environment. The study is focused on modeling the internal combustion engine, which is a highly complex reacting flow. It is a good test case because it includes virtually all aspects of recent CFD modeling, such as moving boundary (valves and pistons), two-phase flow with phase change (fuel injection/spray), and chemical reaction (ignition and combustion). Particularly, the fuel injection model is an interesting topic because it relies on the random number sequence, which is known to be sensitive to the parallel computing environment.

The study is composed as follows. First, the recent research and current understanding of the solution inconsistency in parallel computing is reviewed in Chapter 2. Then, reported solution inconsistency in CFD is reviewed in Chapter 3 with focus on the numerical error and its population in CFD. The solution inconsistency in the CFD of IC engines is examined in subsequent chapters. The numerical setup is summarized in Chapter 4, whose results are described in Chapter 5. The results are analyzed in Chapter 6 and a summary in Chapter 7 will wrap up the study.

In a computer system, numbers are stored with a finite accuracy. Consequently, there are round-off errors in any kind of scientific computing, regardless of being serial or parallel. We will first review what kind of numerical errors are present, and how such errors may impact parallel computing and affect solution determinacy.

A computer uses a floating point number system with finite precision [

One of the most well-known effect of the round-off error is the floating point arithmetic, which is not associative [

Numerical issues in serial computing is rather well defined and understood to come from modeling continuum physics with discretized equation with finite accuracy [

In parallel computing, the computing environment has one additional variable compared to the serial counterpart. It is the parallel computing environment, which includes any parallel setup for a parallel code to run, including the computer algorithms for parallel implementation, parallel execution schedule, operating conditions, compiler, and hardware setup. It has been reported that computing results may vary in a parallel computing when the parallel computing environment is changed. The loss of the determinacy in the parallel scientific computing is an actively researched topic in the computer science [

In a serial computing, the order of calculation in a loop is already determined at compilation. However, in a parallel computing, a loop is split and distributed over a number of cores for better performance. As such, the order of calculation is determined at runtime, based on the number of cores used in the calculation. For example, a summation operation of an array with N element in a serial code will occur serially from element 1 to N. In a parallel computation with 2 cores will split the loop in two, which will first calculate summation from (1 to N/2) and (N/2 + 1 to N), then the two sums will be added. As such, any variation in the number of processor will vary the order of addition at runtime. The order of arithmetic can be varied even with the same number of cores, dependent on the overhead of each node. Say, let’s assume that a simulation was repeated with three cores each time. In one run, core 0 finished first, followed by core 1 and core 2. The order of summation will be (sum from core 1) + (sum from core 2) + (sum from core 3). In another run, core 0 had overhead from other process, and ended last. In this case the total value would be (sum from core 2) + (sum from core 3) + (sum from core 1). In either case, the order of arithmetic is varied at runtime and the final sum will be affected by the non-associativity. As described before, the variation from non-associativity occurs mostly at machine precision limit, which is 15 - 16 digits for double precision in most of numerical simulations.

Another possible source of additional numerical error in parallel computing is the random number generation. The random number is used extensively in CFD simulation, particularly for modeling spray. A random number is generated by a pseudo-random number generator, which provides a sequence of random numbers based on its initial seed value. If the initial seed value is the same, resulting random number sequence will be identical. In most of scientific computing, the seed number is held constant for the sake of determinacy, except in some statistical analysis. In a serial environment, as the order of computation is predetermined during compilation, elements in a random number sequence were used sequentially in any single simulation in a deterministic fashion.

However, in a parallel environment, it is not clear how a random number sequence will be used over multiple processes. One option is a replicated approach, where each processor keeps the same random number sequence and use its own. Another options is a distributed approach, where one processor keeps the random number sequence and sends out each element to each core whenever a random number is needed. In either approach, it is clear that the usage of the random number element will vary when the number of processors is changed.

Other possible sources for the non-determinacy have been investigated, e.g., the effect of cut-cell method on numerical oscillation [

As the numerical errors in scientific computing has been described, now we focus on the numerical errors and their behavior specific to the CFD modeling. The effect of numerical error in serial CFD has been reviewed by Freitas [

The viscosity in any CFD simulation can be categorized into three viscosities [

With the three viscosities, the Navier-Stokes equation in CFD can be represented as the following with an effective viscosity, ν_{eff}.

∂ u ∂ t + u ⋅ ∇ u = ∇ P ρ + ν e f f ∇ 2 u (1)

ν e f f = ν + ν N + ν t (2)

In Equation (1), u is the velocity and P is the pressure. In Equation (2), v, v_{N}_{,} and v_{t} represents molecular viscosity, numerical viscosity and turbulent viscosity, respectively. If the viscosity is very low to the physical viscosity, the simulation will provide the most accurate results. However, the numeric will be sensitive to small oscillations and the code can be unstable. Consequently, the code will require much finer time step and grid size, which will make overall simulation very expensive. This is what is called the direct numerical simulation (DNS). On the other hand, if the effective viscosity is much higher than the modeled flow, CFD simulation results will be dampened out and fail to reproduce the physics. Typical engineering type simulations are based on RANS formulation, where large amount of turbulence viscosity is used for a good compromise in numerical stability and accuracy. For higher accuracy and time-resolved simulation, a large eddy simulation (LES) is used, where the turbulence is modeled only at sub-grid level, which typically has much smaller turbulent viscosity than RANS. Poinsot [

At the DNS level, the code uses very fine mesh to resolve the smallest flow structure where viscous dissipation occurs. As it resolves the viscous dissipation directly, it requires no model about the turbulent viscosity, and its value is zero. It requires very fine numeric to resolve the small scale structure. Along with the fine mesh, the numerical viscosity is very small close to zero. In LES, the turbulence in small structure is modeled by the turbulent viscosity. The grid size is a bit larger than DNS, which invokes small amount of numerical viscosity as well.

However, as summarized in

Model | ν N ν : Numerical viscosity | ν t ν : Turbulent viscosity |
---|---|---|

DNS | ~0 | 0 |

LES | 1 - 10 | 5 - 50 |

RANS | 10 - 500 | 100 - 1000 |

larger by an order of magnitude. Consequently, Poinsot concluded that RANS simulation may not be able to propagate small numerical error [

Numerical modeling of internal combustion engine requires modeling of reacting flow with often two-phase fluid including phase change over very complex geometry with moving boundaries. A number of different sub-models have been developed and implemented by different researchers, which makes evaluation of the numerical error not a trivial task.

To simplify the analysis, numerical tests are designed to start from one model to the combination of different models. Total of seven numerical test cases were designed, which are summarized in

The test cases are first grouped in closed and open cycle simulations. A diagram is shown in

In the closed cycle simulations, the simulation period does not involve the gas exchange such as intake and exhaust. This setup would let us investigate each individual sub models such as spray, spark and combustion models separate from complex gas exchange process. The first case in closed cycle (Case C1) is spray modeling, which is actually in a spray vessel, rather than engine. The spray vessel has a fixed volume and no moving boundary, and it is expected to serve as an ideal condition to examine the numerical errors from non-associativity and, in particular, the parallel random number effect on spray modeling. Starting from the spray vessel case, different cases were setup with additional model complexity. Combustion was added to the spray in the diesel combustion case (Case C2). And spark ignition is added to the spray and combustion models in gasoline combustion case (Case C4). To separately examine the spark ignition model from spray, a homogeneous spark ignition case was included (Case C3).

In the open cycle case, we started with examining only the flow field in both diesel (O1) and gasoline engine (O2) geometry. In Gasoline case (O2), spray is added for modeling. Then, a full cycle simulation (except exhaust process) of gasoline direct injection spark ignition case was examined, which is Case O3.

Case | Hardware | Intake | Spray | Spark | Combustion | |
---|---|---|---|---|---|---|

Closed Cycle | C1 | Spray Bomb | N | Y | N | N |

C2 | Diesel Engine | N | Y | N | Y | |

C3 | Gasoline Engine A | N | N | Y | Y | |

C4 | Gasoline Engine B | N | Y | Y | Y | |

Open Cycle | O1 | Diesel Engine | Y | N | N | N |

O2 | Gasoline Engine A | Y | N | N | N | |

O3 | Gasoline Engine B | Y | Y | Y | Y |

Case O3 has the most complete set of numerical models, which includes intake flow modeling with valve and piston motion, spray injection, spark ignition and combustion. This will be a good case to examine the effect of numerical error from parallel CFD code on overall IC engine modeling.

For all cases, parallel computing environment was varied by changing the number of cores. Otherwise, the computing environments are kept the same for each test.

CONVERGE of Convergent Science Inc. [

CONVERGE generates computational mesh on-the-fly. The computational mesh is Cartesian with boundary cells fitted to the geometry by cut-cell method [

Numerical diffusion from discretization is a key factor which can affect the effect of any numerical errors. CONVERGE provides two numerical schemes. One is central differencing scheme (CDS), which is 2nd order with minimum numerical diffusion. The other is upwind differencing scheme (UDS), which adds significant amount of numerical diffusion to solution. As described in any numerical analysis textbooks [

The operating conditions for the test matrix is summarized in

Case C1: Spray bomb

As discussed in introduction, two sources of numerical errors are considered in this study. One is the numerical error from non-associativity and the other is parallel random number generation. As the spray model is heavily dependent on random number to model particle physics, care has been taken to examine the effect of processor numbers and subsequent numerical error in the spray vessel case. The spray characteristics from the simulations are plotted in

First, the spray penetration is compared in

Case | Hardware | MAP | RPM | |
---|---|---|---|---|

Closed Cycle | C1 | Spray Bomb | N/A | N/A |

C2 | Diesel Engine | 280 kPa | 3750 | |

C3 | Gasoline Engine A | 95 kPa | 1300 | |

C4 | Gasoline Engine B | 95 kPa | 1300 | |

Open Cycle | O1 | Diesel Engine | N/A | N/A |

O2 | Gasoline Engine A | 95 kPa | 1300 | |

O3 | Gasoline Engine B | 95 kPa | 1300 |

shows that the penetration length is hardly affected by the number of processors. Similar trends were observed in other key characteristics, such as vaporized fuel mass (

Often times, overall characteristics such as penetration may not be a good measure for comparing spray simulation results. Even though overall characteristics are similar, spatial distribution and spray structure may be different. The spray structure is compared in

On the top two rows are the comparison of liquid phase parcels with three different processor numbers at different simulation times. It is clearly shown that the liquid spray structure is not affected by the number of CPUs. The gas phase fuel species concentration is compared in the bottom row. The overall structure is very similar with different process numbers. It is noted here that the contour level was set up very narrow to examine difference in fuel concentration. The contour level in fuel mass fraction is also shown in the same figure. So even though there are some difference in the fuel concentration contour, absolute variation with the number of CPU is relatively small.

Case C2: Spray + Combustion (Diesel Combustion)

The effect of numerical error from different number of CPUs is examined for a combination of spray and combustion model in diesel combustion (Case C2). The results are compared in

Case C3: Ignition and Combustion (Gasoline Combustion)

The gasoline combustion case is shown in

Case C4: Spray, Ignition and Combustion (Gasoline DISI Combustion)

This is the most complex case in the closed cycle simulation. It involves spray, ignition and combustion model in a direct-injection spark-ignition engine. Injection occurs 50 CA bTDC, followed by spark at 37 CA bTDC. The same ignition model from Case C3 is used with 60mJ of ignition energy. The results are compared in

The results, again shows virtually no dependency to the number of cores and related numerical error. As a quick summary, closed cycle RANS simulations showed very little sensitivity to the core numbers and changes in the parallel computing environment. The spray vessel case is particularly interesting. With

different number of CPUs, the random number sequence should have been different with the number of CPUs. Subsequently the spray model was expected to be affected, showing variation to processor numbers. However, it was found that the spray modeling results were virtually unaffected. It can be interpreted that the randomness of the random number sequence is maintained in the parallel computing. Ignition and combustion models did not show any dependency to the numerical error.

Case O1: Gas Exchange in Diesel Port Flow

A pure flow simulation was carried out over a diesel port flow geometry. There is no spray event, and chemical kinetics model is completely turned off. The results are shown in

As shown in

As the variation in simulation results start to appear near the maximum valve lift, CFD simulation results are visualized to examine the flow field near the maximum valve lift. The velocity fields from 16 and 32 CPU cases are shown in

Surprisingly, it shows two very distinct flow patterns even with high viscosity from RANS simulations. When 16 CPU is used (left plot in

distance. On the other hand, when 32 CPU is used with exactly the same set of input files, the jet from the valve was found to penetrate much shorter distance. Considering that more complex cases (spray, ignition and combustion) did not show any dependency to the number of CPUs, it is quite surprising that the flow field shows variation with the processor numbers, especially with RANS modeling.

It was questioned if the number of CPU dependency is due to a particular geometry. A gasoline engine geometry was used to examine the number of CPU dependency (not listed in

The gasoline port flow simulation shows similar results. Both swirl (

Case O2: Gas Exchange in Gasoline Port Flow with Spray

As mentioned previously, the gasoline engine takes relatively homogenous mixture for combustion. For the purpose, the injection occurs during the intake process, either into the port or cylinder. The port flow simulation without spray showed some variation. And it is investigated how the spray will behave in the CPU-number dependent flow field. The results are shown in

The results in

generate 48 CPU results by changing input and running with 32 CPU, and vice versa. It is clear that the variation in flow field affects the spray simulation, which alone is not sensitive to the change in parallel computing environment.

Case O3: Gasoline DI SI combustion with Intake Process

This is the case with largest model complexity, including flow through moving valves, piston motion, spray, spark ignition and combustion. As discussed in the closed cycle results, most of sub-models have shown virtually no dependency to the number of cores when there is no open boundaries. On the other hand, the flow field showed sensitivity to the core numbers particularly in valve flow simulation. The results are shown in

From previous port flow simulation and port flow + spray simulation, it was already observed that mixing field is already affected by the changes in parallel computing environment. As such, it was expected that combustion event is similarly affected.

As the number of processor is changed, combustion phasing and peak pressure, heat release profile are all varied.

An engineering approach was tried to reduce the variability. In all simulations, the central difference scheme (CDS) was used. However, when the solution field shows strong gradient, upwind difference scheme (UDS) was used locally. This is a common engineering approach in most CFD simulations, and details can be found in the CONVERGE documentation. As the CDS with local UDS showed non-determinacy, a different approach was developed and implemented into CONVERGE. In the old approach, the local UDS was applied based on variable. For example, if a computational cell shows strong gradient in x-direction velocity u, u was discretized by UDS, while all other variable are modeled by CDS. In the new approach, the local UDS was applied based on the computational cell. If any of the three velocity components show strong variability, the particular cell is discretized UDS in all directions. The flow and combustion simulations results are shown in

The results clearly show improvement in the determinacy, showing much smaller variation in the results against different number of cores. The new method is available in CONVERGE version 2.3.

It was found that the RANS simulation in closed cycle simulation (i.e., no open boundary) is not affected by the number of cores and additional numerical errors with the current code CONVERGE. One of the most interesting results is the spray simulation with quiescent ambient background, case C1. The case is interesting because it is a good test bed to examine the effect of random number sequence in the parallel computation. And it was found that the effect of random number is small enough to be neglected. However, it is noted here that the

finding should not be considered as a general conclusion. The spray simulation is affected by a number of different factors, such as the random number generator and the sample size (parcel count). In the current simulation, rather a large sample size of 105 to 106 was used. So, it would be fair to say that the current spray simulation was not affected by the parallel computing environment because a large number of parcels (lager sample size) is used. Albeit there are such caveats, the results that parallel random number sequence doesn’t affect the results when the sample size is large enough, is very encouraging.

In addition to spray model, ignition and combustion models were added in case C2 (diesel compression ignition) and C3 (gasoline spark ignition). In both cases, the results are the same with different number of CPUs. From these results, it can be interpreted that the RANS simulation is not affected by the additional numerical errors from the change in parallel computing environment.

One important finding is that the additional numerical errors from parallel computing environment variation do not affect results even with the central differencing scheme. As discussed above, the numerical errors are typically very low, which is thought to be easily smeared out by any diffusion, either from the numerical viscosity or the turbulent viscosities. As reviewed in the introduction (refer to

In these simulations, only the momentum equation was discretized with central difference. All other passive variables, including turbulence kinetic energy, dissipation rate and temperature, were discretized in upwind differencing, which may add artificial viscosity to the solution. To clarify, the same simulation was carried out with central difference for all variables. The results are shown in

It is clearly shown that the numerical sensitivity does not show up with central difference for all variables, and the previous conclusion remains the same. In parallel RANS simulation, the additional viscosity from turbulence model is enough to reduce the additional numerical error from parallel computing environment variation and ensure determinacy.

Unlike closed cycle simulations, all of open cycle simulations showed sensitivity to the number of cores and variability in results. It is interesting to note that such strong variation is only observed during open cycle simulation, and does not diffused out by the additional eddy viscosity. Moreover, the variability only appears during when there is flow through the valve. The CFD visualization in

flow simulation. The impact of the flow variability on other models are significant, as it has been shown in case O2 (flow and spray) and O3 (flow, spray, spark ignition and combustion.).

One may question the spray model and the random number sequence for the simulation variability. To address this issue, test O2 was repeated with spray replaced by a gas jet. Instead of spray parcels, fuel vapor was introduced as external source into equivalent multi-hole spray region. So the effect of random number sequence is not present in the new test. The results are shown in

The solution variability observed in the open cycle simulations starts from the intake process to grow over the rest of the cycle, which is similar to what was found in LES simulations [

A thorough error quantification study should reveal the source of such strong non-determinacy during the intake process, which is beyond the scope of the current study. The authors propose possible hypothesis to the source of the errors, which is non-associative floating arithmetic and numerical oscillation from the cut-cell method. Let us first take a look at a typical velocity field during the intake process, as shown in

One of the characteristics of the intake process is the large velocity gradient over the flow through the valve. It can be expected that the local residual at such large gradient can be quite large, while other region, especially the in-cylinder mixture apart from the intake jet, might have low local residual. The global residual, which is the sum of the local residual, will be a classic example of numerical errors from the non-associative floating number arithmetic. It is clear that different arithmetic will be used when a different domain decomposition is used. However, as aforementioned, the numerical error from the non-associative arithmetic is very small at the machine limit and might be diffused out easily by the model viscosity in RANS simulations. On the other hand, we discussed large unphysical oscillation in the cut-cell method [

Non-determinacy was observed from the RANS simulation using a commercial CFD code. The question is how to understand such variability. Attempts have been made to correlate the simulation variability to experimental variability and explain the simulation variability as something physical, such as cyclic

variability of an IC engine observed from experiments. However, this approach requires further understanding of the numerical methods in the CFD code.

Any engine experiment data will exhibit some level of cyclic variability at the most stable operating condition. Such variability has been one of the major topic in engine research, and often attributed to a number of sources such as variability in flow, spark ignition, and/or direct injection. To simulate such variability, the model should be able to capture such variability. As such, LES has been considered as a proper tool for modeling the cyclic variability. On the other hand, in general, RANS has been considered an improper tool to model the variability. It relies on very strong model viscosity, which washes out any small structure [

Apart from the model viscosity, one might be tempted to correlate the variability observed in parallel RANS simulation to that of variability from experiment. However, a thorough error analysis is pre-requisite for such approach. It is reminded that getting similar trend with the experiment does not always guarantee the validity of the methodology. Addition of two errors may cancel each other to make results look good, but it does not make the method right. The variability in experiment comes from various sources, such as variation in experimental boundaries, initial conditions, and ambient conditions. On the other hand, particularly from the parallel RANS simulations in this study, the most significant source of error is the valve flow, and it is purely numerical. The variability from experiment has hardly any correlation to the variability appears in numerical simulations. Moreover, it is important to notice that variability in parallel computation will exhibit itself in simulation of (rather ideal) perfectly deterministic experiment. Correlating variability between numerical simulation and physical experiment can lead to unphysical answers and should not be practiced by any CFD researchers. One may consider the numerical error from parallel computation as an uncertainty quantification analysis. However, uncertainty quantification requires detailed and careful design of the uncertainty factor [

The analysis of variability in parallel RANS simulations requires further research to be fully understood. Until the numerical errors are fully quantified and analyzed, any attempt to correlate the non-determinacy to any physically observed variation should be avoided.

Numerically, the RANS formulation relies on the strong numerical diffusivity, which washes out any small numerical noise. Considering the high level of numerical diffusion, the non-determinacy observed in the current study may not be explained solely by the numerical errors in the parallel computing environment, and further study and analysis will be required for full understanding. A thorough study on the source and effect of the determinacy should be followed. The authors proposed a hypothesis that the large unphysical error from cut-cell method may propagate throughout the computing domain via the non-associative floating number arithmetic.

Analysis of the determinacy should be carried out carefully not to be misguided into unphysical conclusion. One may be tempted to correlate the solution variability from the simulations to the uncertainties in the measurement. However, there exists no physical nor statistical correlation between the physically observed variability to the numerical solution variability. In addition, RANS model does not have a mechanism to transfer the small errors and subsequent variability. As such, the variability in the solution observed in the current RANS simulations can be interpreted that the numerical error generated solely from the parallel computing environment is quite large. Such error is purely numerical, and should have no physical meaning. Source of such strong numerical error should be identified and resolved for accurate modeling. As a conclusion, the variability in RANS should be carefully investigated if it is physical or purely numerical prior to any analysis using the RANS results.

The solution determinacy, which is the critical issue in parallel CFD simulations, has been investigated in this study. First, general numerical error in any scientific computing, namely the round-off error, was reviewed. In a serial computing, the round-off error is deterministic. In a parallel computing, the round-off error may grow differently when the parallel computing environment is changed (i.e., different number of processors). Such behavior is due to the non-associativity of the floating point arithmetic. Other numerical errors related to parallel computing were also reviewed, such as the random number sequence. From the literature survey, it was learned that any parallel scientific computation will experience new issues with parallelism, particularly with respect to the determinacy.

Then, the effect of the numerical error and the effect of viscosity in CFD was reviewed. Interestingly, there have been no reports about the effect of parallelization on RANS simulations to the authors’ knowledge. The only related work is the LES simulations with the RANS consideration, where it has been shown and discussed that a LES simulation may be sensitive to parallel parameters, and may generate different instantaneous solution fields [

The parallel computing environment was varied by changing the number of cores. It was found that RANS simulation results using CONVERGE showed variations in the results, thus the RANS simulation is not deterministic. As the IC engine simulation includes a number of sophisticated sub-models, the authors tried small problems to identify which sub-model is the most sensitive to the change in the parallel computing environment. It was found that the spray and combustion models showed deterministic behavior, while the flow simulation is very sensitive to the changes in the parallel environment. The variability is purely numerical and does not have any physics behind it, and it should not be correlated against experimentally observed cyclic variability.

The authors thank Convergent Science Inc. for technical discussion and implementing the new discretization method.

Keum, S., Grover Jr., R.O., Gao, J., Yang, X.F. and Kuo, T.-W. (2017) Effect of Parallel Computing Environment on the Solution Consistency of CFD Simulations―Focused on IC Engines. Engineering, 9, 824-847. https://doi.org/10.4236/eng.2017.910049