30. Models vs. Data 3 by MIT OpenCourseWare

Summary by www.lecturesummary.com: 30. Models vs. Data 3 by MIT OpenCourseWare

0:00 - Introduction

• Welcome and Topic Overview:

  • The lecture starts with a welcome, mentioning it's a special class with fewer participants.
  • The topics for discussion are models versus data and handling really big boundary value problems.

  [0:35] - Models vs. Data

  Model Consistency and Best Fit Parameters

  • A key question is whether a model is consistent with the data.
  • Consistency can be tested with a chi-squared (kai square) test.
  • A maximum tolerable kai square value (Q) is set.
  • If the kai square of best fit parameters is below this maximum (Q), the model is deemed plausible or consistent with the data.
  • If all possible values of kai square are greater than Q, the model is false, or there is an issue with the experiment.
  • A main problem is to determine the very best fit parameters values.
  • In simple problems involving linear dependence upon parameters, there's usually one best fit when equations are not singular.
  • More typically, there are singular or near-singular equations and nonlinear dependence of model predictions on parameters.
  • Nonlinear dependence is exhibited by rate coefficients (K's) in chemical kinetics, which become nonlinear upon integration of the differential equations.

  [2:10] - Models vs. Data

  Minima in Kai Square Surface and Global Optimization

  • Due to nonlinear dependence, the kai square of parameters (theta) is a nonlinear function.
  • This function generally has many local minima instead of one global minimum.
  • The global minimum is the best possible fit.
  • It's hard to know how many minima there are and if the global minimum has been discovered.
  • Discovery of only local minima higher than the maximum tolerable kai square does not ensure the model is false, since the actual global minimum can be lower than the limit.
  • The solution to this challenge is having a guaranteed method of arriving at the global optimum.

  [3:50] - Models vs. Data

  Case Studies and Global Optimization Techniques

  • There are methods for determining the global optimum, with some researchers being experts in this field, like Professor Barton.
  • Singer, Barton, Fiske, and the author used global optimization on kinetic models.
  • A case study includes experimental data for James Taylor with error bars, at numerous time points.
  • In a 2004 paper, James Taylor asserted that a model was incorrect based on his local fits, which were beyond error bars.
  • Applying global optimization to the same model and data, Adam Singer obtained a true best fit which passes right through all the points.
  • This demonstrated that the initial publication stating the model was refuted was utterly incorrect.
  • In a second case study, James' local best fit appeared quite acceptable by eye, but a statistical test revealed only a 5% probability of observing such data if the model were correct.
  • Global optimization indicated James' fit was near the global optimum, but even the best fit left only a 16% probability that the data were measured if the model held.
  • The inference was that the data were "fishy" for the second case.
  • This illustrates the value of locating the global optimum and also provokes questions regarding interpreting fit results.
  • Stochastic approaches can usually locate the global minimum, yet a guaranteed global minimum demands committed global optimization techniques.
  • Professor Barton has a distributed code named GDOC for kinetic model global optimization.
  • The difficulty with global optimization is that it scales exponentially with the number of adjustable parameters.

  [8:25] - Models vs. Data

  • Interpreting results near kai square max and confidence intervals involves various situations:

    • When kai^2 best is considerably greater than kai^2 max, the model is ready to be refuted by data.
    • If kai^2 best is considerably lower than kai^2 max, model and data are consistent. Next, estimate confidence intervals on the adjusted parameters (thetas).
    • A troublesome case occurs if kai^2 best is lower but very close to kai^2 max.
    • If kai^2 max permits a large region of theta values to produce good fits, confidence intervals are wide.
    • If kai^2 best is almost equal to kai^2 max, the region of theta values that fits the data is very small.
    • In this erroneous example, using the standard procedure to compute confidence intervals produces very narrow intervals that indicate high accuracy in parameter estimation, which is misleading.

    Methods for Addressing Issues

    • Approach 1 (Believe the Model): If one is certain the model is right, the data or error bars can be doubted. Tight confidence intervals signal high precision, warranting publication.
    • Method 2 (Conservative/Numerical Recipes): Calculate confidence intervals according to the criterion kai^2 of theta < kai^2 best fit + kai^2 max for wider intervals, recognizing potential difficulties.

    Operator Splitting

    The discussion turns to operator splitting for solving large boundary value problems.

    • A case in point is the species conservation equation for reacting flow, involving convection, diffusion, and reaction terms.
    • Problems typically consist of a large number of species (e.g., 200) combined with momentum and continuity equations.
    • The equations are stiff due to differing time scales, specifically quick chemistry time scales vs. simulation times.
    • There are large differences in spatial scales, necessitating a tremendous number of mesh points (e.g., 10¹² in 3D).
    • The number of state variables can be huge (e.g., 200 * 10¹² = 10¹⁴).
    • This number of state variables outpaces standard computer memory (RAM) (e.g., 16 GB ~ 10¹⁰ bytes).

    Solving Huge Problems

    There are two general schools of thought for solving these huge, stiff problems:

    • **School 1: Employ Government Supercomputers:** Construct the largest possible computer to hold the entire state vector (10¹⁴ elements) and parallelize calculations.
    • One serious drawback is the enormous amount of data output (e.g., 10¹⁸ numbers), often requiring distributed storage.

    Approximation Methods

    **School 2: Use Approximation Methods:** Develop methods to solve equations without assuming huge supercomputer power.

    • **Method of Approximation 1: Reduce Spatial Mesh (Turbulence):** Employ a coarser mesh and subgrid scale models to represent small eddies.
    • **Method 2: Reduce Time Mesh (Stiffness):** Differentiate fast time scales (chemistry) from slow time scales (transport).
    • Employ stiff solvers that permit far larger time steps without numerical instability.
    • This time step reduction helps considerably save data that needs to be stored.
    • The drawback is that it necessitates solving a nonlinear algebraic equation at every time step, which can be costly.

    • Operator Splitting - Splitting the Equations

      Principle of operator splitting: Split the complete equation (dy/dt = T + R) into its non-local (Transport, T) and local (Reaction, R) components and solve them separately.

      • The Reaction term (R) is local (only depends on local conditions) and stiff.
      • The Transport term (T) is non-local (depends on values elsewhere in the grid) and generally less stiff, usually linear or almost linear.
      • Splitting enables utilization of special solvers tuned for each portion (stiff solvers for R, convection-diffusion solvers for T).
      • The chemistry step can be readily parallelized since the ODE solution is local to every mesh point.

      Operator Splitting - Strang Splitting Method

      One straightforward splitting procedure: Solve dy/dt = T for one time step, and then dy/dt = R for the same time step. This is not very accurate and necessitates incredibly small time steps.

      A more sophisticated and well-used method is Strang splitting, introduced by Gil Strang in 1968.

      Strang splitting recipe:

      1. Find dy/dt = T for a time step of delta t / 2.
      2. Find dy/dt = R for a time step of delta t from the result in step 1.
      3. Find dy/dt = T for another time step of delta t / 2 from the result in step 2.

      This technique converges to the correct coupled solution with second-order accuracy. Halving the time step improves accuracy by a factor of four.

      Strang splitting is very popular because it is amenable to applying specialized solvers and can be easily parallelized.

      Operator Splitting - Issues with Strang Splitting for Steady State

      • One major issue was encountered when applying Strang splitting to compute steady state solutions (e.g., flame locations).
      • Depending on the selected time step (delta t), Strang splitting may generate qualitatively distinct steady state solutions (e.g., anchored vs. lifted flames). This is a critical problem for combustion scientists.
      • Convergence to steady state is unreliable using Strang splitting.
      • The convergence is not continuous or smooth; the solution may jump abruptly between distinct steady states as delta t decreases. This makes convergence judgment tricky and causes reluctance to publish results.

      Operator Splitting - Balancing Methods

      • There were attempts to enhance the steady state convergence of Strang splitting.
      • A clever post-doc, Ray Speth, created balancing methods.
      • The general concept is to add and subtract an artfully selected constant (C) to the T and R equations ((dy/dt = (T+C) + (R-C)), without changing the original equation sum.
      • The balancing method of the simple variety selects C as the average of the right-hand side (T+R) at the last time step.
      • As the solution converges to steady state, this average value tends to zero.
      • Whereas Strang splitting uses large excursions from the steady state trajectory at each step, expecting them to cancel out.
      • Balancing techniques ensure the steps remain much closer to the steady state trajectory, basically the steady state remains close to convergence.
      • Simple balancing demonstrates exponential convergence towards the steady state; errors decrease significantly as the solution approaches.
      • One disadvantage of simple balancing is it may be numerically less stable as it is explicitly based on the previous time step.
      • Ray Speth created an implicit version and higher-order methods known as rebalance splitting. The formulas are intricate.
      • The implicit rebalance splitting is as stable as Strang splitting but maintains the benefit of exponential convergence near steady state. This is in contrast to the fast convergence of Newton's method.