Our increased dependence on complex models for engineering design, coupled with our decreased dependence on experimental observation, leads to the question: How does one know that a model is valid? As models become more complex (i.e., multiphysics models), the ability to test models over the full range of possible applications becomes more difficult. This difficulty is compounded by the uncertainty that is invariably present in the experimental data used to test the model; the uncertainties in the parameters that are incorporated into the model; and the uncertainties in the model structure itself. Here, the issues associated with model validation are discussed and methodology is presented to incorporate measurement and model parameter uncertainty in a metric for model validation through a weighted $r2$ norm. The methodology is based on first-order sensitivity analysis coupled with the use of statistical models for uncertainty. The result of this methodology is compared to results obtained from the more computationally expensive Monte Carlo method. The methodology was demonstrated for the nonlinear Burgers’ equation, the convective-dispersive equation, and for conduction heat transfer with contact resistance. Simulated experimental data was used for the first two cases, and true experimental data was used for the third. The results from the sensitivity analysis approach compared well with those for the Monte Carlo method. The results show that the metric presented can discriminate between valid and invalid models. The metric has the advantage that it can be applied to multivariate, correlated data.

1.
ISI
2.
Oberkampf
,
W. L.
, and
Trucano
,
T. G.
, 2002, “
Verification and Validation in Computational Fluid Dynamics
,”
Prog. Aerosp. Sci.
0376-0421
38
(
3
), pp.
209
272
.
3.
Oberkampf
,
W. L.
,
Trucano
,
T. G.
, and
Hirsch
,
C.
, 2004, “
Verification, Validation, and Predictive Capability in Computational Engineering and Physics
,”
Appl. Mech. Rev.
0003-6900
57
(
3
), pp.
345
384
.
4.
Hills
,
R. G.
, and
Trucano
,
T. G.
, 1999, “
Statistical Validation of Engineering and Scientific Models: Background
,” SAND99-1256,
Sandia National Laboratories
, Albuquerque.
5.
Dowding
,
K. J.
,
Hills
,
R. G.
,
Leslie
,
I.
,
Pilch
,
M.
,
Rutherford
,
B. M.
, and
Hobbs
,
M. L.
, 2004, “
Case Study for Model Validation: Assessing a Model for Thermal Decomposition of Polyurethane Foam
,” SAND2004-3632,
Sandia National Laboratories
, Albuquerque.
6.
Easterling
,
R. C.
, 2003, “
Statistical Foundations for Model Validation: Two Papers
,” SAND2003-0287,
Sandia National Laboratories
, Albuquerque.
7.
Rutherford
,
B. M.
, and
Dowding
,
K. J.
, 2003, “
An Approach to Model Validation and Model-Based Prediction—Polyurethane Foam Case Study
,” SAND2003-2336,
Sandia National Laboratories
, Albuquerque.
8.
Hills
,
R. G.
, and
Trucano
,
T. G.
, 2001, “
Statistical Validation of Engineering and Scientific Models with Application to CTH
,” SAND2001-0312,
Sandia National Laboratories
, Albuquerque.
9.
Hills
,
R. G.
, and
Trucano
,
T. G.
, 2002, “
Statistical Validation of Engineering and Scientific Models: A Maximum Likelihood Based Metric
,” SAND2001-1783,
Sandia National Laboratories
, Albuquerque.
10.
Hills
,
R. G.
, and
Leslie
,
I. H.
, 2003, “
Statistical Validation of Engineering and Scientific Models: Validation Experiments to Application
,” SAND2003-0706,
Sandia National Laboratories
, Albuquerque.
11.
Hills
,
R. G.
,
Leslie
,
I. H.
, and
Dowding
,
K.
, 2004, “
Statistical Validation of Engineering and Scientific Models: Application to the Abnormal Environment
,” SAND2004-1029,
Sandia National Laboratories
, Albuquerque, March.
12.
Trucano
,
T. G.
,
Easterling
,
R. G.
,
Dowding
,
K. J.
,
Paez
,
T. L.
,
Urbina
,
A.
,
Romero
,
V. J.
,
Rutherford
,
B. M.
, and
Hills
,
R. G.
, 2001, “
Description of the Sandia Validation Metrics Project
,” SAND2001-1339,
Sandia National Laboratories
, Albuquerque.
13.
Trucano
,
T. G.
,
Pilch
,
M.
, and
Oberkampf
,
W. L.
, 2002, “
General Concepts for Experimental Validation of ASCI Applications
,” SAND2002-0341,
Sandia National Laboratories
, Albuquerque.
14.
Warren-Hicks
,
W.
,
Carbone
,
J. P.
, and
Havens
,
P. L.
, 2002, “
Using Monte Carlo Techniques to Judge Model Prediction Accuracy: Validation of the Pesticide Root Zone Model 3.12
,”
Envir. Toxicol. Chem.
0730-7268
21
(
8
), pp.
1570
1577
.
15.
Roberson
,
S.
, 1999, “
Simulation Verification, Validation and Confidence: A Tutorial
,”
Trans. Soc. Comput. Simul. Int.
0740-6797
16
(
2
), pp.
63
69
.
16.
Brownlee
,
K. A.
, 1965,
Statistical Theory and Methodology in Science and Engineering
,
John Wiley & Sons
, New York.
17.
Strang
,
G.
, 1986,
Introduction to Applied Mathematics
,
Wellesley-Cambridge Press
, Massachusetts.
18.
Hills
,
R. G.
,
Fisher
,
K. A.
,
Kirkland
,
M. R.
, and
Wierenga
,
P. J.
, 1994, “
Application of Flux-Corrected Transport to the Las Cruces Trench Site
,”
Water Resour. Res.
0043-1397
30
(
8
), pp.
2377
2385
.
19.
Roe
,
P. L.
, 1985, “
Some Contributions to the Modelling of Discontinuous Flows
,”
Lect. Appl. Math.
0075-8485
22
, pp.
163
193
.
20.
Roe
,
P. L.
, 1986, “
Characteristic-Based Schemes for the Euler Equations
,”
Annu. Rev. Fluid Mech.
0066-4189
18
, pp.
337
365
.
21.
Sweby
,
P. K.
, 1984, “
High Resolution Schemes using Flux Limiters for Hyperbolic Conservation Laws
,”
SIAM (Soc. Ind. Appl. Math.) J. Numer. Anal.
0036-1429 ,
21
(
5
), pp.
995
1011
.
22.
Yang
,
H. Q.
, and
Przekwas
,
A. J.
, 1992, “
A Comparative Study of Advanced Shock-Capturing Schemes Applied to Burgers’ Equation
,”
J. Comput. Phys.
0021-9991
102
, pp.
139
159
.
23.
MatLab® Mathematics (2004), Version 7, The MathWorks, Inc., Natick, MA.
24.
IMSL
, 1997, “
IMSL, Math Library, Vols. 1 and 2
,” Visual Numerics, Inc., Houston.
25.
Blackwell
,
B. F.
,
Gill
,
W.
,
Dowding
,
K. J.
, and
Easterling
,
R. G.
, 2000, “
Uncertainty Estimation in the Determination of Thermal conductivity of 304 Stainless Steel
,”
Proceedings of the International Mechanical Engineering Congress
,
ASME
,
Orlando, FL
.
26.
Emery
,
A. F.
,
Blackwell
,
B. F.
, and
Dowding
,
K. J.
, 2001, “
The Relationship Between Information, Sampling Rates, and Parameter Estimation Models
,”
Proceedings of the 35th National Heat Transfer Conference
, June,
Anaheim, CA
.
27.
Voth
,
T. E.
, and
Gill
,
W.
, 1999, “
Assessment of Sub-Grid Contact Resistance Physics Effects on Thermal Simulation
,” Sandia National Laboratories technical memo.
28.
Dowding
,
K. J.
, 2000, personal communication, Sandia National Laboratories, Albuquerque.
29.
Leonard
,
T.
, and
Hsu
,
J. S. J.
, 1999,
Bayesian Methods
,
Cambridge University Press
, Cambridge.
30.
Sharer
,
G.
, 1976,
A Mathematical Theory of Evidence
,
Princeton University Press
, Princeton, NJ.
31.
Oberkampf
,
W. L.
, and
Helton
,
J. C.
, 2002, “
Investigation of Evidence Theory for Engineering Applications
,” AIAA 2002-1569, 4th Non-Deterministic Approaches Forum, American Institute of Aeronautics and Astronautics, April 22–25, Denver, Colorado.