Research Papers

Assessment Criteria for Computational Fluid Dynamics Model Validation Experiments

[+] Author and Article Information
William L. Oberkampf

W. L. Oberkampf Consulting,
5112 Hidden Springs Trail,
Georgetown, TX 78633
e-mail: wloconsulting@gmail.com

Barton L. Smith

Department of Mechanical
and Aerospace Engineering,
Utah State University,
Logan, UT 84322
e-mail: bsmith@engineering.usu.edu

1Corresponding author.

An earlier version of this article was presented at the AIAA Science and Technology Forum and Exposition, National Harbor, MD, Jan. 13–17, 2014, as Paper No. AIAA-2014-0205.Manuscript received March 12, 2017; final manuscript received August 31, 2017; published online October 11, 2017. Assoc. Editor: Yassin A. Hassan.

J. Verif. Valid. Uncert 2(3), 031002 (Oct 11, 2017) (14 pages) Paper No: VVUQ-17-1013; doi: 10.1115/1.4037887 History: Received March 12, 2017; Revised August 31, 2017

Validation assesses the accuracy of a mathematical model by comparing simulation results to experimentally measured quantities of interest. Model validation experiments emphasize obtaining detailed information on all input data needed by the mathematical model, in addition to measuring the system response quantities (SRQs) so that the predictive accuracy of the model can be critically determined. This article proposes a framework for assessing model validation experiments for computational fluid dynamics (CFD) regarding information content, data completeness, and uncertainty quantification (UQ). This framework combines two previously published concepts: the strong-sense model validation experiments and the modeling maturity assessment procedure referred to as the predictive capability maturity method (PCMM). The model validation experiment assessment requirements are captured in a table of six attributes: experimental facility, analog instrumentation and signal processing, boundary and initial conditions, fluid and material properties, test conditions, and measurement of system responses, with four levels of information completeness for each attribute. The specifics of this table are constructed for a generic wind tunnel experiment. Each attribute’s completeness is measured from the perspective of the level of detail needed for input data using direct numerical simulation of the Navier–Stokes equations. While this is an extraordinary and unprecedented requirement for level of detail in a model validation experiment, it is appropriate for critical assessment of modern CFD simulations.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.


Oberkampf, W. L. , and Aeschliman, D. P. , 1992, “ Joint Computational/Experimental Aerodynamics Research on a Hypersonic Vehicle—Part 1: Experimental Results,” AIAA J., 30(8), pp. 2000–2009. [CrossRef]
Marvin, J. G. , 1995, “ Perspective on Computational Fluid Dynamics Validation,” AIAA J., 33(10), pp. 1778–1787. [CrossRef]
Aeschliman, D. P. , and Oberkampf, W. L. , 1998, “ Experimental Methodology for Computational Fluid Dynamics Code Validation,” AIAA J., 36(5), pp. 733–741. [CrossRef]
Oberkampf, W. L. , Trucano, T. G. , and Hirsch, C. , 2004, “ Verification, Validation, and Predictive Capability in Computational Engineering and Physics,” ASME Appl. Mech. Rev., 57(5), pp. 345–384. [CrossRef]
Oberkampf, W. L. , Pilch, M. , and Trucano, T. G. , 2007, “ Predictive Capability Maturity Model for Computational Modeling and Simulation,” Sandia National Laboratories, Albuquerque, NM, Report No. SAND2007-5948.
Oberkampf, W. L. , and Roy, C. J. , 2010, Verification and Validation in Scientific Computing, Cambridge University Press, Cambridge, UK. [CrossRef]
Oberkampf, W. L. , and Trucano, T. G. , 2002, “ Verification and Validation in Computational Fluid Dynamics,” Prog. Aerosp. Sci., 38(3), pp. 209–272. [CrossRef]
Trucano, T. G. , Pilch, M. , and Oberkampf, W. L. , 2002, “ General Concepts for Experimental Validation of ASCI Code Applications,” Sandia National Laboratories, Albuquerque, NM, Report No. SAND2002-0341.
NEA, 1977, “ Loss of Coolant Accident Standard Problems,” Nuclear Energy Agency, Committee on the Safety of Nuclear Installations, Paris, France.
CSNI, 2004, “ CSNI International Standard Problem Procedures,” Nuclear Energy Agency, Committee on the Safety of Nuclear Installations, Paris, France, Revision 4, CSNI Report No. 17.
NPARC, 2000, “ CFD Verification and Validation: NPARC Alliance,” NASA Glenn Research Center, Cleveland, OH, accessed Jan. 15, 2017, http://www.grc.nasa.gov/WWW/wind/valid/homepage.html
ERCOFTAC, 2017, “ European Research Community on Flow, Turbulence and Combustion,” European Research Community on Flow, Turbulence and Combustion, London, accessed Jan. 15, 2017, http://www.ercoftac.org/
QNET-CFD, 2003, “ The ERCOFTAC QNET-CFD Wiki Home Page,” European Union Network on Quality and Trust in the Industrial Application of Computational Fluid Dynamics, accessed Sept. 1, 2003, http://qnet-ercoftac.cfms.org.uk/w/index.php/Main_Page
Rizzi, A. , and Vos, J. , 1998, “ Toward Establishing Credibility in Computational Fluid Dynamics Simulations,” AIAA J., 36(5), pp. 668–675. [CrossRef]
Vos, J. B. , Rizzi, A. , Darracq, D. , and Hirschel, E. H. , 2002, “ Navier-Stokes Solvers in European Aircraft Design,” Prog. Aerosp. Sci., 38(8), pp. 601–697. [CrossRef]
Oberkampf, W. L. , Aeschliman, D. P. , Tate, R. E. , and Henfling, J. F. , 1993, “ Experimental Aerodynamics Research on a Hypersonic Vehicle,” Sandia National Laboratories, Albuquerque, NM, Report No. SAND-92-1411.
Cosner, R. R. , 1995, “ CFD Validation Requirements for Technology Transition,” AIAA Paper No. 95-2227.
Oberkampf, W. L. , Aeschliman, D. P. , Henfling, J. F. , and Larson, D. E. , 1995, “ Surface Pressure Measurements for CFD Code Validation in Hypersonic Flow,” AIAA Paper No. 95-2273.
Porter, J. L. , 1996, “ A Summary Overview of Selected Computational Fluid Dynamics (CFD) Code Validation/Calibration Activities,” AIAA Paper No. 96-2053.
Barber, T. J. , 1998, “ Role of Code Validation and Certification in the Design Environment,” AIAA J., 36(5), pp. 752–758. [CrossRef]
Benek, J. A. , Kraft, E. M. , and Lauer, R. F. , 1998, “ Validation Issues for Engine—Airframe Integration,” AIAA J., 36(5), pp. 759–764. [CrossRef]
Roache, P. J. , 1998, Verification and Validation in Computational Science and Engineering, Hermosa Publishers, Albuquerque, NM.
NASA, 2008, “ Standard for Models and Simulations,” National Aeronautics and Space Administration, Washington, DC, Standard No. NASA-STD-7009A.
Kaizer, J. S. , Heller, A. K. , and Oberkampf, W. L. , 2015, “ Scientific Computer Simulation Review,” Reliab. Eng. Syst. Saf., 138, pp. 210–218. [CrossRef]
Ferson, S. , and Ginzburg, L. R. , 1996, “ Different Methods are Needed to Propagate Ignorance and Variability,” Reliab. Eng. Syst. Saf., 54(2–3), pp. 133–144. [CrossRef]
Ferson, S. , Kreinovich, V. , Ginzburg, L. , Myers, D. S. , and Sentz, K. , 2002, “ Constructing Probability Boxes and Dempster-Shafer Structures,” Sandia National Laboratories, Albuquerque, NM, Report No. SAND2002-4015.
ANSI, 1997, “ U.S. Guide to the Expression of Uncertainty in Measurement,” American National Standards Institute, Boulder, CO.
AIAA, 1999, “ Assessment of Experimental Uncertainty With Application to Wind Tunnel Testing,” American Institute of Aeronautics and Astronautics, Reston, VA, Standard No. AIAA S-071A-1999.
Beckwith, T. G. , Marangoni, R. D. , and Lienhard, J. H. , 2006, Mechanical Measurements, Prentice Hall, Upper Saddle River, NJ.
Coleman, H. W. , and Steele, W. G., Jr. , 2009, Experimentation, Validation, and Uncertainty Analysis for Engineers, Wiley, New York. [CrossRef]
Wu, C. F. J. , and Hamada, M. S. , 2009, Experiments: Planning, Analysis, and Optimization, Wiley, Hoboken, NJ.
Rhode, M. N. , and Oberkampf, W. L. , 2017, “ Estimation of Uncertainties for a Model Validation Experiment in a Wind Tunnel,” J. Spacecr. Rockets, 54(1), pp. 155–168. [CrossRef]
Montgomery, D. C. , 2001, Design and Analysis of Experiments, Wiley, Hoboken, NJ.
Box, G. E. P. , Hunter, J. S. , and Hunter, W. G. , 2005, Statistics for Experimenters: Design, Innovation, and Discovery, Wiley, New York.
Hinkelmann, K. , and Kempthorne, O. , 2008, Design and Analysis of Experiments (Introduction to Experimental Design), Vol. 1, Wiley, Hoboken, NJ.
Salicone, S. , 2007, Measurement Uncertainty: An Approach Via the Mathematical Theory of Evidence, Springer, Berlin.
Drosg, M. , 2009, Dealing With Uncertainties: A Guide to Error Analysis, Springer, Berlin. [CrossRef]
Grabe, M. , 2014, Measurement Uncertainties in Science and Technology, Springer, Berlin. [CrossRef]


Grahic Jump Location
Fig. 1

Sequence of measurement signals, processing, and recording




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In